Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jul 1.
Published in final edited form as: Nurs Outlook. 2021 Feb 13;69(4):609–616. doi: 10.1016/j.outlook.2021.01.010

Methodology for a Six-State Survey of Primary Care Nurse Practitioners

Jordan M Harrison a, Hayley D Germack b, Lusine Poghosyan c, Thomas D’Aunno d, Grant R Martsolf e,f
PMCID: PMC8362822  NIHMSID: NIHMS1664910  PMID: 33593667

Abstract

Primary care practices employing nurse practitioners (NPs) can play an important role in improving access to high quality health care services. However, most studies on the NP role in health care use administrative data, which have many limitations. To overcome the limitations of administrative data, we fielded a cross-sectional survey of primary care NPs in six states to collect data directly from NPs on their clinical roles, practice environments, job outcomes, and the structural capabilities available to support their practice. Here we provide an overview of our survey methods, including a description of the sampling frame, procedures for data collection, and the non-response analysis. We also describe the challenges we encountered in surveying a representative sample of primary care NPs and develop recommendations to enhance survey methodologies in future large-scale NP surveys.

Keywords: nurse practitioners, primary care, surveys, workforce

Introduction

Over the past ten years, the nurse practitioner (NP) workforce has grown dramatically across all regions of the U.S. (Auerbach, Buerhaus, & Staiger, 2020). With increasing demand for health care services and expansion of NP degree programs, the number of full-time NPs more than doubled between 2010 and 2017, from 91,000 to 190,000 (Auerbach et al., 2020). The NP workforce will be especially important as aging and population growth lead to increased demand for primary care, while primary care physicians retire faster than new graduates can replace them (Dall, Reynolds, Jones, Chakrabarti, & Iacobucci, 2019; National Center for Health Workforce Analysis, 2016; UnitedHealth Group, 2018). Approximately 90% of NPs are certified in an area of primary care, and an estimated 69% of all NPs deliver primary care (American Association of Nurse Practitioners, 2020). The primary care NP workforce is well positioned to meet demand for health care services and expand access to high quality care for underserved patients. Primary care NPs are more likely than primary care physicians to practice in rural, underserved, and socioeconomically disadvantaged areas (Buerhaus, DesRoches, Dittus, & Donelan, 2015; Davis et al., 2018; Xue, Smith, & Spetz, 2019). Numerous studies and systematic reviews demonstrate that NP care is associated with improved chronic disease outcomes (Buerhaus et al., 2018; Litaker et al., 2003; Muench, Guo, Thomas, & Perloff, 2019; Mundinger et al., 2000; Newhouse et al., 2011), and the National Academy of Medicine (formerly the Institute of Medicine) has recommended greater use of NPs as a strategy to address health disparities (Institute of Medicine, 2011). As of 2016, NPs represented one in four providers in primary care practices nationally (Barnes, Richards, McHugh, & Martsolf, 2018).

Research on primary care NPs is critically important to produce evidence about this fast growing workforce. However, research on NPs often relies on administrative data sources such as the National Provider and Plan Enumeration System (NPPES), Medicare billing data, and licensure and certification data (Barnes et al., 2017; Barnes & Novosel, 2018). While these data sources have been used to study the NP workforce, care, and practice in many studies and have yielded important evidence about the NP workforce, administrative data sources have many limitations, including lack of consistent and comprehensive data collection across states, out-of-date information, and lack of individual NP identifiers to enable linkage to other data sources (Barnes & Novosel, 2018; Kaplan, Skillman, Fordyce, McMenamin, & Doescher, 2012). For example, studies that rely on billing data may underestimate the proportion of care delivered by NPs, as NPs often may not bill directly (Barnes et al., 2017; Kaplan et al., 2012). Surveys of NPs are a cost-effective method to better understand various aspects of NP practice and can address the limitations of administrative data (Klabunde et al., 2012). However, researchers face many challenges in identification of a representative sample of clinicians and achievement of adequate survey response rate (DesRoches et al., 2015; DiGaetano, 2013; Klabunde et al., 2012).

Previous surveys of primary care NPs have examined NP practice patterns, clinical roles, and practice environments (Buerhaus et al., 2015; Donelan et al., 2019; Donelan, DesRoches, Dittus, & Buerhaus, 2013; Poghosyan, Norful, Liu, & Friedberg, 2018; Poghosyan et al., 2015; Poghosyan, Liu, Shang, & D’Aunno, 2017). Additional work is needed to understand how these factors impact patient outcomes in primary care practices employing NPs. To address this gap in evidence, we fielded a cross-sectional survey of primary care NPs in six states to collect data on their clinical roles, practice environments, job outcomes, and the structural capabilities available to support their practice. In this paper, we provide an overview of our survey methods, including a description of the sampling frame, procedures for data collection, and the non-response analysis. We also describe the challenges we encountered in surveying a representative sample of primary care NPs and develop recommendations to enhance survey methods in future NP surveys. Specifically, we address strategies to improve the accuracy of NP sampling frames and increase survey response rates.

Methods

Sampling Frame

We identified primary care NPs using IQVIA OneKey, a health care industry database that collects data on health care providers and practices across the U.S. (IQVIA Inc., 2020). OneKey integrates data from IMS Health, Healthcare Data Solutions, and SK&A. We used facility and physician specialty data in OneKey to identify NPs working in primary care practices, defined as practices in which more than half of physicians had individual specialties of family practice, general practice, geriatrics, internal medicine, preventive medicine, or pediatrics. NPs within these primary care practices were considered eligible for inclusion in the sample.

We sampled eligible primary care NPs from six states with varying NP scope of practice (SOP) regulations at the time of the survey, as defined by the American Association of Nurse Practitioners (2019): Arizona and Washington (full SOP), New Jersey and Maryland (reduced SOP), and California and Texas (restricted SOP). To ensure similar counts of NPs by state, we sampled all eligible NPs in Arizona, New Jersey, and Washington; a 50% random sample in California and Texas; and a 75% random sample in Pennsylvania. For each NP sampled, we obtained name, gender, National Provider Identifier (NPI), contact information (i.e., practice name, mailing address, and phone number), and type of practice organization (medical group or independent physician practice). The resulting sampling frame included 10,237 NPs in the six states. OneKey data was first delivered in September 2018, and updated data for the sampled NPs was delivered in February 2019 in order to have the most up-to-date contact information.

Survey

The survey inquired about characteristics of the NPs and the practices in which they worked. We asked NPs about their demographics, education, licensure, and certification. We also asked NPs to identify their main practice setting (e.g., physician practice, community health center, hospital based clinic, etc.), years employed in their current primary position, and average hours worked per week in their primary position. Survey domains included patient panels (independent patient panel – yes/no), practice patterns (hours per week spent performing various tasks), provider mix (number and type of other providers in the practice), NP practice environment measured using the Nurse Practitioner-Primary Care Organizational Climate Questionnaire (NP-PCOCQ) (Poghosyan, Nannini, Finkelstein, Mason, & Shaffer, 2013) practice structural capabilities (measured using the Structural Capability Index) (Martsolf, Ashwood, Friedberg, & Rodriguez, 2018) and job outcomes (job satisfaction, perceived quality of care in their practice, burnout, and intent to leave).

Data Collection Procedures

Mailed Survey

The Survey Research Institute (SRI) at Cornell University administered the survey between November 2018 and October 2019. In November of 2018, we sent a cover letter to all 10,237 NPs in the sampling frame describing the purpose of the study and its voluntary nature, a paper survey, an online link for completing the survey, and a unique individual identifier. NPs could complete the survey on paper or online. After completing the survey, respondents could participate in a lottery drawing for one of 250 $50 gift cards. This study was approved by the Institutional Review Boards of Columbia University Medical Center and the University of Pittsburgh.

We employed a Dillman approach for mixed-mode surveys to maximize the response rate (Dillman, Smyth, & Christian, 2014). Two weeks after the initial survey mailing, we sent postcard reminders to non-respondents. Two weeks after the postcard reminder, we sent a second survey mailing to non-respondents. We sent a second postcard reminder two weeks after the second survey mailing. Before the third mailing, the research team obtained updated contact information for the sampled NPs from OneKey. In total, we distributed three surveys and two postcard reminders to NPs.

Phone Follow-Up

In the Spring and Summer of 2019, we called all NPs that had not yet responded to the survey. We called all non-respondents to confirm that they had received the survey. If they had received the survey, we encouraged them to fill it out. If they had not received the survey or no longer had a copy, we offered to resend the survey via mail or email. During the follow-up calls, we also confirmed with either the NP or a practice representative that the address we had on file was valid, the practice specialty was primary care, and the NP still worked at the practice. We made three attempts to contact each non-respondent by phone. At the end of the data collection period, we assigned each NP in the sampling frame a final disposition (“eligible non-respondent,” or “ineligible” and reason for ineligibility, if applicable). If the non-respondent or a practice representative could not be reached after three phone attempts, they were listed as “non-respondent with unknown disposition.”

Findings

Survey Response Rate

Final disposition of the original sample of 10,237 NPs is reported in Table 1. A total of 1,244 NPs in 1,109 practices responded to the survey, including 954 surveys completed by mail (76.7%) and 290 completed online (23.3%). Of those that did not respond to the survey, 2,742 non-respondents were confirmed as eligible in phone follow-up (eligible non-respondents); 4,548 non-respondents were confirmed as ineligible in phone follow-up (ineligible non-respondents); and the remaining 1,703 non-respondents could not be reached by phone (non-respondents with unknown disposition).

Table 1.

Final Disposition of Nurse Practitioner Sample

Final Disposition N (%)
Survey respondents 1,244 (12.2%)
Eligible non-respondents 2,742 (26.8%)
Ineligible non-respondents 4,548 (44.4%)
Non-respondents with unknown disposition 1,703 (16.6%)
Total 10,237
Open in a new tab

Reasons for NP ineligibility are reported in Table 2. In total, 4,548 NPs (44.4% of our original sampling frame) were confirmed as ineligible and were excluded from our response rate calculations. The most common reason for ineligibility was that the NP no longer worked at the practice (34.7% of ineligible NPs). Other reasons for ineligibility were bad mailing address, the NP was not currently working in primary care, the NP had never worked at the practice, or the individual was not a NP.

Table 2.

Reasons for Nurse Practitioner Ineligibility

Reason N (%)
No longer works at the practice 1,577 (34.7%)
Bad address 1,163 (25.6%)
Not in primary care 1,029 (22.6%)
Never worked at the practice 760 (16.7%)
Not a nurse practitioner 19 (0.4%)
Total Known Ineligible 4,548
Open in a new tab

Given that 1,703 non-respondents could not be reached to confirm eligibility, we calculated the final survey response rate under three scenarios, ranging from most conservative to least conservative: 1) assuming all non-respondents with unknown disposition were eligible for inclusion in the survey; 2) estimating the proportion of non-respondents with unknown disposition that were likely to be eligible; and 3) assuming that all non-respondents were ineligible. For Scenario 2, we used methods endorsed by the American Association for Public Opinion Research (AAPOR, 2016) to estimate the proportion of non-respondents with unknown disposition that were eligible. We applied the eligibility rates from those in the sampling frame whose final disposition was known – we assumed that 53% of those that we could not reach were ineligible for the survey. We used this assumption because of those with a known disposition (n=8,534), approximately 53% (n=4,548) were known to be ineligible for the survey. We have used similar assumptions in previous response rate estimates (Martsolf, Schofield, Johnson, & Scanlon, 2013).

Survey response rates under the three scenarios are reported in Table 3. The final response rate was 21.9% under the assumptions of Scenario 1 (all unknowns eligible), 25.4% under Scenario 2 (some unknowns eligible), and 31.2% under scenario 3 (all unknowns ineligible).

Table 3.

Survey Response Rate

Scenario 1: Assuming all unknown are eligible Scenario 2: Assuming some unknown are ineligible* Scenario 3: Assuming all unknown are ineligible
Respondents 1,244 1,244 1,244
Assumed eligible non-respondents 4,445 3,650 2,742
Total assumed eligible 5,689 4,894 3,986
Response rate 21.9% 25.4% 31.2%
Open in a new tab
*

In Scenario 2, we assume that 53% of the 1,703 non-respondents with unknown disposition are ineligible for the survey. This estimate is based on eligibility rates among those in the sampling frame whose final disposition was known.

Non-Response Analysis

To evaluate the potential for non-response bias, we conducted a non-response analysis to assess whether survey respondents differed from non-respondents in important ways, including NP, practice, and community characteristics. We obtained NP and practice characteristics from OneKey, including NP gender, state, ZIP code (used to derive urban/rural practice setting), practice organization (medical group or independent physician practice), and the number of NPs in the practice. From the American Community Survey’s 2017 data, we obtained characteristics of the communities in which NPs practiced, including median household income, percent below poverty, percent with public insurance coverage, percent uninsured, and unemployment rate.

We compared NP, practice, and community characteristics across three groups: 1) survey respondents, 2) eligible non-respondents, and 3) non-respondents with unknown disposition. We used Chi-square tests (categorical variables) and one-way ANOVA tests (continuous variables) to identify significant differences in characteristics across the three groups. We performed analyses in Stata (Version 16.1; StataCorp, College Station, TX). For all analyses we used two-sided tests with alpha level of 0.05.

Characteristics of the survey respondents, eligible non-respondents, and non-respondents with unknown disposition are reported in Table 4. Gender distribution was similar across the three groups (87.2–89.0% female). Survey respondents were slightly more likely to practice in rural settings and more likely to be the only NP in their practice compared to the other groups. The distribution of NPs by state, and accordingly, state scope of practice, differed across the three groups. A greater share of non-respondents with unknown disposition practiced in states with full SOP (Arizona and Washington). There were also some statistically significant differences across the three groups in characteristics of the communities where NPs practiced, including the distribution of median household income, as well as the percent of residents with public insurance coverage or no insurance coverage. Non-respondents with unknown disposition practiced in communities with slightly higher poverty rates.

Table 4.

Characteristics of Survey Respondents and Non-Respondents

Variable Survey respondents
(N=1,244)		 Eligible non-respondents
(N=2,742)		 Non-respondents, disposition unknown
(N=1,703)		 P
NP and Practice Characteristics
Gender(%) 0.11
Female 87.2% 89.5% 89.0%
Male 12.8% 10.3% 11.0%
Unknown 0 0.2% 0
State (%) <0.001
AZ 10.7% 11.6% 15.8%
CA 23.6% 27.2% 19.0%
NJ 8.8% 10.5% 9.9%
PA 23.1% 18.5% 6.3%
TX 18.6% 20.5% 26.7%
WA 15.3% 11.8% 22.3%
Scope of practice (%) <0.001
Full 26.0% 23.4% 38.1%
Reduced 31.8% 28.9% 16.3%
Restricted 42.2% 47.7% 45.7%
Practice setting (%) <0.01
Urban 86.0% 87.4% 89.8%
Rural 14.0% 12.6% 10.2%
Practice organization (%) 0.86
Medical group 97.6% 97.8% 97.6%
Independent physician practice 2.4% 2.2% 2.4%
Number of NPs in the practice (%)
1 56.3% 50.4% 45.8% <0.001
2 24.2% 25.3% 23.5%
3–4 14.4% 15.6% 17.0%
5+ 5.1% 8.8% 13.7%
Community Characteristics
Median household income (%)
<$45,000 23.0% 28.8% 25.7% <0.001
$45,000-$65,000 35.6% 33.6% 37.3%
>$65,000 40.7% 37.1% 36.0%
Unknown 0.7% 0.4% 1.0%
Percent below poverty (mean, SD) 14.7 (9.2) 14.1 (8.7) 15.3 (9.4) <0.001
Percent with public insurance coverage (mean, SD) 10.8 (6.8) 10.9 (6.8) 12.1 (7.6) 0.02
Percent uninsured (mean, SD) 34.8 (11.1) 33.8 (11.0) 34.5 (11.9) <0.001
Unemployment rate (mean, SD) 6.6 (3.5) 6.4 (3.2) 6.6 (3.7) 0.11
Open in a new tab

Note: P-values are for the 3-way comparison across groups using Chi-square tests (categorical variables) or one-way ANOVA tests (continuous variables).

Discussion and Recommendations

In this paper, we report the methods of the largest survey of primary care NPs to date, conducted in six states with varying SOP regulations. We fielded a mixed-mode survey (mail and online), distributing a total of three surveys and two postcard reminders to NPs by mail and following up with non-respondents by phone. We found that a response rate between 21.9–31.2% can be achieved by applying our survey methodology. Notably, the majority of surveys were completed by mail (76.7%) rather than online, suggesting that many clinicians prefer to complete paper questionnaires. The survey data collected directly from NPs will allow critical investigations about NP practice in primary care.

While we were able to collect data from over 1,200 NPs, we encountered several challenges with our sampling frame that, if addressed, can significantly enhance response rates in future large scale NP surveys. First, provider turnover created challenges for data collection. Among the NPs excluded from our sample, the most common reason for ineligibility was no longer working at the practice (34.7%). Provider turnover is a common issue in surveys of health care providers, and contact information quickly becomes outdated (DesRoches et al., 2015; DiGaetano, 2013). Researchers have encountered similar issues in verifying address information for primary care physicians sampled from the NPPES, the American Medical Association Masterfile, and the SK&A physician file (DesRoches et al., 2015). In our study, although we obtained the NP data two months prior to the first mailing and then obtained updated data prior to the third mailing and phone follow-up, we still faced challenges with provider turnover.

Second, identification of NP specialty was challenging. We had to rely on the specialties of the physicians in the practice to identify primary care NPs, and this strategy was inaccurate in some cases. Among the NPs excluded from our sample, a common reason for ineligibility was that the NP did not work in primary care, i.e., the practice was not truly primary care or the NP did not provide primary care (22.6%). Additional issues with the sampling frame included a substantial number of bad mailing addresses (25.6%) and NPs who had never worked at the practices to which they were attributed in our database (16.7%), as well as some clinicians who were not NPs (0.4%).

In future surveys, researchers can employ strategies to avoid the issues we encountered with the sampling frame and enhance large scale survey data collection from NPs. The most important lesson learned from our survey is that quality checks are necessary to verify the sample and eliminate ineligible NPs prior to survey data collection. Pre-calls to each NP or a practice representative to confirm eligibility can improve accuracy of the sampling frame. Phone calls are useful to 1) confirm that the address on file is valid, 2) confirm the practice specialty, and 3) confirm that the NP currently works at the practice. Another strategy to enhance survey data collection from NPs is collaboration with a professional organization, which may increase the visibility of large scale surveys. Professional organizations may be willing to assist with identification of a sample of NPs, and they may also be willing to endorse the survey and encourage participation, potentially boosting response rates.

Researchers can also employ various strategies in survey implementation to maximize response rates. We used multiple strategies to encourage survey completion, including monetary incentives, alternating survey mailings and postcard reminders, and phone follow-up with non-respondents. In a review of 117 large scale surveys of health care providers published between 2000 and 2010, surveys that achieved the highest response rates often provided monetary incentives; made initial contact by mail as opposed to email; used a telephone, mail, or mixed-mode approach as opposed to online only; and had more intensive follow-up with non-respondents (McLeod, Klabunde, Willis, & Stark, 2013). Common follow-up strategies to convert non-respondents included extending the field period to provide more time to complete the survey; offering multiple sequential response modes (e.g., online survey after initial mailed survey); switching contact mode to encourage response to a single mode (e.g., telephone follow-up to encourage mailed survey response); and offering incentives to non-respondents at follow-up (McLeod et al., 2013).

Our study had some limitations. Our survey response rate of 21.9–31.2% is comparable to previous large scale nurse surveys (Brooks Carthon et al., 2020; Lasater et al., 2019), yet lower than some clinician surveys reported in the literature (McLeod et al., 2013). The non-response analysis was limited to variables available from OneKey and the American Community Survey, as we were unable to collect additional data from non-respondents to compare all NP characteristics reported in the survey. Survey respondents and non-respondents differed in some ways, including differences in distribution across the six states as well as some differences in characteristics of the communities in which they practiced, but more rigorous analysis is needed to assess non-response bias. However, the final sample of 1,244 NPs shows sufficient variation to characterize NP practice characteristics across different settings (i.e., rural/urban) and by state SOP regulations.

Conclusions

NPs are the fastest growing segment of the primary care workforce and play a critical role in primary care delivery in the U.S. As an alternative to administrative data sources, collection of survey data from NPs can generate important evidence about NP practice. The mixed-mode approach we used in this six-state NP survey is a promising approach for future large scale NP surveys. However, in future surveys researchers should use enhanced methodologies to ensure identification of a representative NP sampling frame and improve survey response rates. In particular, quality checks are important to verify the sample and eliminate ineligible NPs prior to survey data collection. Collaboration with professional organizations may also be an important strategy to increase visibility of future surveys and encourage participation. Finally, various survey implementation strategies can help to achieve higher response rates.

Highlights.

  • A mixed-mode approach (mail/web) was effective for surveying nurse practitioners.

  • Quality checks are important to verify the sample prior to data collection.

  • Collaboration with professional organizations may increase survey visibility.

  • Various strategies in survey implementation can enhance response rates.

Acknowledgments:

NIMHD R01MD011514

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Competing interest statement: The authors declare no conflict of interest.

References

  1. American Association for Public Opinion Research. (2016). Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. Retrieved from https://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf
  2. American Association of Nurse Practitioners. (2019). State Practice Environment.
  3. American Association of Nurse Practitioners. (2020). More Than 290,000 Nurse Practitioners Licensed in the United States. Retrieved fromAANP News website: https://www.aanp.org/news-feed/290-000-nps-licensed-in-us
  4. Auerbach DI, Buerhaus PI, & Staiger DO (2020). Implications Of The Rapid Growth Of The Nurse Practitioner Workforce In The US. Health Affairs, 39(2), 273–279. 10.1377/hlthaff.2019.00686 [DOI] [PubMed] [Google Scholar]
  5. Barnes H, Maier CB, Altares Sarik D, Germack HD, Aiken LH, & McHugh MD (2017). Effects of Regulation and Payment Policies on Nurse Practitioners’ Clinical Practices. Medical Care Research and Review, 74(4), 431–451. 10.1177/1077558716649109 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Barnes H, & Novosel LM (2018). A scoping review of nurse practitioner workforce data: Part two of a four-part series on critical topics identified by the 2015 Research Agenda Roundtable. Journal of the American Association of Nurse Practitioners, 30(12), 685–695. 10.1097/JXX.0000000000000069 [DOI] [PubMed] [Google Scholar]
  7. Barnes H, Richards MR, McHugh MD, & Martsolf G (2018). Rural and nonrural primary care physician practices increasingly rely on nurse practitioners. Health Affairs, 37(6), 908–914. 10.1377/hlthaff.2017.1158 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Brooks Carthon JM, Brom H, Poghosyan L, Daus M, Todd B, & Aiken L (2020). Supportive Clinical Practice Environments Associated With Patient-Centered Care. Journal for Nurse Practitioners, 16(4), 294–298. 10.1016/j.nurpra.2020.01.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Buerhaus PI, DesRoches CM, Dittus R, & Donelan K (2015). Practice characteristics of primary care nurse practitioners and physicians. Nursing Outlook, 63(2), 144–153. 10.1016/j.outlook.2014.08.008 [DOI] [PubMed] [Google Scholar]
  10. Buerhaus P, Perloff J, Clarke S, O’Reilly-Jacob M, Zolotusky G, & DesRoches CM (2018). Quality of Primary Care Provided to Medicare Beneficiaries by Nurse Practitioners and Physicians. Medical Care, 56(6), 484–490. 10.1097/MLR.0000000000000908 [DOI] [PubMed] [Google Scholar]
  11. Dall T, Reynolds R, Jones K, Chakrabarti R, & Iacobucci W (2019). The Complexities of Physician Supply and Demand: Projections from 2017 to 2032. In Association of American Medical Colleges. Washington, D.C. [Google Scholar]
  12. Davis MA, Anthopolos R, Tootoo J, Titler M, Bynum JPW, & Shipman SA (2018). Supply of Healthcare Providers in Relation to County Socioeconomic and Health Status. Journal of General Internal Medicine, 33(4), 412–414. 10.1007/s11606-017-4287-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. DesRoches CM, Barrett KA, Harvey BE, Kogan R, Reschovsky JD, Landon BE, … Rich EC (2015). The results are only as good as the sample: Assessing three national physician sampling frames. Journal of General Internal Medicine, 30(3), 595–601. 10.1007/s11606-015-3380-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. DiGaetano R (2013). Sample Frame and Related Sample Design Issues for Surveys of Physicians and Physician Practices. Evaluation and the Health Professions, 36(3), 296–329. 10.1177/0163278713496566 [DOI] [PubMed] [Google Scholar]
  15. Dillman D, Smyth J, & Christian L (2014). Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method (Vol. 4) Hoboken, NJ: John Wiley & Sons, Inc. [Google Scholar]
  16. Donelan K, Chang Y, Abebe JB, Spetz J, Auerbach DI, Norman L, & Buerhaus PI (2019). Care management for older adults: The roles of nurses, social workers, and physicians. Health Affairs, 38(6), 941–949. 10.1377/hlthaff.2019.00030 [DOI] [PubMed] [Google Scholar]
  17. Donelan K, DesRoches CM, Dittus RS, & Buerhaus P (2013). Perspectives of physicians and nurse practitioners on primary care practice. New England Journal of Medicine, 368, 1898–1906. 10.1056/NEJMsa1212938 [DOI] [PubMed] [Google Scholar]
  18. Institute of Medicine. (2011). The Future of Nursing: Leading Change, Advancing Health. In Washington, DC: The National Academies Press. 10.17226/12956 [DOI] [PubMed] [Google Scholar]
  19. IQVIA Inc. (2020). OneKey Reference Assets. Retrieved from https://www.iqvia.com/locations/united-states/solutions/commercial-operations/essential-information/onekey-reference-assets
  20. Kaplan L, Skillman SM, Fordyce MA, McMenamin PD, & Doescher MP (2012). Understanding APRN distribution in the United States using NPI data. Journal for Nurse Practitioners, 8(8), 626–635. 10.1016/j.nurpra.2012.05.022 [DOI] [Google Scholar]
  21. Klabunde CN, Willis GB, McLeod CC, Dillman DA, Johnson TP, Greene SM, & Brown ML (2012). Improving the Quality of Surveys of Physicians and Medical Groups: A Research Agenda. Evaluation and the Health Professions, 35(4), 477–506. 10.1177/0163278712458283 [DOI] [PubMed] [Google Scholar]
  22. Lasater KB, Jarrín OF, Aiken LH, McHugh MD, Sloane DM, & Smith HL (2019). A Methodology For Studying Organizational Performance. Medical Care, 57(9), 742–749. 10.1097/MLR.0000000000001167 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Litaker D, Mion LC, Planavsky L, Kippes C, Mehta N, & Frolkis J (2003). Physician - Nurse practitioner teams in chronic disease management: The impact on costs, clinical effectiveness, and patients’ perception of care. Journal of Interprofessional Care, 17(3), 223–237. 10.1080/1356182031000122852 [DOI] [PubMed] [Google Scholar]
  24. Martsolf GR, Ashwood S, Friedberg MW, & Rodriguez HP (2018). Linking structural capabilities and workplace climate in community health centers. INQUIRY: The Journal of Health Care Organization, Provision, and Financing, 55, 0046958018794542. 10.1177/0046958018794542 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Martsolf GR, Schofield RE, Johnson DR, & Scanlon DP (2013). Editors and researchers beware: Calculating response rates in random digit dial health surveys. Health Services Research, 48(2pt1), 655–676. 10.1111/j.1475-6773.2012.01464.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. McLeod CC, Klabunde CN, Willis GB, & Stark D (2013). Health Care Provider Surveys in the United States, 2000–2010: A Review. Evaluation and the Health Professions, 36(1), 106–126. 10.1177/0163278712474001 [DOI] [PubMed] [Google Scholar]
  27. Muench U, Guo C, Thomas C, & Perloff J (2019). Medication adherence, costs, and ER visits of nurse practitioner and primary care physician patients: Evidence from three cohorts of Medicare beneficiaries. Health Services Research, 54(1), 187–197. 10.1111/1475-6773.13059 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Mundinger MO, Kane RL, Lenz ER, Totten AM, Tsai WY, Cleary PD, … Shelanski ML (2000). Primary care outcomes in patients treated by nurse practitioners or physicians: A randomized trial. Journal of the American Medical Association, 283(1), 59–68. 10.1001/jama.283.1.59 [DOI] [PubMed] [Google Scholar]
  29. National Center for Health Workforce Analysis. (2016). National and Regional Projections of Supply and Demand for Primary Care Practitioners: 2013–2025. Retrieved from https://bhw.hrsa.gov/sites/default/files/bhw/health-workforce-analysis/research/projections/primary-care-national-projections2013-2025.pdf
  30. Newhouse RP, Stanik-Hutt J, White KM, Johantgen M, Bass EB, Zangaro G, … Weiner JP (2011). Advanced practice nurse outcomes 1990–2008: a systematic review. Nursing Economic$, 29(5), 230–250; quiz 251. 10.1378/chest.126.3.903 [DOI] [PubMed] [Google Scholar]
  31. Poghosyan L, Norful AA, Liu J, & Friedberg MW (2018). Nurse Practitioner Practice Environments in Primary Care and Quality of Care for Chronic Diseases. Medical Care, 56(9), 791–797. 10.1097/MLR.0000000000000961 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Poghosyan L, Shang J, Liu J, Poghosyan H, Liu N, & Berkowitz B (2015). Nurse practitioners as primary care providers: Creating favorable practice environments in New York State and Massachusetts. Health Care Management Review, 40(1), 46–55. 10.1097/HMR.0000000000000010 [DOI] [PubMed] [Google Scholar]
  33. Poghosyan Lusine, Liu J, Shang J, & D’Aunno T (2017). Practice environments and job satisfaction and turnover intentions of nurse practitioners: Implications for primary care workforce capacity. Health Care Management Review, 42(2), 162–171. 10.1097/HMR.0000000000000094 [DOI] [PubMed] [Google Scholar]
  34. Poghosyan Lusine, Nannini A, Finkelstein SR, Mason E, & Shaffer JA (2013). Development and psychometric testing of the nurse practitioner primary care organizational climate questionnaire. Nursing Research, 62(5), 325–334. 10.1097/NNR.0b013e3182a131d2 [DOI] [PubMed] [Google Scholar]
  35. UnitedHealth Group. (2018). Addressing the Nation’s Primary Care Shortage: Advanced Practice Clinicians and Innovative Care Delivery Models. Retrieved from https://www.unitedhealthgroup.com/content/dam/UHG/PDF/2018/UHG-Primary-Care-Report-2018.pdf
  36. Xue Y, Smith JA, & Spetz J (2019). Primary Care Nurse Practitioners and Physicians in Low-Income and Rural Areas, 2010–2016. JAMA, 321(1), 102–105. 10.1001/jama.2018.17944 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES