Use of a Set-up Fee to Encourage Survey Participation and Electronic Health Record Submission for a National Health Care Survey

Abstract

In 2021, the National Ambulatory Medical Care Survey Health Center (NAMCS HC) Component modernized its data collection and began collecting clinical visit data from health centers using transmission of electronic health record (EHR) data. With this redesign, there are potential cost implications for a health center to participate. Beginning in 2021, a one-time set-up fee (i.e., payment) of up to $10,000 was offered to centers who participated in the NAMCS HC Component. Starting in 2022, a Set-up Fee Questionnaire was used to capture data on the use of this money. Results show on average the fee appeared adequate in covering participation costs, with the fee most often used to cover costs of health center and EHR vendor information technology staff. Although this fee was offered only during the initial year of participation, retention rates for centers appeared high. Implications from the findings of this case study on use of a set-up fee in establishment surveys are also briefly discussed.

Introduction

Since 2012, the National Center for Health Statistics (NCHS) has sampled health centers as part of one of its establishment surveys: the National Ambulatory Medical Care Survey (NAMCS). To continue NAMCS’s mission of meeting the need for objective, reliable information about the provision and use of ambulatory medical care services in the United States, the survey underwent a redesign to modernize its data collection, including the movement in 2021 to collection of electronic health record (EHR) data from health centers (Williams et al. 2023). These centers include federally qualified health centers (FQHCs) and FQHC look-alikes (i.e., health centers who meet all requirements of the Health Resources and Services Administration [HRSA] Health Center Program, but do not receive Program funding) (HRSA, 2024). Spawning this modernization were several factors including use of EHR systems by ≥99% of health centers (HRSA 2023), barriers to in-person data collection accelerated by the COVID-19 pandemic (Ward et al. 2021), and declining response rates among physician/provider surveys (Cho, Johnson, and VanGeest 2013).

Decreasing response rates are not unique to NAMCS. Broader literature shows response among health care provider surveys tends to be low and declining (Cho, Johnson, and VanGeest 2013), further exacerbated by the COVID-19 pandemic (Young et al. 2022). To combat this decline, incentives and other monetary and non-monetary benefits have long been used to increase participation and response rates in physician surveys (Delnevo, Abatemarco, and Steinberg 2004), with larger monetary incentives yielding higher rates (Cho, Johnson, and VanGeest 2013; Kellerman and Herold 2001; Young et al. 2022). This is consistent with the broader (but limited) literature on use of incentives in establishment surveys, where a meta-analysis by Torres van Grinsven and Hox (2015) found a small but significant effect of incentives for increasing response in organizational surveys.

Use of incentives for establishment surveys requires not only consideration of what should be used as an incentive (e.g., non-monetary vs. monetary) and when it should be given (e.g., pre- or post-survey), but also to whom at the establishment it should be given. While incentives can motivate individual respondents to answer a survey request (Cook et al. 2009), when used with establishment surveys it is critical to consider characteristics of the establishment (including its available resources). These characteristics play a role in survey cooperation (Willimack, Nichols, and Sudman 2002) and may impact how to effectively use an incentive or similar benefit. Such resources may include employees’ time, knowledge, and skills or expertise, all which could be required for survey participation (Willimack and Snijkers 2013). When an establishment utilizes their resources for participation, these resources may need to be (temporarily) reallocated from routine, non-survey tasks that go towards accomplishing the establishment’s mission. For entrepreneurial businesses this mission entails making a profit through revenue from goods and services while simultaneously minimizing operating costs, while for nonprofit organizations (such as health centers) this would imply the reduction of operating costs (Willimack and Snijkers 2013). For example, survey participation in the NAMCS Health Center (HC) Component requires not only voluntarily agreeing to participate, but the use of resources such as health information technology (IT) staff expertise, coordination with an EHR vendor (i.e., company from whom the EHR system is purchased), and health IT infrastructure/configuration to support bulk submission of clinical data in a specified format. Use of these resources all have potential for accruing (not reducing) cost, which may be a deterrent from participation in the survey.

Motivated by an understanding of the need for resource allocation and potential cost implications by a health center to participate in the NAMCS HC Component, in 2021 NCHS began providing a set-up fee of up to $10,000 to centers who agreed to participate in the data collection. Similar to an incentive, this payment may encourage participation in the survey. However, this set-up fee is not an incentive. Instead, its main purpose was more similar to a reimbursement (see Toronto Metropolitan University 2017) where it was given as a one-time payment to a participating health center who could use it to offset the resources/cost of setting up their EHR system to transmit data to NCHS as part of the NAMCS HC Component. Specifically, this set-up fee was provided in two separate payments with up to $5,000 provided upon successful submission of valid test data to NCHS, and up to $5,000 provided upon receipt of a full year of production data files. Here we describe details of the NAMCS HC Component data collection, use of this set-up fee during recruitment, and its administration to participating health centers throughout the data collection process. To the authors’ knowledge, this is the first time a study has described the use of a set-up fee that is contingent on the delivery of valid data to encourage participation and retention of health centers or similar establishments. Findings from a supplemental Set-up Fee Questionnaire administered to health centers recruited in 2022 are also presented, as well as results from participation and retention of those centers recruited in 2021–2022. Finally, lessons learned from this case study and their considerations to broader establishment surveys are discussed.

Methods and Procedures

Data Description and Collection

The NAMCS HC Component is a nationally representative establishment survey that collects data on ambulatory medical care visits to federally qualified health centers (FQHCs) and FQHC look-alikes throughout the 50 U.S. states and District of Columbia. Additional information on the survey can be found here: https://www.cdc.gov/nchs/namcs-health-center/about/index.html. Note: for the remainder of the manuscript, we will use the term “health center” to refer to both FQHCs and FQHC look-alikes. EHR data from these visits are transmitted from participating health centers to NCHS. The NAMCS HC Component universe included 1,463 and 1,482 health centers, with sampling frames of 1,400 and 1,269 health centers for 2021 and 2022, respectively (Williams et al. 2023). In the 2021 NAMCS HC Component, a primary sample of 50 health centers was drawn, along with a reserve sample of 100 health centers. If one of the 50 health centers in the primary sample declined or was unable to participate, a similar health center from the reserve sample was then contacted for participation. Additional details on sampling procedures are available elsewhere (Williams et al. 2023).

Participation by health centers in the 2021 NAMCS HC Component included establishing initial contact/agreement, followed by answering a 45-minute facility-level questionnaire to gather information about the center, which was used by NCHS to calculate weights that were used to generate national estimates of health center visits. NCHS worked with health center staff, and often EHR vendors, to ensure the center was able to submit data via the HL7 CDA R2 Implementation Guide (IG): National Health Care Surveys, Release 1, DSTU Release 1.2 – US Realm (Health Level Seven International 2021). This guide serves as an electronic “template” for EHR systems and allows a center to submit the data requested for participation in the NAMCS HC Component through their EHR systems in the required format, for the specified variables requested. If a sampled health center was unable to submit data via the Implementation Guide, another option for health center participation was to send a custom data extract from their EHR system. Using these submission options, the health center then worked with NCHS to submit a testing and validation (T&V) file. This T&V file contained a sample of the health center’s EHR data, allowing NCHS to perform basic quality assessments to ensure that the data were properly formatted and met requirements for NAMCS HC Component. T&V was often an iterative process. In some instances, T&V data files did not pass on the first try and required coordination with NCHS, health center, and sometimes EHR vendor staff to remediate any issues and re-send a T&V file that was deemed adequate. Once T&V submission was successful, NAMCS HC Component production data files were then sent to NCHS, (often in batches) by a health center, where each record was a single medical encounter. These files contained all encounters that occurred at a health center for the entire calendar year. Figure 1 provides an example of the NAMCS HC Component participation process.

Figure 1.

Example of National Ambulatory Medical Care Survey Health Center Component Participation and Corresponding EHR Data Submission Process by a Health Center

Notes: CATI = computer-assisted telephone interview; EHR = electronic health record; NCHS = National Center for Health Statistics; IG = HL7 CDA R2 Implementation Guide: National Health Care Surveys, Release 1, DSTU Release 1.2 – US Realm; T&V = testing and validation; XML = extensible markup language.

For the 2022 NAMCS HC Component, a primary sample of 54 health centers was drawn, along with a reserve sample of 120. Similar to 2021, if one of these 54 health centers in the primary sample declined or was unable to participate, a center from the reserve sample was contacted. Centers newly recruited in 2022 followed the same data submission process detailed above.

In addition, those health centers who initially participated in 2021 were asked to again participate by submitting production data using the Implementation Guide or a custom extract. Having already moved through the T&V process, these returning health centers did not repeat the T&V process again for 2022, but instead moved directly to the submission of production data for the 2022 calendar year. Note: a similar process was used in 2023 with those centers recruited in 2021 and 2022.

Set-up Fee and Set-up Fee Questionnaire

It was recognized that participation in the NAMCS HC Component would require use of health center resources. Therefore, a one-time set-up fee of up to $10,000 was offered to participating health centers. To determine an amount appropriate to offset/cover the resources needed for participation, NCHS used knowledge from its own experience conducting other National Health Care Surveys and subject matter expertise from HRSA (who oversees the Health Center Program). Participating health centers were provided this fee in two separate payments. Up to $5,000 was provided upon successful submission of a valid T&V file, while a second fee of up to $5,000 was provided upon receipt of a full year of production data files. Prior to participation agreement, health centers were informed these payments could be used to reimburse them for the costs and resources used while participating. Here a health center could specify the dollar amount based on their cost of participation. This could be up to $10,000, but a health center could request less than this amount. This fee was “one-time” (i.e., a center recruited in 2021 could receive this set-up fee in 2021 but would not receive any additional fees for participation in later years).

Approval for this one-time set-up fee for the NAMCS HC Component was received from the U.S. Office of Management and Budget (OMB) (No. 0920–0234). As this was the first time NCHS used a set-up fee for NAMCS, OMB suggested the creation and administration of a ~15-minute “Set-up Fee Questionnaire” to health centers after submission of their final NAMCS production data file. This questionnaire was administered to health centers recruited in 2022 and contained five questions asking how much of the fee was issued to the participating health center, what resource/cost categories and amounts the fee was used for, and if incurred costs exceeded $10,000. While brief, this allowed NCHS to gather information on use of this fee by health centers to better understand its impact and ability to offset costs accrued during participation.

Results

Responses to the NAMCS Set-Up Fee Questionnaire were received from 32 of 36 health centers recruited in 2022. The majority of these health centers requested and were issued amounts close to the maximum available set-up fee, where the average set-up fee issued to centers was $9,700 (standard deviation [SD]=$1,200; range=$5,000-$10,000). When examining the specific resources/categories that incurred costs for the health centers (Table 1), 25.0% of health centers used a portion of this fee on IT staff, 21.9% on IG installation/configuration (i.e., the process required to set up the IG within a health center’s EHR system), and 18.8% on EHR vendor staff. Of those health centers who used the fee for IT staff, an average of $5,300 was spent. Of those who used a portion of the fee for EHR vendor staff, an average of $3,400 was spent. Although just over 1-in-5 centers reported using a portion of the fee for IG installation/configuration, many who reported using the fee for this category were unable to report a dollar amount; some citing the reason being that the cost was not mutually exclusive from EHR vendor staff costs. Results are not reported for the hardware, software, Health Information Service Provider, or other categories for confidentiality reasons, as fewer than five health centers reported using any portion of the set-up on these resources.

Table 1.

Usage of Set-up Fee Offered to Newly Recruited Health Centers for Participation in the 2022 National Ambulatory Medical Care Survey Health Center Component, by Resource/Cost Category (n=32)

Resource/Cost Category	% Health Centers Using Set-up Fee for this Category	Average $ Spent (SD)	Range of $ Spent
Health center IT staff	25.0	5,300 (3,000)	1,500–10,000
EHR vendor staff	18.8	3,400 (1,800)	1,000–6,000
IG Installation and configuration	21.9	---	---
Hardware	---	---	---
Software	---	---	---
Health Information Service Providera	---	---	---
Other	---	---	---

Note: Resource/cost categories are not mutually exclusive and are listed in the order they appear on the Set-up Fee Questionnaire. Averages (and SDs) are rounded to the nearest hundred.

Data source: National Ambulatory Medical Care Survey Health Center Component Set-up Fee Questionnaire, 2022

SD = standard deviation; IT = information technology; EHR = electronic health record; IG = HL7 CDA R2 Implementation Guide: National Health Care Surveys, Release 1, DSTU Release 1.2 – US Realm; — = data not shown for confidentiality reasons as fewer than five health centers used fees for this category, or reported the average monies spent.

^aHealth Information Service Provider, or HISP, is an accredited network service that allows for the secure transportation of clinical health information (such as EHRs) through Direct Secure Messaging.

In 2021, production data were received from 30 health centers, 20 of which were from the reserve sample, although 50 health centers were targeted. In 2022, data were received from 36 newly recruited health centers, 23 of which were from the reserve sample, with 54 targeted health centers. Even with the use of a set-up fee, securing initial health center recruitment remained a challenge. However, with the fee only offered to health centers during the year of recruitment, the year-to-year retention rate could also be an indicator of the effectiveness of this fee. In 2021, the first year that the NAMCS HC Component began collecting data from health centers via EHR submission, 30 health centers participated (Table 2). In 2022, 28 of these health centers were retained (2021 to 2022 retention rate=0.933), while in 2023 there were two additional centers from this recruitment year who participated, bringing the number of centers back to 30 (2021 to 2023 retention rate=1.000). For the 36 health centers that were recruited in 2022, all 36 health centers continued to participate in 2023 (2022 to 2023 retention rate=1.000).

Table 2.

Number of Health Centers Recruited and Retained for the National Ambulatory Medical Care Survey Health Center Component During the Years 2021–2023

	# of Participating Health Centers by Year			Retention Rates
Recruitment Year	2021	2022	2023	2021 to 2022	2021 to 2023	2022 to 2023
2021	30	28	30a	0.933	1.000	---
2022	---	36	36	---	---	1.000

NA = not applicable.

^aTwo health centers were sampled for recruitment in 2021, but did not participate or provide production data until 2023.

Discussion

The $10,000 set-up fee appears to sufficiently offset costs for participating in the NAMCS HC Component. Furthermore, it appears that IT staffing – both by the health center and EHR vendor – was the most common use of the fee. Few (if any) health centers reported using the fee for reasons other than IT staffing and IG installation/configuration. Second, even with a set-up fee, securing initial participation by health centers was a challenge. Yet even with the fee only available to health centers during the year they were recruited, retention rates remained high. It appears once a health center’s EHR system was set up and configured to submit data for the NAMCS HC Component, the retention rate was high, indicating that the health center would continue to participate in future years. Therefore, it is planned to continue to utilize this set-up fee for recruitment in future years of the NAMCS HC Component.

As this case study uses data from a single establishment survey, the NAMCS HC Component, caution should be used to not overgeneralize the findings. It is also based on a small number of health centers, with Set-up Fee Questionnaire data from only one year. Furthermore, while health centers clearly understood when fees were used for IT staff, there was less clarity on its use/amount for other resource categories. A more in-depth examination could provide a better understanding of the use of this set-up fee. Finally, the long-term return on investment of this fee remains unknown. Additional years of data will allow for a better understanding of any long-term impact.

Even with these limitations, this study of the NAMCS HC Component does offer insight and lessons for establishment surveys that collect large quantities of data via electronic submission, and where installation or use of specialized IT by the establishment is needed for participation. First, there may be numerous resources and costs (specifically IT staff-related) that serve as critical components in facilitating participation by establishments. Even with a sizeable set-up fee, recruitment may remain a challenge. Competition for resources may be a barrier to survey participation (Willimack and Snijkers 2013). As shown in our study, the biggest challenges for these surveys may be recruitment and getting initial procedures/systems in place needed for submitting data, even when using a set-up fee. Furthermore, surveys using a set-up fee or similar type of payment also need to determine the appropriate amount of the fee to be offered. Leveraging knowledge from experts familiar with the establishments being surveyed may help with this determination; however, this also must be balanced with the funds available by the organization conducting the survey. A final lesson from this study is that even when a set-up fee is offered, recruitment and initial survey participation remains a challenge; however, this fee does appear to help maintain high retention and continued data submission/participation. Use of such a fee could be seen as an investment and more useful for establishment surveys that are longitudinal in nature.

Even with these lessons learned, health IT advances at a rapid pace and inevitably new technologies, systems, and procedures will continue to change not only the manners in which establishments continue to collect, store, and organize their data, but also the methods and procedures used to conduct establishment surveys. Survey researchers should be aware of this consideration and continue to monitor how the use of a set-up fee (or similar payment) can be used and leveraged to obtain survey data. This case study of the NAMCS HC Component serves as a novel exploration of the use of a set-up fee that is contingent on the delivery of valid data, and future research on this topic could provide additional understanding for improving recruitment, participation, and retention in establishment surveys.