A comparison of survey incentive methods to recruit rural cancer survivors into cancer care delivery research studies

Abstract

Unconditional (upfront) incentives are proposed to improve acceptance of cancer research among underrepresented, racial/ethnic minority populations, but few studies have tested incentive strategies among rural cancer survivors. Descriptive statistics summarized demographic characteristics of survey respondents, and response rates by arm were compared using Chi-square tests. We compared upfront ($2) and response-based ($10 conditional) incentives in a mailed survey of adult post-treatment rural survivors. Individuals meeting eligibility criteria from the electronic medical record (n = 2,830) were randomized into two incentive arms (n = 1,414 for the upfront arm and n = 1,416 for the contingent arm). Of the total delivered, presumed eligible participants (n = 1,304 upfront arm; n = 1,317 contingent arm), 67.8% were aged 65y+, 49.8% were female, and 95.1% were non-Hispanic white. The response rate for all participants was 18.5%. We received eligible surveys from 281 rural survivors in the first arm (response rate: 21.5%); and 205 surveys in the second arm (response rate: 15.6%). Participants who received the upfront incentive had a higher response rate than those receiving a response-based incentive, X² (1, 2,621) = 15.53, p < 0.0001. Incentivizing survey completion with an upfront $2 bill encouraged a higher survey response rate; other supplemental strategies are needed to achieve a higher response rate for this population.

Introduction

Rural residents account for approximately 14% to 19% of the general population and 21% of cancer survivors in the US [1-3]. Rural survivors also experience 10% higher cancer mortality when compared to urban residents, despite lower cancer incidence rates (442 cases per 100,000 vs 455 cases per 100,000) [1-3]. Data examining differences in stage of diagnosis are mixed, although some studies indicate later stage diagnosis for breast cancer among rural women [3, 4]. Research examining causes of and interventions to address these disparities is vital to improve cancer outcomes among rural residents, yet rural residents face both barriers to care and barriers to research participation. Limited clinical trial knowledge and awareness, lack of provider support, and mis-trust of providers and the health care system have resulted in rural residents being underrepresented in cancer-related studies [5-9]. Incentives are proposed to enhance participation in cancer research among underrepresented populations, but few studies have formally tested various incentive strategies to promote participation among cancer survivors from these populations [7].

Incentive strategies for mailed surveys can include variations in the amount, type, and timing [10, 11]. Mailed survey research typically incentivizes participation following completion (e.g., gift card after survey completion), known as a conditional or contingent incentives; however, upfront, or unconditional, incentives have also been studied to address declining response rates to mailed surveys [12]. Among cancer survivors, one study found higher response rates among childhood cancer survivors receiving a $10 upfront incentive, but other studies of diverse breast and prostate cancer survivors have found no difference in response by incentive timing [13-16]. A study of lung cancer screening participants comparing incentive methods demonstrated a higher response rate to a mailed survey for those receiving an upfront versus a contingent incentive [17].

It is unclear how these findings might generalize to a population-based survey of rural cancer survivors who had completed initial treatment and may have additional barriers to study participation (e.g., transportation issues, low health literacy, and medical trust). In this study, we compared upfront (unconditional) and response-based contingent (conditional) incentive strategies in a mailed survey of this understudied survivor population. We hypothesize an increased response rate by providing an upfront incentive compared to a contingent incentive.

Methods

Our team developed a mailed survey to understand the psychosocial and medical needs of rural individuals with a history of cancer residing within the Wake Forest Baptist Health Comprehensive Cancer Center’s catchment area. The Cancer Center services 58 counties with a total population of more than 4.1 million residents where approximately 59% reside in rural, primarily Appalachian, areas. This area is known for high rates of poverty and higher than average rates of smoking. This study was approved by the Wake Forest Health Sciences Institutional Review Board (IRB00056939) prior to implementation.

We identified individuals from the institutional electronic medical record who met the following eligibility criteria: (1) aged 18 + years; (2) a cancer diagnosis other than non-melanoma skin cancer; (3) > 6 months post-definitive cancer treatment and/or receiving ongoing systemic maintenance therapies; (4) were seen at any institutional clinical facility from January 2014 to January 2019 for any reason regardless of where they were diagnosed or treated for their cancer; and, (5) resided in 1 of 7 priority rural counties in our catchment area (all had a Rural–Urban Commuting Area [RUCA] code of 4–10). These counties were designated as priority due to their high prevalence of smoking and greater cancer incidence and mortality rates compared to the other counties in the catchment area. The survey was only available in English, effectively excluding respondents unable to complete the survey if they could not understand the survey in its original form as no translation services were available.

Participants were mailed a recruitment packet in bright orange envelopes with an introductory letter (including a link to complete the survey online if preferred), paper survey, and a postage paid return envelope. We intentionally did not use branded stationery or white envelopes to distinguish the packets from medical bills.

Potential respondents were randomly assigned to one of two incentive arms. The first arm mailings included a $2 bill, and respondents could opt into a drawing for one of five $50 gift cards upon survey completion (upfront incentive). The second arm provided respondents with a $10 gift card upon completion and return of the survey (contingent incentive). Respondents received the contingent incentive and could participate in the $50 gift card drawing even if the returned survey was not complete. Respondents could also skip any questions (e.g., income).

The total surveys mailed and completed were summarized according to the Consolidated Standards for Reporting Trials (CONSORT) criteria [18]. Descriptive statistics summarized demographic characteristics and response rates by arm; response rates were compared using a Chi-square test. Demographic characteristics of respondents were compared by arm using Chi-square tests. We also compared responders to non-responders in each arm across demographic strata using Cochran-Mantel–Haenszel tests. All analyses were performed in SAS (version 9.4, Cary, NC) at a two-sided alpha level of 0.05.

Results

Individuals meeting initial eligibility criteria from the electronic medical record were randomized into two incentive arms (n = 1,414 for the upfront arm and n = 1,416 for the contingent arm), totaling 2,830 potential participants (Fig. 1).

Fig. 1.

CONSORT diagram of survey randomization. *5 participants were deemed ineligible and not returned in Arm 1, and 3 participants were deemed ineligible and not returned in Arm 2. ^†X² (1, 2,621) = 15.53, p < 0.0001

Analysis of non-responses

In Arm 1 (upfront), 60 packets were returned undeliverable by post office and 21 indicated that the intended respondent was deceased, reducing the sample to 1,333 delivered survey packets. In Arm 2 (contingent), 62 packets were returned undeliverable by post office and 23 intended respondents were deceased, for a total of 1,331 survey packets delivered to potentially eligible respondents (Fig. 1). Of the 29 excluded individuals in Arm 1, 24 were excluded based on information received in the returned survey: they currently reside in an urban location; they reported no history of cancer although the medical record contained a cancer diagnosis code; they had received cancer diagnosis within the past 6 months. Five were excluded who did not return a survey and who were identified as ineligible after a more extensive review of the medical record that revealed the individual did not have a final cancer diagnosis, had received cancer treatment within past 6 months, and/or currently resided in an urban residence. In Arm 2, 11 returned surveys were deemed ineligible (urban residence, no cancer diagnosis, cancer treatment within past 6 months), and 3 were excluded who did not return a survey (no cancer diagnosis, urban residence). This resulted in a total sample size of 2,621 (1,304 in Arm 1 and 1,317 in Arm 2). We did not receive a response or respondents indicated that they did not wish to participate for n = 1,023 packets in Arm 1 and n = 1,112 packets in Arm 2. The response rate for all delivered surveys (including eligible and ineligible) was 22.9% in Arm 1 and 16.2% in Arm 2.

Analysis of eligible responses

We received completed, eligible surveys from 281 rural cancer survivors in the upfront incentive arm (Arm 1 response rate: 21.5%); 205 surveys were received from the contingent incentive arm (Arm 2 response rate: 15.6%). A comparison of the response rates demonstrated a significantly higher response rate for participants who received the upfront incentive, X² (1, 2,621) = 15.53, p < 0.0001. The response rate of total eligible responses including both arms was 18.5%.

Of the total delivered, presumed eligible participants, approximately two-thirds were aged 65y+ (Table 1). Females represented about half of the overall sample. Non-Hispanic white participants represented the majority of the sample at 95.1% with only 4.9% representing any other race/ethnicity, approximating the racial/ethnic distribution of the general population in these counties. We compared the characteristics of eligible participants by incentive arm (Table 1). Age category differed by arm with a higher proportion of those aged 65y+ randomly assigned to Arm 2 than Arm 1 (p = 0.0002); no significant differences by arm were observed by sex or race/ethnicity. Therefore, to compare responders to non-responders, we explored the results stratified by arm. The results indicated that there was no difference in demographic characteristics between responders and non-responders controlling for arm (Table 1).

Table 1.

Comparison of proportions (%) of eligible participants by demographic characteristics and incentive arm (Arm 1: n = 1,304, 49.8%; Arm 2: n = 1,317, 50.2%) and response status (Arm 1: responder n = 281, 21.5%, non-responder 78.5%; Arm 2: responder n = 205, 15.6%, non-responder n = 1,112, 84.4%)

	Overall (n = 2,621)	Arm 1: $2 Upfront Incentive (n = 1,304)	Arm 2: $10 Contingent Incentive (n = 1,317)	p value
Age (years)				0.0002*
18–64	32.2	35.7	28.9
65+	67.8	64.3	71.2
Sex				0.2189*
Female	49.8	51.0	48.6
Male	50.2	49.0	51.4
Race/ethnicity				0.6744*
Non-Hispanic white	95.1	94.9	95.3
Other	4.9	5.1	4.7

	Responder (n = 281)	Non-Responder (n = 1,023)	Responder (n = 205)	Non-Responder (n = 1,112)
Age (years)					0.4938 †
18–64	36.3	35.5	24.9	29.6
65+	63.7	64.5	75.1	70.4
Sex					0.6926 †
Female	48.0	51.8	50.7	48.2
Male	52.0	48.2	49.3	51.8
Race/ethnicity					0.1088 †
Non-Hispanic white	97.2	94.3	95.6	95.2
Other	2.9	5.7	4.4	4.8

^*p value for comparison of eligible participants by demographic characteristics in Arm 1 and Arm 2 using Chi-square test

^†p value for comparison of responders to non-responders by demographic characteristics stratified by arm using Cochran-Mantel–Haenszel test

Discussion

Among rural cancer survivors, incentivizing survey completion with an upfront $2 bill and a drawing of a $50 gift card encouraged a higher survey response rate compared to a contingent gift card; however, the response rate was < 22% in both groups. The response rate in this study was much lower than similar studies comparing incentive methods that ranged from 31.8 to 73.0% [11, 13-17, 19]. Similar to the Rosoff et al. [16] study of pediatric cancer survivors and the Kumar et al. [17] study of lung cancer screening participants, we observed higher response with upfront incentives [16, 17].

In contrast, other studies did not observe differences in response by incentive structure among prostate cancer survivors [14, 15]. However, we included female survivors and focused particularly on rural survivors with a variety of diagnoses not represented in these studies. Some rural surveys of the general population report notably higher response rates (71.0–73.0%) using strategies such as repeated mailings at regular intervals to non-responders, larger incentives, and hand delivering from a trusted source [19, 20]. Similar strategies including phone call follow-up to non-responders, use of proxies, and multiple mailings also resulted in higher response rates among childhood cancer survivors [16]. Despite a lower overall response rate in this study, the recruited sample represented a broad population of potentially eligible rural cancer survivors identified from the electronic medical record.

Limitations

We did not test additional strategies suggested by Dillman for increasing survey completion that may improve response rates, such as postcard reminders or follow-up phone calls, due to time and cost constraints [12]. Inclusion of the gift card drawing with the upfront incentive was done to even out the possible total value of the two incentives for both arms, but it meant the comparison was not solely between upfront and contingent incentives. Also, this study was conducted in 7 counties in our institution’s catchment area which were selected based on their higher cancer incidence, mortality, and smoking prevalence but were not geographically weighted to be representative of the population of rural cancer survivors in our catchment area nor the US.

The instrument was also limited to those who could read and understand English as budget and time restrictions did not allow for translation of the survey into other languages. Rural populations generally have lower rates of health literacy, while variation in education, age, gender, and race/ethnicity further confound literacy rates in these populations [21]. Future studies should examine methods to recruit non-English speaking participants, particularly Spanish speaking participants, as the rural, Hispanic population is growing in rural North Carolina, and they experience significant cancer-related health disparities [22, 23]. While the population of the priority counties examined in this study are overwhelmingly non-Hispanic white and the cancer survivorship population is even more concentrated in this racial/ethnic group, greater effort to recruit underserved populations with limited English proficiency would improve representation of understudied populations and advance scientific understanding of their cancer experiences.

Conclusion

Results from this study may be helpful for researchers planning population-based survey assessments of cancer center catchment populations to promote community outreach and engagement or pilot data for future research especially in rural areas. Upfront incentives may be useful, but they will likely need to be combined with other strategies to increase survey responses and rural participation in cancer research.

Data availability

Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data are not available.