Abstract
Purpose: Contacting childhood cancer survivors (CCS) to assess reasons for declining receipt of follow-up care after treatment is difficult and participation in surveys may be low, resulting in biased results. We sought to demonstrate effective recruitment and population-based sampling methods to improve response and minimize bias.
Methods: Four hundred and seventy CCS diagnosed between 2000 and 2007 at two hospitals in Los Angeles County were selected from the California Cancer Registry and were 15–25 years of age at the time of interview. Surveys of survivors and their parents were completed by multiple methods including mail, online, and telephone. Effectiveness of “plain” versus “designer” formatting of study materials was tested. Variables associated with response were analyzed using univariate and multivariable methods. Effort required for recruitment was quantified.
Results: Fifty percent of survivors (n = 235) and 36.5% of parents (n = 171) responded, and there were 160 parent-child dyads among them. Among located survivors, 61% participated. Response was higher for women, parents of younger survivors, and those from higher socioeconomic status areas. Among Hispanics, no variables were related to response. More effort was required to reach men and older survivors, but efforts beyond 15 calls and 7 remailings were unproductive. Formatting (i.e., plain vs. designer) did not affect response.
Conclusion: Efforts to reach survivors must include multiple methods to be successful. Use of an intensive recruitment strategy and population-based sample resulted in a largely representative sample of CCS, especially for Hispanics. Expensive design efforts had little effect on recruitment, suggesting that plainer materials are sufficient. This example may inform similar studies.
Keywords: survey methods, response rate, response bias, cancer registry, parents, population-based sample
Introduction
Recruiting childhood cancer survivors (CCS) for research studies is challenging. Contact with survivors under the age of 18 requires parental consent and locating subjects after diagnosis is difficult because young adults are highly mobile.1 Furthermore, CCS who have completed their treatment may not want to be reminded of their past.2 Nevertheless, reaching them is important, especially to increase our understanding of factors that contribute to adherence with long-term follow-up care recommendations and risk and protective factors for late and long-term effects as a result of their treatment regimens.3–5
The largest research cohort of CCS is the Childhood Cancer Survivor Study, which recruited over 20,000 childhood cancer cases diagnosed under 21 years of age between 1970 and 1986 who had survived at least 5 years at the time of study entry.6 This cohort has been a critical resource for studying CCS, however, it was not population-based and Hispanics comprised just 5% of the total. Nevertheless, because the cases were recruited from hospitals, the investigators were able to compare characteristics of the respondents (representing 69% of the eligible cohort) to nonrespondents based on age, gender, cancer site, and initial treatment (but not on race/ethnicity) and found no differences.6 Thus, while the study does not represent the source population of all CCS in the United States, the findings based on data collected from within the cohort are likely to be valid.
The response rate to observational studies has been a factor used to assess the likelihood of selection bias based on differences between respondents and non-respondents. Some academic journals require, for example, at least a 60% response rate.7 Response rates to epidemiological studies have been declining from about 80% to 30%–40% over the past 30 years.8,9 Other studies of young adult survivors of childhood cancer have reported generally low response rates (e.g., 30%–31%),10,11 and previous studies of adolescent and young adult (AYA) cancer patients have shown that younger age, male gender, and residence in lower socioeconomic areas were associated with lower response12 as were racial/ethnic minorities.13 In survey research studies, in general, response rates are lower among younger people than among older adults (other than the elderly who may be institutionalized).14
The relationship of the response rate to nonresponse bias is complex. To determine whether nonresponse bias exists it is important to assess whether variables associated with response are also related to the study outcomes.15 In a study of patient perceptions with hospital care, for example, nonresponse was found to be associated with more negative perceptions of hospital care, and early responders had fewer problems than late responders.16 However, increasing the response rate from 30% to 70% did not have a large effect on the conclusions of the study. A similar insensitivity in results to the response rate was found in a large national telephone survey17; whereas another study of adherence to medication found considerable bias in results of a survey that achieved a 24.5% response rate, where nonresponders had 11% lower adherence than responders.18
Efforts expended to increase response, while generally considered beneficial, may actually be associated with greater bias if certain subgroups are selectively targeted (e.g., those with higher literacy).19 On the other hand, in the CCS study, tracing of nonrespondents was shown to locate those who were less likely to have accessed healthcare in the past 2 years, and, if not done, may have resulted in biased results for some survey outcomes.20
With few available population-based studies of CCS available that have achieved higher response rates, information on the extent of long-term health problems, use of follow-up care, psychosocial issues, and other problems unique to this population is scarce. Here, we describe a study of CCS and their parents that was based on a sampling frame (a population-based cancer registry) that permitted the comparison of clinical and demographic characteristics between respondents and nonrespondents to assess potential nonresponse bias. In addition, multiple methods used to increase response are described, several of which were shown to be effective in other studies,21 including the Dillman method22 that involved follow-up calling and second mailings of the questionnaire, and use of incentives, pre-notification, and university sponsorship.21 We also tested whether professionally designed recruitment materials contributed to increased response, which has previously been shown to increase participation in online studies.23
Materials and Methods
Data sources
The sources of data included cancer registry information on clinical and demographic variables for the CCS selected for the study and the survey completed by CCS on use of follow-up care and other items.24 Surveys of parents were also included to understand issues of concern to the parent and to compare the parent's perspective on the child's experience with the self-reported information from the child.
Source of cases
Cases were selected from the Los Angeles Cancer Surveillance Program, the Surveillance, Epidemiology, and End Results program (SEER) cancer registry for Los Angeles County (and member of the California Cancer Registry) and included patients diagnosed at Children's Hospital Los Angeles (CHLA) or Miller Children's Hospital in Long Beach. Cases were 5–18 years old at diagnosis between 2000 and 2007 (with any cancer except for Hodgkin Lymphoma), were a minimum of 2 years past diagnosis, and aged 15–25 years when contacted, beginning in 2009 when the study was initiated. A total of 515 CCS (who were AYA survivors of pediatric cancer) and their parents were initially sampled.
Development of survey and recruitment methods
Both the CCS and parents' surveys included similar questions related to the CCS's treatment and utilization of care that have been previously described.24 We held focus groups separately with CCS and parents to ensure that questions were of importance to CCS and parents, were easily understood, that the questionnaire could be completed in 30–45 minutes, and that our approach of mailing the questionnaire along with a cover letter and brochure would encourage survivors and their parents to participate. Based on feedback from the participants in these groups we (1) shortened and simplified the questionnaire because some participants were unable to complete it, (2) held meetings with physicians to obtain their endorsement because participants mentioned how important their physician was to their care, (3) provided monetary incentives to compensate subjects for time and effort required to participate as a result of suggestions we received, and (4) developed a study brochure with a “cool design” based on input from the groups. Two versions of the design were tested, one with professionally designed layout and formatted pictures featuring multicultural young adults (“designer” version) and one with the study logo and wording in text format with no photographs or design elements (“plain” version) (Fig. 1).
FIG. 1.
Comparison of formatting styles for plain and designer versions of introductory postcard. (a) Designer. (b) Plain.
Data collection procedures
A courtesy letter was mailed to the treating physician describing the study and our plan to contact their patient within 2 weeks unless they informed us otherwise (none did). For survivors currently 18 years of age or older, we mailed a postcard to inform them about the upcoming study (and to identify bad addresses), followed by a survey packet 2 weeks later that included an introductory letter, study brochure, survey, and postage-paid return envelope. We obtained a waiver of written consent for the completion of the mailed survey for participants 18 years of age or older. The introductory letter included the elements of the informed consent including assurance that participation was voluntary and that information would be kept confidential. Possible risks and benefits associated with participating were described, and the participant was advised that any question in the survey could be skipped if so desired. With the letter and survey available, consent was implied if the participant completed the questionnaire and mailed it back. For those younger than 18 years of age, we mailed the postcard (and survey packet) to the parents of the survivor and required written parental consent as well as written informed assent from the minor child before participation.
For the recruitment of the parents we followed two approaches. If the survivor was 18 years of age or older, we requested the name and address of their parents and permission to contact them and then we mailed survey materials to the parent. If the survivor was younger than 18 years of age, we included both the parent and child's questionnaire in our initial packet addressed to the parent, requesting that both the parent and child participate.
Both survivors and their parents were offered a $20 gift card and entry into a $300 lottery for completion of a survey that took 30–45 minutes to complete. After 3 weeks, if no response, survivors (or parents) were contacted by phone (multiple times at different times of day and week) and second mailings were sent. If requested or determined to be necessary, because of low literacy or difficulty in completing the mailed version, a telephone interview was conducted, and in a few cases, the participant requested a personal interview. If addresses or phone numbers were incorrect we traced the survivor (or parent) to find current contact information. This approach, which involved a mixed mode approach (i.e., mail and telephone options, and intensive follow-up including second mailings) has been described as a modified Dillman approach, which has been used in other studies involving a mailed questionnaire to maximize response,22 and may be applied to other surveys using a different initial approach.
The CCS sample was randomized 1:1 to receiving the plain versus designer version of the postcard and brochure included in the first survey packet. Subsequent mailings to the “plain” subgroup were made with the “designer” version of the brochure, thus we were only able to compare the effect of the two versions on the response to the initial mailing.
All procedures were approved by Institutional Review Boards for the California Committee for the Protection of Human Subjects, California Cancer Registry, the University of Southern California, CHLA, and Miller Children's Hospital Long Beach. Informed consent (either written or with a waiver of written consent) was obtained from all individual participants included in the study.
Registry variables
Cancer registry variables on sex, age at diagnosis, race, Hispanic ethnicity, birth date, year of diagnosis, cancer site, stage of disease, hospital, and an ecologic variable based on the socioeconomic status (SES) of the census tract of residence were available for all sampled cases. The SES variable was based on a combined score of the rankings of the census tract by education and income divided into quintiles (1 = very high, 2 = high, 3 = median, 4 = low, and 5 = very low).25,26
Tracking variables
Effort required to reach each participant was monitored, including numbers and dates of initial mailings, remailings, follow-up phone calls, and reminder postcards. Final status codes included completed (English or Spanish), by mail, by telephone, in person or online, direct refusal, passive refusal, lost, and ineligible. Reasons for ineligibility were also identified (i.e., deceased, too ill/incompetent, lived out of the country, or denial of cancer). Randomization to the “plain” versus the “designer” version of the initial mailing was coded. In similar manner, we also monitored the response to the parent's survey.
Statistical methods
Univariate and multivariable methods were used to assess response bias by comparing registry demographic and clinical characteristics of respondents to nonrespondents (for patients, parents, and dyads [pairs with response from parent and child]). Chi-square statistics were used to assess bivariate associations between each variable and response, and multivariable logistic regression models (including variables with bivariate p-values <0.10) were used to identify variables independently associated with response. Mean numbers of mailings and follow-up phone calls made were provided by response, gender, age group, ethnicity, and format version. Analyses were performed using SAS 9.4.
Results
After initial contact, 45 of the 515 sampled cases were determined to be ineligible due to being deceased (n = 25), located out of the country (n = 5), having cognitive or developmental impairment (n = 5), or denial of cancer (n = 10). Of the 470 eligible cases, 50% (235) participated by one of multiple methods including mail (199), phone (4), online (27), and in person (5) (Table 1). Six completed the questionnaire in Spanish. Reasons for nonresponse included refusal by the survivor (directly or passively [i.e., were located but never participated despite repeated calls/mailings]; n = 138), refusal by parents (of those younger than 18 years of age; n = 12), or lost to follow-up after tracing efforts were made (n = 85). Since, for the lost cases, we could not confirm eligibility based on reasons determined after contact, we calculated a participation rate of 61% (235/385) among those we did reach.
Table 1.
Response Rates for Childhood Cancer Survivors, Parents, and Parent-Child Dyads by Child's Characteristics
| Childhood cancer survivor response rate % (n = 470) | Parent response rate % (n = 468) | Dyads response rate % (n = 468) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Child's characteristics | Yes, N (%) | No, N (%) | χ2 | p | Yes, N (%) | No, N (%) | χ2 | p | Yes, N (%) | No, N (%) | χ2 | p |
| Total eligible sample | 235 (50.0) | 235 (50.0) | 171 (36.5) | 297 (63.5) | 160 (34.2) | 308 (65.8) | ||||||
| Age at survey return | ||||||||||||
| <18 | 56 (46.7) | 64 (53.3) | 0.851 | 0.654 | 56 (46.7) | 64 (53.3) | 7.178 | 0.028a | 52 (43.3) | 68 (56.7) | 6.003 | 0.049a |
| 18–20 | 82 (52.2) | 75 (47.8) | 51 (32.5) | 106 (67.5) | 49 (31.2) | 108 (68.8) | ||||||
| 21+ | 97 (50.3) | 96 (49.7) | 64 (33.5) | 127 (66.5) | 59 (30.9) | 132 (69.1) | ||||||
| Sex | ||||||||||||
| Male | 116 (44.8) | 143 (55.2) | 6.270 | 0.012a | 86 (33.3) | 172 (66.7) | 2.547 | 0.110 | 80 (31.0) | 178 (69.0) | 2.585 | 0.108 |
| Female | 119 (56.4) | 92 (43.6) | 85 (40.5) | 125 (59.5) | 80 (38.1) | 130 (61.9) | ||||||
| Year of diagnosis | ||||||||||||
| 2000 | 37 (54.4) | 31 (45.6) | 10.438 | 0.107 | 28 (41.2) | 40 (58.8) | 2.690 | 0.847 | 26 (38.2) | 42 (61.8) | 4.854 | 0.563 |
| 2001 | 30 (44.8) | 37 (55.2) | 22 (33.3) | 44 (66.7) | 21 (31.8) | 45 (68.2) | ||||||
| 2002 | 36 (52.9) | 32 (47.1) | 24 (35.3) | 44 (64.7) | 23 (33.8) | 45 (66.2) | ||||||
| 2003 | 24 (36.9) | 41 (63.1) | 19 (29.7) | 45 (70.3) | 15 (23.4) | 49 (76.6) | ||||||
| 2004 | 43 (56.6) | 33 (43.4) | 30 (39.5) | 46 (60.5) | 28 (36.8) | 48 (63.2) | ||||||
| 2005 | 45 (57.7) | 33 (42.3) | 30 (38.5) | 48 (61.5) | 30 (38.5) | 48 (61.5) | ||||||
| 2006 | 20 (41.7) | 28 (58.3) | 18 (37.5) | 30 (62.5) | 17 (35.4) | 31 (64.6) | ||||||
| Cancer site | ||||||||||||
| Leukemia | 74 (51.8) | 69 (48.2) | 0.437 | 0.979 | 52 (36.4) | 91 (63.6) | 1.40 | 0.844 | 48 (33.6) | 95 (66.4) | 1.372 | 0.849 |
| Brain and other CNS | 38 (47.5) | 42 (52.5) | 29 (36.2) | 51 (63.8) | 27 (33.8) | 53 (66.3) | ||||||
| Bones and joints | 11 (47.8) | 12 (52.2) | 11 (47.8) | 12 (52.2) | 10 (43.5) | 13 (56.5) | ||||||
| Lymphoma | 44 (50.6) | 43 (49.4) | 31 (36.5) | 54 (63.5) | 31 (36.5) | 54 (63.5) | ||||||
| Others | 68 (49.6) | 69 (50.4) | 48 (35.0) | 89 (65.0) | 44 (32.1) | 93 (67.9) | ||||||
| Socioeconomic status quintile of census tract at diagnosis | ||||||||||||
| Very high | 42 (62.7) | 25 (37.3) | 10.120 | 0.038a | 28 (41.8) | 39 (58.2) | 5.522 | 0.238 | 28 (41.8) | 39 (58.2) | 4.534 | 0.338 |
| High | 41 (60.3) | 27 (39.7) | 26 (38.2) | 42 (61.8) | 24 (35.3) | 44 (64.7) | ||||||
| Middle | 37 (45.1) | 45 (54.9) | 36 (43.9) | 46 (56.1) | 32 (39.0) | 50 (61.0) | ||||||
| Low | 55 (44.7) | 68 (55.3) | 43 (35.0) | 80 (65.0) | 39 (31.7) | 84 (68.3) | ||||||
| Very low | 60 (46.2) | 70 (53.8) | 38 (29.7) | 90 (70.3) | 37 (28.9) | 91 (71.1) | ||||||
| Race/ethnicity | ||||||||||||
| Non-Hispanic white | 80 (58.0) | 58 (42.0) | 7.340 | 0.026a | 64 (46.7) | 73 (53.3) | 16.146 | 0.001a | 61 (44.5) | 76 (55.5) | 14.192 | 0.001a |
| Hispanic | 121 (49.2) | 125 (50.8) | 90 (36.6) | 156 (63.4) | 82 (33.3) | 164 (66.7) | ||||||
| All other | 34 (39.5) | 52 (60.5) | 17 (20.0) | 68 (80.0) | 17 (20.0) | 68 (80.0) | ||||||
| Age at diagnosis | ||||||||||||
| <10 | 44 (44.0) | 56 (56.0) | 2.554 | 0.279 | 37 (37.0) | 63 (63.0) | 3.045 | 0.218 | 33 (33.0) | 67 (67.0) | 3.160 | 0.206 |
| 10–14 | 131 (50.2) | 130 (49.8) | 102 (39.2) | 158 (60.8) | 97 (37.3) | 163 (62.7) | ||||||
| 15+ | 60 (55.0) | 49 (45.0) | 32 (29.6) | 76 (70.4) | 30 (27.8) | 78 (72.2) | ||||||
| Format style | ||||||||||||
| Plain | 121 (51.9) | 112 (48.1) | 0.689 | 0.406 | 90 (38.6) | 143 (61.4) | 0.873 | 0.350 | 85 (36.5) | 148 (63.5) | 1.084 | 0.298 |
| Designer | 114 (48.1) | 123 (51.9) | 81 (34.5) | 154 (65.5) | 75 (31.9) | 160 (68.1) | ||||||
| Stage at diagnosis | ||||||||||||
| In situ and localized | 59 (48.4) | 63 (51.6) | 1.698 | 0.637 | 40 (32.8) | 82 (67.2) | 2.208 | 0.530 | 39 (32.0) | 83 (68.0) | 1.323 | 0.724 |
| Regional | 42 (47.2) | 47 (52.8) | 37 (41.6) | 52 (58.4) | 33 (37.1) | 56 (62.9) | ||||||
| Remote | 98 (50.0) | 98 (50.0) | 69 (35.4) | 126 (64.6) | 64 (32.8) | 131 (67.2) | ||||||
| Unknown | 36 (57.1) | 27 (42.9) | 25 (40.3) | 37 (59.7) | 24 (38.7) | 38 (61.3) | ||||||
| Hospital | ||||||||||||
| CHLA | 199 (52.2) | 182 (47.7) | 4.006 | 0.045a | 144 (37.9) | 236 (62.1) | 1.603 | 0.206 | 135 (35.5) | 245 (64.5) | 1.609 | 0.205 |
| Long Beach | 36 (40.4) | 53 (59.6) | 27 (30.7) | 61 (69.3) | 25 (28.4) | 63 (71.6) | ||||||
Significant variable associated with response in univariate analysis.
CHLA, Children's Hospital Los Angeles; CNS, central nervous system.
The sample was evenly divided between men and women, slightly over half of the respondents were of Hispanic ethnicity (54.4%) based on registry data, 59% were younger than 21 years of age at the time of survey completion, and the cancer sites with the largest numbers of respondents included leukemia (29.5%), brain (16.1%), and lymphoma (19.7%).24
A total of 171 parents participated (response rate of 36.5%) out of the 468 parents considered to be eligible (Table 1). Forty parents were not eligible because their child was deceased, denied having cancer, or was located out of the country. Reasons for nonresponse included refusal of permission to contact the parent by their child who was 18 years of age or older (n = 48), direct or passive refusal by the parent (n = 169), or lost to follow-up (n = 84). Among the 340 parents able to be contacted, we obtained a 50.3% participation rate. The parents' age when surveyed ranged from 34 to 69 years, and the responding parent was predominately female (87.7%), as previously described.27 Among the 235 CCS respondents and the 171 parent respondents, there were 160 families in which both the CCS and 1 parent participated. These respondent pairs were referred to as parent-child dyads (response rate of 34.2%).
Variables associated with response
We examined the bivariate relationship between all variables included in the registry database with likelihood of response. These included clinical variables (cancer site, stage, time since diagnosis, sex, marital status, SES of census block of residence at time of diagnosis, year of diagnosis, and hospital where treated). Among these variables the only ones that were significantly associated with response among CCS included sex, SES of census block, race/ethnicity, and hospital (Table 1). Women were significantly more likely to respond than men (56.4% vs. 44.8%, p = 0.012), those living in the highest SES quintile census blocks were more likely to respond than those in the lowest quintile (62.7% vs. 46.2%, p = 0.038), cases from CHLA were more likely to respond than those from Miller's Children's Hospital, Long Beach (52.2% vs. 40.4%, p = 0.045), and non-Hispanic whites (NHW) were more likely to participate (58.0%) than Hispanics (49.2%) or all other racial groups combined (including Blacks, Asians, and others; 39.5%, p = 0.026). Among Hispanics as a group there were no variables associated with response (data not shown).
In a multivariable model (that included age at survey, sex, SES, race/ethnicity, and hospital), only sex and SES (as a trend) remained associated with response (Table 2). Women were 1.52 times as likely to respond as men (95% confidence interval 1.04–2.22, p = 0.03), and there was a significant positive trend with response by SES quintile (p-trend = 0.007).
Table 2.
Adjusted Odds Ratios for Child's Response Based on Child's Characteristics
| Child's selected characteristics | Adjusted OR | 95% CI | p |
|---|---|---|---|
| Age at survey return | |||
| <18 | 1.00 (Ref.) | — | — |
| 18–20 | 1.22 | 0.75–1.99 | 0.4242 |
| 21+ | 1.24 | 0.77–2.00 | 0.3676 |
| p-trend = 0.6043 | |||
| Sex | |||
| Male | 1.00 (Ref.) | — | — |
| Female | 1.52 | 1.04–2.22 | 0.0294a |
| Socioeconomic status quintile of census tract of residence | |||
| Very low | 1.00 (Ref.) | — | — |
| Low | 0.95 | 0.57–1.57 | 0.8251 |
| Middle | 1.04 | 0.57–1.89 | 0.9035 |
| High | 1.71 | 0.88–3.31 | 0.1109 |
| Very high | 1.97 | 0.96–4.05 | 0.0647 |
| p-trend = 0.0072a | |||
| Race | |||
| Non-Hispanic white | 1.00 (Ref.) | — | — |
| Hispanic | 1.00 | 0.60–1.69 | 0.9937 |
| All other | 0.58 | 0.32–1.02 | 0.0596 |
| Hospital | |||
| CHLA | 1.00 (Ref.) | — | — |
| Long Beach | 0.69 | 0.42–1.12 | 0.1305 |
CI, confidence interval; OR, odds ratio.
Significant variable in multivariable model.
Bivariate results for the parents' response indicated parents of CCS younger than 18 years of age were more likely to respond than parents of CCS who were 21 years of age or older (46.7% vs. 33.5%, p = 0.03) (Table 1). Parents of NHW CCS were more likely to respond (46.7%) than parents of Hispanics (36.6%) or parents of CCS of other racial groups (20.0%; p = 0.001). Variables associated with response were the same for dyads. In a multivariable model, the child's age at survey and race/ethnicity remained significantly associated with the parents' response and those in the middle SES quintile were twice as likely to respond than those in the lowest SES quintile (Table 3). While there was no significant difference in response between parents of Hispanic CCS and those of NHW CCS, parents of CCS of other races were less likely to participate than those of NHW.
Table 3.
Adjusted Odds Ratios for Parent's Response Based on Child's Characteristics
| Child's selected characteristics | Adjusted OR | 95% CI | p |
|---|---|---|---|
| Age at survey return | |||
| <18 | 1.00 (Ref.) | — | — |
| 18–20 | 0.50 | 0.30–0.84 | 0.0081a |
| 21+ | 0.59 | 0.36–0.96 | 0.033a |
| p-trend = 0.0315a | |||
| Sex | |||
| Male | 1.00 (Ref.) | — | — |
| Female | 1.30 | 0.57–1.57 | 0.188 |
| Socioeconomic status quintile of census tract of residence | |||
| Very low | 1.00 (Ref.) | — | — |
| Low | 1.39 | 0.80–2.39 | 0.2398 |
| Middle | 2.14 | 1.13–4.04 | 0.0195a |
| High | 1.41 | 0.70–2.85 | 0.3339 |
| Very high | 1.78 | 0.83–3.82 | 0.1373 |
| p-trend = 0.0560 | |||
| Race/ethnicity | |||
| Non-Hispanic white | 1.00 (Ref.) | — | — |
| Hispanic | 0.81 | 0.48–1.39 | 0.4501 |
| All other | 0.30 | 0.15–0.57 | 0.0003a |
| Hospital | |||
| CHLA | 1.00 (Ref.) | — | — |
| Long Beach | 0.83 | 0.49–1.39 | 0.4747 |
Significant variable in multivariable model.
Effort required to recruit participants
Time required to recruit a CCS subject averaged 132.7 days (range 2–544) from the date of the initial mailing. Among recruited cases, the mean number of calls to obtain a response was 3.7 (range of 0–25), while an average of 7.9 calls/person were made in an unsuccessful effort to reach the non-respondents (range 0–26). In total we made 2671 calls (862 for recruited cases and 1809 for non-respondents). The effort expended for non-respondents became unproductive after 15 calls, since we only recruited 2 cases among those for whom we made 16 or more calls (from a total of 408 calls). The average number of mailings for recruited cases was 2.6 (range of 1–10), versus 4.3 for nonrespondents (range 1–9). In total we mailed 1628 packets (610 to recruited cases and 1018 in unsuccessful attempts). A cutoff after 7 mailings would have been economical, since we only obtained 2 of our completions with 8 or more mailings (and sent 81 mailings to this group). We also conducted drop by address verification visits for subjects for whom we had gotten no response to phone calls and mailings. In total we made 110 such visits that resulted in 25 completed cases.
Response did not differ by whether the “plain” versus “designer” formatting was used for the first mailing, but more effort was expended for the “plain” group. A slightly lower percentage of responses occurred with the minimum effort (i.e., one mailing only with no follow-up calls) for those receiving the “plain” materials versus the “designer” (16.5% vs. 22.8%), but the overall distribution of number of mailings of the questionnaire was not significantly different between the two approaches (Fig. 2). In contrast, larger differences were found in the efforts required to recruit participants by the CCS's age at survey (p = 0.0001) and sex (p = 0.027) (Figs. 3 and 4). Only 5.4% of the responses among survivors younger than 18 years of age occurred after one mailing and no follow-up calls, however, 48.2% were completed after two mailings. Among those 21 years of age or older, 35% of the responding cases required four or more mailings. By sex, men required more effort to recruit than women (29.3% required four or more mailings vs. 21.8% for women). No differences were found by ethnicity (data not shown).
FIG. 2.
Percentage distribution of numbers of calls and mailings made to recruit cases by formatting style (ns). ns, not significant.
FIG. 3.
Percentage distribution of numbers of calls and mailings made to recruit cases by age group of respondent at the time of survey (p = 0.0001).
FIG. 4.
Percentage distribution of numbers of calls and mailings made to recruit cases by sex of respondent (p = 0.027).
Discussion
We obtained a 50% response rate among CCS (61% participation rate) who were AYA survivors of pediatric cancer at the time of interview, which was due to an intensive recruitment strategy involving extensive telephone and mail follow-up, drop-by visits, tracing, use of incentives and lottery option, endorsement from physicians, Spanish translation, and multiple options for response (mail, online, telephone, and in person). In the future it may be helpful to also include email contact.28 With these efforts, our CCS respondents were representative of our population-based sample for cancer site, stage of disease, year of diagnosis, and age at survey response. As found in other studies, women and those living in higher SES areas were more likely to respond, as were NHW.12,13,29 Among our Hispanic participants, although they had a lower response rate than NHW, no association with response was found for any clinical or demographic variable, even though the SES distribution of Hispanics was highly skewed toward the lower quintiles. We believe that providing both the Spanish and English versions of the surveys (and contact by our bilingual interviewing staff) was helpful in achieving this result, even though relatively few participants actually responded in Spanish.
Our response rate was higher than other cancer registry-based studies involving cancer patients in the AYA age group at the time of survey. For example, the Adolescent and Young Adult-Health Outcomes and Patient Experience (AYA-HOPE) study, which surveyed patients diagnosed in the AYA age group (i.e., 15–39), achieved a 43% response rate.12 We also achieved a higher response rate than a study of AYA survivors of childhood cancer in Hawaii who were selected from a clinic10 where 29.8% responded, and higher than another study based on patients registered in a hospital-based tumor registry diagnosed within the past 10 years that had a 31% response rate.11
We employed several methods to increase response. Similar to the CCS Study, we used extensive tracing of nonrespondents20 and also used incentives, which have been shown to be effective in increasing response in multiple studies21 and may also increase nonresponse among those are less interested in the study topic.19 However, a review of studies of cancer patients found that financial incentives may not be as effective in this population,30 possibly due to greater trauma and emotional distress associated with being a cancer patient.31 Our timing of contact of survivors was at least 2 years after diagnosis, and, for most, they would have been in remission or past the acute effects of treatment at the time of the survey. The benefit of receiving an incentive was an important factor mentioned in our focus groups, and may be especially well received by patients in this age group. Our 61% participation rate among those we were able to contact may be indicative of the benefit of offering incentives. Other health-related studies, not involving cancer survivors, have found incentives, including lotteries, to be effective.32,33 Monetary incentives may be more effective when not conditional on response,32,34 however, we were not able to offer this option. One alternative may be to offer funds up front after subjects promise to participate after initial contact.
We initiated contact with cases by sending a postcard to identify incorrect addresses to alert them that a survey packet would be coming in the mail. Others have found that pre-notification increased response in some cases,32,35 but not in others.36 Another study found that pre-notification increased speed of response but not total response.37 Given our finding that “plain” versus “designer” versions did not result in a difference in overall response, but may have been related to a quicker initial response, these efforts may reduce cost of recruitment but may not affect final response rates.
We mailed additional copies of the questionnaires to nonrespondents and the study originated from a university; factors that have been shown to increase response.32,38 However, while the survey questions were not of a sensitive nature that may lower response,32 CCS may be reluctant to think about their future and continued use of follow-up care and this focus may have negatively affected response, especially among men and older survivors, who were further removed from their cancer treatment experience.
For adults, this mailed survey study design did not require a signed informed consent by the Institutional Review Board, which has been shown to result in a better response,39 but signed consent was required from parents for children younger than18 years of age. However, response in this age group was not negatively affected after follow-up phone calls were made to provide more information. The mandatory contact with parents of the younger survivors most likely resulted in the higher response rate to the parent's survey among parents of CCS in this younger age category versus parents of older CCS.
Some studies have tried to assess potential bias in nonresponse by comparing those who responded early versus those who responded late.16 In our study more effort was required to recruit older cases and men, and results of our study have shown that older age was associated with lower use of follow-up care.24 Thus, if the nonrespondents were those who were the least likely to be receiving care, then our survey estimates may be underestimating the extent of lack of follow-up care in this population.
Limitations of the study include the restriction to patients diagnosed at two major hospitals in Los Angeles with supportive physicians. Thus, our response rates may not be achieved among studies based on a complete sample of the entire county, without direct involvement/support of all physicians in the area. Also, the subject matter of the study may have been one that CCS did not wish to consider and other topics may be of more direct interest. However, due to selection of cases from a cancer registry we were able to assess factors associated with response and found our respondents to be highly representative of the sampled cases, especially among Hispanics. In addition, by identifying variables associated with response, these factors can be adjusted for in analyses.
In summary, use of a multi-method recruitment approach produced a higher response rate than some other studies of younger cancer survivors, even those based on more recently diagnosed cases (e.g., 43% for the AYA-HOPE study, which surveyed cases within 14 months after diagnosis12), but it required extra effort and persistence to reach these cancer survivors. Even though the AYA HOPE study utilized a similar approach including initial mailing and follow-up phone calls, it did not allow for the extended effort of calling and remailing materials that was able to be done in this study. We found that additional effort was required to recruit men and older survivors, and, had we ceased effort sooner, our final sample would have been less representative of these subgroups. Surveying parents based on this sampling method has some additional challenges due to the need to obtain permission to contact the parent if the survivor is 18 years of age or older. Nevertheless, limitations in the generalizability of the results can be identified.
Acknowledgments
This work was supported by the Whittier Foundation, grant 1R01MD007801 from the National Institute on Minority Health and Health Disparities of the National Institutes of Health and P30CA014089 and T32CA009492 from the National Cancer Institute of the National Institutes of Health. The collection of cancer incidence data used in this study was supported by the California Department of Public Health pursuant to California Health and Safety Code Section 103885; Centers for Disease Control and Prevention's (CDC) National Program of Cancer Registries, under cooperative agreement 5NU58DP003862-04/DP003862; the National Cancer Institute's SEER under contract HHSN261201000140C awarded to the Cancer Prevention Institute of California, contract HHSN261201000035C awarded to the University of Southern California, and contract HHSN261201000034C awarded to the Public Health Institute. The ideas and opinions expressed herein are those of the author(s) and endorsement by the State of California Department of Public Health.
Author Disclosure Statement
The authors declare no potential conflicts of interest.
References
- 1. Benetsky M, Burd C, Rapino M. Young adult migration: 2007–2009 to 2010–2012. Washington, DC: U.S. Census Bureau; 2015 [Google Scholar]
- 2. Kotaniemi JT, Hassi J, Kataja M, et al. . Does non-responder bias have a significant effect on the results in a postal questionnaire study? Eur J Epidemiol. 2001;17(9):809–17 [DOI] [PubMed] [Google Scholar]
- 3. Landier W, Wallace WH, Hudson MM. Long-term follow-up of pediatric cancer survivors: education, surveillance, and screening. Pediatr Blood Cancer. 2006;46(2):149–58 [DOI] [PubMed] [Google Scholar]
- 4. Reppucci ML, Schleien CL, Fish JD. Looking for trouble: adherence to late-effects surveillance among childhood cancer survivors. Pediatr Blood Cancer. 2017;64(2):353–7 [DOI] [PubMed] [Google Scholar]
- 5. Sieswerda E, Font-Gonzalez A, Reitsma JB, et al. . High hospitalization rates in survivors of childhood cancer: a longitudinal follow-up study using medical record linkage. PLoS One. 2016;11(7):e0159518. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Robison LL, Mertens AC, Boice JD, et al. . Study design and cohort characteristics of the Childhood Cancer Survivor Study: a multi-institutional collaborative project. Med Pediatr Oncol. 2002;38(4):229–39 [DOI] [PubMed] [Google Scholar]
- 7. Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012;307(17):1805–6 [DOI] [PubMed] [Google Scholar]
- 8. Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17(9):643–53 [DOI] [PubMed] [Google Scholar]
- 9. Ojha RP, Oancea SC, Ness KK, et al. . Assessment of potential bias from non-participation in a dynamic clinical cohort of long-term childhood cancer survivors: results from the St. Jude Lifetime Cohort Study. Pediatr Blood Cancer. 2013;60(5):856–64 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Wada RK, Glaser DW, Bantum EOC, et al. . Hawai'i's multiethnic adolescent and young adult survivors of childhood cancer: are their health behavior risks similar to state and national samples? Hawaii J Med Public Health. 2013;72(11):380–5 [PMC free article] [PubMed] [Google Scholar]
- 11. Husson O, Zebrack BJ. Perceived impact of cancer among adolescents and young adults: relationship with health-related quality of life and distress. Psychooncology. 2017;26(9):1307–15 [DOI] [PubMed] [Google Scholar]
- 12. Harlan L, Lynch C, Keegan TH, et al. . Recruitment and follow-up of adolescent and young adult cancer survivors: the AYA HOPE study. J Cancer Surviv. 2011;5:305–14 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Arora NK, Hamilton AS, Potosky AL, et al. . Population-based survivorship research using cancer registries: a study of non-Hodgkin's lymphoma survivors. J Cancer Surviv. 2007;1(1):49–63 [DOI] [PubMed] [Google Scholar]
- 14. Cunradi CB, Moore R, Killoran M, Ames G. Survey nonresponse bias among young adults: the role of alcohol, tobacco, and drugs. Subst Use Misuse. 2005;40(2):171–85 [DOI] [PubMed] [Google Scholar]
- 15. Fullam F, Vangeest J. Surveys of patient populations. In: Johnson TP. (Ed). Handbook of health survey methods. Hoboken, NJ: John Wiley and Sons, Inc.; 2015; pp. 561–83 [Google Scholar]
- 16. Perneger TV, Chamot E, Bovier PA. Nonresponse bias in a survey of patient perceptions of hospital care. Med Care. 2005;43(4):374–80 [DOI] [PubMed] [Google Scholar]
- 17. Keeter S, Miller C, Khout A, et al. . Consequences of reducing nonresponse in a national telephone survey. Public Opin Q. 2000;64:125–48 [DOI] [PubMed] [Google Scholar]
- 18. Gadkari AS, Pedan A, Gowda N, McHorney CA. Survey nonresponders to a medication-beliefs survey have worse adherence and persistence to chronic medications compared with survey responders. Med Care. 2011;49(10):956–61 [DOI] [PubMed] [Google Scholar]
- 19. Groves R. Nonresponse rates and nonresponse bias in household surveys. Public Opin Q. 2006;70(5):646–75 [Google Scholar]
- 20. Mertens AC, Walls RS, Taylor L, et al. . Characteristics of childhood cancer survivors predicted their successful tracing. J Clin Epidemiol. 2004;57(9):933–44 [DOI] [PubMed] [Google Scholar]
- 21. Edwards PJ, Roberts I, Clarke MJ, et al. . Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;(3):MR000008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Dillman DA. Mail and internet surveys: the tailored design method, 2nd ed. New York: Wiley; 2000 [Google Scholar]
- 23. Alsultanny Y, Alotaibi M. Evaluating the factors affecting on intension to use of e-recruitment. Am J Inf Sci Comput Eng. 2015;1(5):324–31 [Google Scholar]
- 24. Milam JE, Meeske K, Slaughter RI, et al. . Cancer-related follow-up care among Hispanic and non-Hispanic childhood cancer survivors: the Project Forward study. Cancer. 2015;121(4):605–13 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Liu L, Deapen D, Bernstein L. Socioeconomic status and cancers of the female breast and reproductive organs: a comparison across racial/ethnic populations in Los Angeles County, California (United States). Cancer Causes Control. 1998;9:369–80 [DOI] [PubMed] [Google Scholar]
- 26. Liu L, Cozen W, Bernstein L, Ross R, Deapen D. Changing relationship between socioeconomic status and prostate cancer incidence. J Natl Cancer Inst. 2001;93:705–9 [DOI] [PubMed] [Google Scholar]
- 27. Meeske KA, Sherman-Bien S, Hamilton AS, et al. . Mental health disparities between Hispanic and non-Hispanic parents of childhood cancer survivors. Pediatr Blood Cancer. 2013;60(9):1470–7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Haider S, Dodge LE, Brown BA, et al. . Evaluation of e-mail contact to conduct follow-up among adolescent women participating in a longitudinal cohort study of contraceptive use. Contraception. 2013;88(1):18–23 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Ekholm O, Gundgaard J, Rasmussen NKR, Hansen EH. The effect of health, socio-economic position, and mode of data collection on non-response in health interview surveys. Scand J Public Health. 2010;38(7):699–706 [DOI] [PubMed] [Google Scholar]
- 30. Vangeest JB, Johnson TP. Using incentives in surveys of cancer patients: do “best practices” apply? Cancer Causes Control. 2012;23(12):2047–52 [DOI] [PubMed] [Google Scholar]
- 31. Ransom S, Azzarello LM, McMillan SC. Methodological issues in the recruitment of cancer pain patients and their caregivers. Res Nurs Health. 2006;29(3):190–8 [DOI] [PubMed] [Google Scholar]
- 32. Edwards P, Roberts I, Clarke M, et al. . Increasing response rates to postal questionnaires: systematic review. BMJ. 2002;324(7347):1183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Gajic A, Cameron D, Hurley J. The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey. Eur J Health Econ. 2012;13(6):789–99 [DOI] [PubMed] [Google Scholar]
- 34. Rosoff PM, Werner C, Clipp EC, et al. . Response rates to a mailed survey targeting childhood cancer survivors: a comparison of conditional versus unconditional incentives. Cancer Epidemiol Biomarkers Prev. 2005;14(5):1330–2 [DOI] [PubMed] [Google Scholar]
- 35. Mitchell N, Hewitt CE, Lenaghan E, et al. . Prior notification of trial participants by newsletter increased response rates: a randomized controlled trial. J Clin Epidemiol. 2012;65(12):1348–52 [DOI] [PubMed] [Google Scholar]
- 36. Carey RN, Reid A, Driscoll TR, et al. . An advance letter did not increase the response rates in a telephone survey: a randomized trial. J Clin Epidemiol. 2013;66(12):1417–21 [DOI] [PubMed] [Google Scholar]
- 37. Koopman L, Donselaar LCG, Rademakers JJ, Hendriks M. A prenotification letter increased initial response, whereas sender did not affect response rates. J Clin Epidemiol. 2013;66(3):340–8 [DOI] [PubMed] [Google Scholar]
- 38. Nakash RA, Hutton JL, Jorstad-Stein EC, et al. . Maximising response to postal questionnaires—a systematic review of randomised trials in health research. BMC Med Res Methodol. 2006;6:5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Ziegenfuss JY, Burmeister KR, Harris A, et al. . Telephone follow-up to a mail survey: when to offer an interview compared to a reminder call. BMC Med Res Methodol. 2012;12:32. [DOI] [PMC free article] [PubMed] [Google Scholar]