Factors associated with response to a follow-up postal questionnaire in a cohort of American Indians

Abstract

Objective

The aim of the study was to collect pilot data on response rates to a follow-up postal questionnaire in a cohort of American Indians living in the Southwestern United States. We tested the effect of questionnaire length on response.

Methods

Cohort members were American Indian adults aged 18 and over who completed the baseline study visit. Study participants (N=1587), cohort members who completed the baseline study visit during the first year of enrollment, were randomized into two groups to receive either an 18-page or 3-page follow-up postal questionnaire. Data were collected between October 2005 and March 2006.

Results

The response rates after two questionnaire mailings and a reminder postcard were significantly higher for the short versus the long (56.2%, 48.1% p=<0.01) questionnaire. Being female and being aged 50 or older were associated with returning a completed questionnaire. A reminder postcard and second mailing improved response by 11.7% and 13.4% respectively.

Conclusions

These results show that a postal questionnaire can be used in a cohort of American Indians living in the Southwest, but suggest that questionnaires should be short and repeat mailings are needed.

Introduction

Achieving a high response rate to a postal follow-up questionnaire in a cohort has two components: 1) maintaining contact with the participants and 2) for those contacted, maximizing participation (Hunt and White, 1998). Evaluation of potential barriers to questionnaire completion and strategies to improve response to postal questionnaires are needed to plan successful follow-up data collection.

The Navajo Education and Research Towards Health (EARTH) study is part of a multi-centered study funded to test the feasibility of establishing and following a cohort of American Indian and Alaska Native (AIAN) people. Past experience with conducting cohort studies in an AIAN population is limited. There are many reasons AIAN communities are not enthusiastic about participating in research including lack of trust in researchers due to past exploitation and abuse from research studies; individuals may not feel as if they are stakeholders in the research, i.e. they are not getting anything out of it; lack of understanding most researchers have for American Indian culture; and lack of a long standing relationship with the community (Davis and Reid, 1999; Sambo, 2001) Other potential barriers to achieving a high response rate to a postal follow-up questionnaire include: 1) the preponderance of postal boxes versus street addresses; 2) mobility of the population; 3) age of the population; 4) isolated and remote geographic location of some communities; 5) economic disparities in some communities (Choudhary, 2000); 6) the wide range of educational level (Choudhary, 2000); 7) scarcity of telephones making follow-up calls difficult (Choudhary, T; White et al., 1997); 8) variable frequency of mail pick-up (White et al., 1997); and 9) language barriers in some communities (White et al., 1997).

The goals of the present study were to collect pilot data on response rates to a postal follow-up questionnaire among individuals who enrolled in a cohort study on the Navajo Nation and to identify factors that might increase response to future follow-up questionnaires mailed to the entire cohort.

Methods

Study background

Detailed methods have been described elsewhere (Edwards et al., 2007; Slattery et al., 2007, 2008; Edwards et al., 2008). Briefly, Navajo EARTH study participants are 18 years of age or older, self-identify as American Indian or Alaska Native, have given informed consent, understand English or Navajo, and are eligible to receive care at the Indian Health Service. Participants enrolled in the cohort by coming to a study visit site to provide informed consent; complete study questionnaires including diet history and physical activity; and have blood pressure, blood lipids, and body size measurements taken. The Institutional Review Boards of the University of Utah, the Indian Health Service, and the Navajo Nation approved the study. Local health boards and chapters within local health boards also approved and supported the study.

Participants were asked to provide contact information at the baseline study visit. The contact information included name; spouse or partner name; maiden name (women); phone, including home, cell, message, work, and other; street address or physical location of residence for participants with a postal box; name, address, and phone for two friends or relatives; usual medical care provider; and Social Security number. This information was stored in the study tracking system. Initial follow-up contact was made with participants through a study newsletter. Addresses for participants whose newsletters were returned by the post office with change of address information were updated in the tracking system and a newsletter was resent. Participants whose newsletters were returned by the post office without change of address information were put into tracing. Attempts were made to call all telephone numbers, contacts, and obtain updated address information from medical records where the medical record consent was signed. If a new address was obtained, the tracking system was updated and the newsletter resent.

Follow-up questionnaires and data collection

Two follow-up questionnaires were developed. Each questionnaire included a one page medical history questionnaire asking the participant to update his or her health information. The short questionnaire also included the short format International Physical Activity Questionnaire (Booth, 2000) making it three pages in length, while the long questionnaire included a 17 page physical activity log developed for the Navajo EARTH Study physical activity validation study, making it 18 pages in length. Both follow-up questionnaires were in English only.

Participants were selected for the pilot follow-up study if their baseline study visit had been completed over 1 year prior to the mailing of the follow-up questionnaire (Fig. 1). All participants who completed a study visit between March 2004 and October 2004, N=1587, were selected. Participants were randomized into two groups using a computerized random-number table by the study statistician. One group received the long questionnaire, N=794, while the other group received the short questionnaire, N=792. The questionnaires were mailed in October, 2005. The study design did not allow for staff and participants to be blinded.

Fig. 1.

Participant flowchart. This figure summarizes the flow of participants through the study as recommended by the CONSORT statement, Navajo Education and Research (EARTH) Study, 2005–2006.

Following the recommendations of the total design method for implementing mail surveys (Dillman, 1978) a cover letter and stamped return envelope were included with the questionnaire mailout packet, a post card reminder was sent after the first mailing, a letter and second questionnaire were mailed to all nonrespondents approximately 1 month after the post card was sent. The recommended mailing intervals were adjusted due to the potential delays in picking up mail. We included a 5$ incentive with the first mailing and a sharpened pencil with each mailing (White et al., 2005).

Study variables and data analysis

Demographic data including age, education, employment status, marital status, and income were collected at the baseline study visit. Other variables collected or calculated from the baseline study visit included: sex, number of years at current residence; language spoken at home; identification with non-Native culture; Diet History Questionnaire calorie outlier; perceived health status; the 12-item Short Form health survey questionnaire (SF-12) summary scores (the Mental Health Component Summary (MCS) score and the Physical Health Component Summary (PCS) score) (Ware et al., 1996); number of medical conditions reported; body mass index; risky behaviors (seatbelt and helmet use and drinking while driving); cigarette smoking status; and parity (women).

Study variables relating to participant contact were generated from the tracking system and included: signed medical records release; provided telephone number at baseline study visit (home and any other, cell and any other but home, other only, no phone); had postal box address; provided Social Security number (SSN); and provided contact information such as name and phone number for at least one friend or relative.

Response rates (RR) were calculated using the “RR1 formula for mailed surveys to specifically named persons” recommended by the American Association For Public Opinion Research (2006).

Statistical methods

We calculated the overall response rates for the two questionnaire length groups and compared using chi-square tests. We evaluated the influence of other characteristics including sex and age.

Analysis was performed using Cox regression. Predictors of completion were determined in a univariate model stratified for each questionnaire type. Multivariate analysis was performed using backward stepwise Cox regression, adjusted for questionnaire type, age, provided SSN, and signed medical consent to determine which set of variables best predicted completion of the postal follow-up questionnaire. SAS 9.1 (SAS Institute Inc., Cary, NC) was used for descriptive statistics and related tests, and Intercooled Stata 8.0 (Stata Corporation, College Station, TX) for tests of proportions and statistical analysis of predictors.

Results

Of the 1587 participants selected for the pilot, one was determined to be ineligible because of death. Overall, more women than men were mailed a questionnaire (63.9% versus 36.1%), 49.8% were under age 40, 75.1% had completed high school or beyond, 44.2% were currently employed, and 69.6% reported speaking their “Native” language, that is, the language of their American Indian or Alaska Native tribe, or both “Native” language and English at home. (Table 1) Only 6.7% of participants had a mailing address that was not a post office box. The consent to review medical records was signed by 91.3% of participants, 83.5% provided a SSN, 63.1 % had a “home” telephone number (versus a cell/other phone only or no phone), and 87.8% gave a telephone number for at least one contact. The characteristics of study participants receiving the long and short follow-up questionnaires were similar, although participants receiving the long questionnaire were younger (p=0.01), were more likely to provide an SSN (p=0.01), and signed the medical consent (p=0.05) more often.

Table 1.

Characteristics of participants in the follow-up postal questionnaire pilot study, Navajo EARTH Study, 2005–2006.

	Long (N = 794)		Short (N = 792)		p	Total (N = 1586)
	No.	%a	No.	%a		No.	%a
Sex					0.47
Male	280	35.3	293	37.0		573	36.1
Female	514	64.7	499	63.0		1013	63.9
Age					0.01
18–29	236	29.7	204	25.8		440	27.7
30–39	194	24.4	156	19.7		350	22.1
40–49	183	23.0	221	27.9		404	25.5
50–59	118	14.9	145	18.3		263	16.6
60+	63	7.9	66	8.3		129	8.1
Education					0.67
Less than high school	197	24.8	193	24.4		390	24.6
High school or equivalent	213	26.8	228	28.8		441	27.8
Beyond high school	382	48.1	368	46.5		750	47.3
Other	2	0.3	3	0.4		5	0.3
Current employment					0.78
Currently employed	338	42.6	363	45.8		701	44.2
Not currently employed	220	27.7	205	25.9		425	26.8
Student	76	9.6	66	8.3		142	9.0
Retired	37	4.7	34	4.3		71	4.5
Disabled	13	1.6	14	1.8		27	1.7
Homemaker	90	11.3	96	12.1		186	11.7
Other	19	2.4	14	1.8		33	2.1
Post office box					0.93
Address is post office box	740	93.2	739	93.3		1479	93.3
Other address	54	6.8	53	6.7		107	6.7
Years at address					0.66
Less than 1 year	93	11.7	82	10.4		175	11.0
1–5 years	199	25.1	185	23.4		384	24.2
6–10 years	100	12.6	108	13.6		208	13.1
11+ years	401	50.5	411	51.9		812	51.2
Language at home					0.21
Native	83	10.5	78	9.8		161	10.2
English	248	31.2	215	27.1		463	29.2
Both	458	57.7	484	61.1		942	59.4
Other	1	0.1	5	0.6		6	0.4
Contact name					0.43
No	53	6.7	61	7.7		114	7.2
Yes	741	93.3	731	92.3		1472	92.8
Contact phone					0.16
No	88	11.1	106	13.4		194	12.2
Yes	706	88.9	686	86.6		1392	87.8
Signed medical consent					0.05
No	58	7.3	80	10.1		138	8.7
Yes	736	92.7	712	89.9		1448	91.3
Gave social security number					0.01
No	112	14.1	150	18.9		262	16.5
Yes	682	85.9	642	81.1		1324	83.5
Diet history questionnaire outlier					0.60
No	608	76.6	614	77.5		1222	77.0
Yes	165	20.8	156	19.7		321	20.2
Household participants					0.36
One	440	55.4	457	57.7		897	56.6
More than one	354	44.6	335	42.3		689	43.4
Phone					0.31
Home phone	518	65.2	483	61.0		1001	63.1
Cell only	98	12.3	104	13.1		202	12.7
Other phones	146	18.4	164	20.7		310	19.5
No phone	32	4.0	41	5.2		73	4.6

^aPercentages do not add up to 100 due to item nonresponse.

Table 2 summarizes response outcomes by questionnaire length and mailing along with the final outcome. As shown, the response rates were significantly higher for the short questionnaire versus the long (56.2%, 48.1% p<0.01). This difference remained significant when adjusted for age and other covariates. The response rate was also significantly higher for the short questionnaire versus the long after the initial mailing. However, upon each subsequent mailing there was no difference in the proportion of questionnaires returned by questionnaire length with an additional 11.7% returning a completed questionnaire after mailing the reminder postcard and 13.4% after mailing a second questionnaire.

Table 2.

Effect of questionnaire length on response rates to the follow-up postal questionnaire, Navajo EARTH Study, 2005–2006.

	Long		Short		pa
	Total (N = 794)		Total (N = 792)
	No.	%	No.	%
Final response status
Questionnaire received
After mailing 1b	189	23.4	240	30.3	<0.01
After reminder postcardc	92	11.6	93	11.7	0.38
After mailing 2d	101	13.1	112	14.1	0.20
Final disposition
Nonresponders
Bad addresse	35	5.2	41	6.1
Deceased	0	0	1	0.1
Returned refused	5	0.6	0	0
Not returned	372	46.1	305	37.6
Total not complete	412		347
Responders
Total completed	382	48.1	445	56.2	<0.01
Response rate		48.1		56.2

^aTwo-sided p-value from two-sample test of proportions comparing long to short.

^b776 long and 767 short questionnaires mailed with incentive.

^cReminder postcards mailed to 577 long and 513 short group nonresponders with good addresses.

^d485 long and 420 short questionnaires mailed to those with good addresses.

^eIncludes 18 long and 25 short questionnaires never mailed due to bad address and those returned without updated follow-up address.

When subject characteristics were considered as predictors of questionnaire response, females were 35% more likely to return the long questionnaire (p=<0.01) and 32% (p=<0.01) more likely to return the short questionnaire (data not shown). In the multivariate model, older age and parity were associated with response for women, and older age, not always using a seatbelt, and a higher score on the MCS of the SF-12 were associated with response for men (data not shown).

Discussion

The results of the study showed that for this population, receiving a shorter questionnaire did increase response. As the overall response rate was 52.1% after two follow-up mailings to non-responders, a reminder postcard and then a second questionnaire, more intensive follow-up would be necessary to achieve higher response rates.

While a focus of the main study was to address issues of lack of trust in research that exists when doing research with American Indians, many of the other potential barriers to response to a postal follow-up questionnaire, e.g. scarcity of telephones, or remote locations, could not be directly addressed in the design of pilot study. However, some of these barriers might have had an impact on response. For instance 93.3% of participants had a post office box making it impossible to know how many of the 677 non-respondents actually opened the questionnaire and decided not to return it. Due to the distance many participants have to travel to their local postal box, other household members could have picked up the mail and potentially the participant might not have ever received the questionnaire. Certified mail might help address this barrier, but due to the travel distance to a post office it would be hard for most participants to sign for mail.

Language is another barrier that might have had an impact on the response to the follow-up questionnaire. It is unlikely that this is the case however. During the first year of data collection, we “deferred” participants who needed their study visit conducted only in Navajo as the translations were not complete. Thus, all the participants in the pilot could read and understand English.

Conclusions

The results of this study show that a postal questionnaire can be used in a cohort of American Indians living in the Southwest, but suggest that future mailings should be kept short. In addition, repeat mailings to increase the likelihood that the postal questionnaire will be received and opened by the participant are important in this population. Although the study was conducted in American Indians living in the Southwest, it is reasonable that these findings may have implications for participant follow-up in other populations.