The Effects of a Between-Wave Incentive Experiment on Contact Update and Production Outcomes in a Panel Study

Abstract

Since 1969, families participating in the U.S. Panel Study of Income Dynamics (PSID) have been sent a mailing asking them to update or verify their contact information in order to keep track of their whereabouts between waves. Having updated contact information prior to data collection is associated with fewer call attempts, less tracking, and lower attrition. Based on these advantages, two experiments were designed to increase response rates to the between-wave contact mailing. The first experiment implemented a new protocol that increased the overall response rate by 7 – 10 percentage points compared to the protocol in place for decades on the PSID. This article provides results from the second experiment which examines the basic utility of the between-wave mailing, investigates how incentives affect article cooperation to the update request and field effort, and attempts to identify an optimal incentive amount. Recommendations for the use of contact update strategies in panel studies are made.

1. Overview

Since 1969, families participating in the U.S. Panel Study of Income Dynamics (PSID) have been sent a mailing asking them to update or verify their contact information in order to keep track of their whereabouts between waves of data collection. Having updated contact information prior to data collection is associated with fewer call attempts and refusal conversion efforts, less tracking, and lower attrition (Budowski and Scherpenzeel 2005; Couper and Ofstedal 2009; Calderwood 2010; Ribisl et al. 1996). Two experiments were designed with the goal of increasing response rates to this between-wave contact mailing. The first experiment in 2008 increased the overall response rate to the between-wave contact update by approximately 7 – 10 percentage points by manipulating the design of the mailing, its timing and frequency, whether a study newsletter was also mailed, and use of prepaid versus postpaid incentives (McGonagle et al. 2011). As reported in this article, a second experiment was undertaken in 2010 to determine the overall utility of the contact update mailing. Families were randomly assigned to a condition in which no mailing was sent in order to examine the effect on data collection effort. A second goal was to examine the effect of providing an incentive on response to the mailing as well as on data collection effort, and to identify an optimal incentive amount. The experimental design randomly assigned PSID panel members to one of four treatment conditions: no mailing, a mailing that included no incentive, or a postpaid incentive of either $10 or $20 in exchange for returning the contact update postcard with a verification or update of their address and/or telephone number.

The goal of this article is to describe the effects of the new experiment on the provision of contact update information as well as its impact on data collection outcomes in Wave 37 of the PSID (2011). The differential responsivity to the incentives for returning the postcard by key socioeconomic characteristics of sample members is examined, and information is provided on the cost-effectiveness of the between-wave mailing.

2. Background

A substantial literature exists on the benefits of providing incentives in exchange for participation in surveys (e.g., Laurie et al. 1999; Laurie and Lynn 2009; Singer et al. 1999a; Singer et al. 1999b; Singer 2002). Research based on longitudinal studies generally finds evidence of a positive association between incentive amount and response rate (Fumagalli et al. 2010; Laurie 2007; Martin et al. 2001; Rodgers 2002) and data collection efforts such as number of calls to complete a case (James 1997; Rodgers 2002). Additional research has documented enduring effects of incentives provided at one wave that persist over time, reducing cumulative nonresponse over multiple waves (Laurie 2007; Mack et al. 1998; Scherpenzeel et al. 2002). Moreover, several studies find that economic characteristics of sample members, such as having low income or being in poverty, increase responsivity to financial incentives, possibly due to the greater value that incentives provide for those who need it most (Laurie 2007; Martin et al. 2001; Mack et al. 1998; Ryu et al. 2006).

In contrast to the large literature on incentives and survey completion, the experimental research on alternative between-wave contact strategies is limited. Fumagalli and colleagues (Fumagalli et al. 2010) found a positive association between incentive amount and return of an address change card. In our 2008 study (McGonagle et al. 2011), we attempted to improve upon the between-wave contact strategy used for many decades in the PSID, which has provided families with a postpaid incentive of $10 in exchange for their return of a contact update postcard. We found no effect of a prepaid versus postpaid $10 incentive on return of the contact update postcard, but we did find a positive effect of a second mailing sent to a subgroup of families who did not return the postcard in response to the first mailing.

The current study drew on these results and the existing literature to design an experiment examining the utility of a four-decade long practice in the PSID of providing incentives for a between-wave contact update mailing. The key research questions that this experiment seeks to answer are the following:

1. Does the between-wave contact update mailing lead to improved response rates to the contact update request and reductions in data collection effort, such as number of calls during production, the tracking of families, and provision of an interview?

2. Does the offer of a conditional incentive compared to no incentive yield similar improvements?

3. Does doubling the incentive from the amount used in many prior waves increase response rates and reduce data collection effort?

4. Do these treatment effects differ by key characteristics of panel members, including being young and having low income?

3. Methods

3.1. Sample and Dataset

The sample of families included in this experiment consisted of 8,690 families from the Panel Study of Income Dynamics (PSID), who had provided a completed interview in the 2009 wave and were eligible to be followed in the 2011 wave. The PSID is a longitudinal study of a nationally representative sample of U.S. families that began in 1968 (see McGonagle et al. 2012 for more information). The original 1968 PSID sample was drawn from two independent samples: a nationally representative sample of roughly 3,000 families designed by the Survey Research Center at the University of Michigan (the “SRC sample”) and an over-sample of roughly 2,000 low-income African American families from the Survey of Economic Opportunity (the “SEO sample”). In 1997, 511 families who had immigrated to the U.S. after 1968 were added to enhance the national representativeness of the sample. The study is a genealogical panel, following the original 1968 panel members and the offspring in these households that grow up and form their own economically independent families (known as “split-offs”). Thus the active panel includes related families, with up to four generations of families participating in a given wave. Data were collected annually from 1968 to 1997, and have been collected biennially from 1999 through the most recent wave in 2011. The mode of data collection is via computer-assisted telephone interview (CATI) for approximately 97.5% of panel members, with computer-assisted personal interviewing (CAPI) for the balance. Data collection occurs over a nine-month period from March to December in odd-numbered calendar years.

3.2. Experimental Design and Assignment to Treatment Conditions

As shown in the first row of Table 1, approximately ten percent (n = 876) of the families were randomly assigned to receive no contact update mailing; ten percent (n = 940) were assigned to receive a mailing but no incentive ($0); approximately forty percent (n = 3,460) were assigned to receive a mailing and a $10 postpaid incentive (which has been the PSID status quo amount for many waves) and about forty percent were assigned to receive a mailing and a $20 postpaid incentive (n = 3,414). Because families in the PSID are related and may communicate with each other, all related families received assignment to the same treatment condition, which is why the proportion of families assigned to each condition is slightly variable. Table 1 also provides information on key characteristics of the families as of 2009, with approximately 28 percent reporting a high likelihood of moving, about 33 percent having a household head younger than age 35, and about 71 percent requiring four or more calls to finalize their 2009 interview. Approximately 63 percent of the families are from the SRC sample, 29 percent from the low-income SEO over-sample, and seven percent from the immigrant sample. About 6.5 percent of the families in total are “split-offs” who became eligible to participate in the PSID for the first time in the 2009 wave.

Table 1.

Sample sizes and characteristics of families for each treatment condition group (n = 8,690)

	Treatment conditions
	Mailing sent			No mailing sent
	$0	$10	$20
Number of families	940	3,460	3,414	876
Characteristics of families in 2009 (%)
Likelihood of moving before 2011:
Probably or definitely	28.5	27.3	28.4	28.0
None or uncertain	67.9	69.2	69.2	68.9
Missing	3.6	3.5	2.4	3.1
Family income is less than or equal to bottom quintile	19.6	20.3	19.4	21.9
Number of calls in 2009 to finalize the case:
1 – 3	29.3	29.2	28.0	30.5
4 or more	70.7	70.8	72.0	69.5
Age of head of family:
Less than 35	35.0	32.4	31.8	32.2
35 or older	65.0	67.6	68.2	67.8
Sample types:
SRC	63.4	59.0	65.8	64.3
SEO	28.4	34.2	27.3	25.6
Immigrant	8.2	6.8	6.8	10.2
Split-off family	6.4	6.7	6.7	6.0

The contact update mailing consisted of a black and white postcard labeled with the last known address and telephone numbers of the respondent (see Appendix I). The postcard included prepaid postage to cover the cost of returning the mailing. The text of the mailing sent to families in the $10 and $20 incentive conditions read: “Here’s how to receive your $10 ($20) check!” The text of the mailing sent to families in the no incentive condition read: “Here’s how to update your contact information!” Families in the three mailing conditions were sent the initial contact update mailing in August 2010, approximately seven months before the start of 2011 production interviewing. Drawing on the success of including a second mailing for families who did not return the postcard in the 2008 experiment, the current experiment re-mailed the materials to all families who did not respond to the initial mailing within two months.

Appendix I.

3.3. Measures

3.3.1. Outcome Measures

Results for two sets of outcome measures are reported. The first set is referred to as “contact update outcomes” and captures information about respondent behavior in returning the contact information postcard or providing a new telephone number. Analysis of this set is based on the group of families in the three treatment conditions that were all sent the contact update mailing (n = 7,814). Contact update outcomes consist of “postcard return” and “new telephone number”. “Postcard return” is defined as a dummy variable coded as ‘yes’ = 1 for instances when the respondent returned the postcard and either verified the current information, updated the information, or fixed the current information (e.g., changed ‘street’ to ‘avenue’), and ‘no’ = 0 for instances when no postcard was received back from the respondent. The overall rate of postcard returns was 68.5 percent, with 7.8 percent of returners providing a new address, 19.4 percent providing an address fix, and 72.8 percent verifying their contact information.

“New telephone number” is defined as a dummy variable coded ‘yes’ = 1 for instances of receiving a postcard back from the respondent that includes the provision of a new telephone number that was not available in the prior wave, and ‘no’ = 0 when a new telephone number was not provided. New telephone numbers were provided by 13.5 percent of the postcard returners. Having an accurate telephone number at the beginning of field production is important because the PSID completes more than 97% of its interviews over the telephone.

A second set of measures was designed to assess the effect of the treatment conditions on subsequent data collection outcomes. Analysis of this set is based on families in all four of the treatment conditions, including those sent the contact update mailing and the condition that was not sent the mailing. “Total calls in 2011” is a continuous variable from 1 – 270 capturing the full range of telephone calls that were made to reach the final disposition of the case during the 2011 field effort. The average number of calls is 15.3 with a median of 6.0. An indicator variable for “high calls in 2011” was constructed to examine whether the treatment conditions were related to a reduction in the number of high effort cases. Cases requiring calls above the mean of 15 were coded as ‘yes’ = 1 and those below the mean were coded as ‘no’ = 0. More than 15 calls were required to finalize 24 percent of all cases. A third variable captures information about whether the case needed to be tracked during the field effort, either due to the telephone number on record not being answered or being out of order, or the respondent having moved with no forwarding contact information (‘yes’ = 1 / ‘no’ = 0). Nearly 22 percent of the families required tracking in 2011. Finally, the effect of the experiment on the overall 2011 interview response rate was also examined (‘yes’ = 1 / ‘no’ = 0). The response rate for the families in this study during the 2011 wave was 94.3 percent.

3.3.2. Measures to Assess Differential Impact

Four key measures were constructed to investigate whether the treatment conditions had differential effects for key subgroups on contact updates and production outcomes. These variables were obtained from the public use data available at the PSID website (http://psidonline.org; Panel Study of Income Dynamics 2009) and were also included in all the models as covariates. First, a dummy variable for likelihood of moving over the next couple of years as reported in 2009 was created with ‘no’ = 0 indicating “none” or “uncertain” likelihood of moving and ‘yes’ = 1 indicating a “definite” or “probable” move. A second indicator variable for whether the total family income reported in 2009 was equal to or below the bottom quintile in family income was created (‘yes’ = 1/‘no’ = 0). A third indicator variable was constructed to signify whether the age of the household head was under 35 (‘yes’ = 1) or 35 and older (‘no’ = 0). Finally, a variable for sample type was included that coded families who were part of the original low-income SEO oversample as ‘yes’ = 1 and families who were part of the original SRC national probability sample as ‘no’ = 0.

3.3.3. Control Variables

Three additional variables known to be related to the outcome measures, thereby increasing the precision of the estimated effects of the treatment conditions, were included in all models. In addition to the variables for likelihood of moving, family income, age of household head, and sample type, these included whether the family was a member of the immigrant refresher sample, and whether the family was new to the study as of 2009. A dummy variable coded families who came from the 1997/1999 post-1968 immigrant refresher sample as ‘yes’ = 1 and those who did not come from this sample as ‘no’ = 0. A second dummy variable was included that coded families who were designated as new split-off families during the 2009 wave as ‘yes’ = 1 and non-split-off families as ‘no’ = 0. Finally, in order to control for high effort cases, an indicator variable for “high calls in the prior wave (2009)” was included, with cases requiring 4 or more calls to complete coded as ‘yes’ = 1 and those below 4 coded as ‘no’ = 0.

3.3.4. Analysis Strategy

The first step in the analysis is to provide a description of the bivariate results of the effects of the treatment conditions on the contact update and 2011 production outcomes. The second step describes results from multivariate regression analyses predicting each of the outcome measures from the four treatment conditions. Logistic regression was used to model the contact update outcomes: “postcard return,” and “new telephone number;” as well as the 2011 production outcomes: “tracking,” “high calls,” and “completed an interview.” Because of its skewed distribution as a count variable, Poisson regression was used to model “total calls.”

4. Results

4.1. Contact Update and Production Outcomes after Treatment: Bivariate Results

Table 2 displays two sets of results. The top panel presents the proportion of families who were sent the mailing that returned the postcard and provided a new telephone number by each of the incentive conditions. The bottom panel includes families who were sent the mailing as well as those assigned to the no mailing condition, and presents the proportion requiring tracking, needing high calls, completing an interview, and the average number of telephone calls to finalize the case, by each of the treatment conditions. All of the statistical tests of mean differences reported in the table control for the comparison-wise error rate using Duncan’s multiple range test (Duncan 1955).

Table 2.

Contact update and production outcomes after treatment for each treatment condition group

	Treatment conditions
		Mailing sent		No mailing sent (D) n = 876
	$0 (A) n = 940	$10 (B) n = 3,460	$20 (C) n = 3,414
Contact update outcomes (n = 7,814)
% Returning postcard	59.3 (b**,c**)	68.1 (a**,c*)	71.5 (a**,b*)	NA
On first mailing	42.8 (b**,c**)	54.7 (a**,c*)	59.0 (a**,b*)
On second mailing	16.5 (b**,c**)	13.4 (a**)	12.5 (a**)
% Providing new telephone number	10.5 (b**,c**)	13.6 (a**)	14.2 (a**)	NA
On first mailing	6.8 (b**,c**)	9.8 (a**)	10.7 (a**)
On second mailing	3.7	3.8	3.4
Production outcomes in 2011 (n = 8,690)
% Tracking required	21.3	22.7	20.6 (d*)	24.4 (c*)
% High calls required to finalize case	22.4 (d*)	24.7	23.8	26.7 (a*)
Average number of calls to finalize case	14.9	15.4	15.3	15.9
% Completing interview	94.7	94.4	94.4	92.9

Notes: Duncan’s multiple range test was used to test mean differences.

Superscripts a, b, and c designate the groups for which the mean is being tested, with the level of statistical significance of the difference indicated by **p < = 0.01, *p < = 0.05.

Contact update outcomes

There are three findings of note in the top half of Table 2. First, among families sent the mailing, there is a consistent positive effect of the non-zero incentive conditions on contact outcomes. The $10 and $20 incentive conditions resulted in a significantly greater proportion of respondents returning the postcard (68.1% and 71.5%, respectively) and providing a new telephone number (13.6% and 14.2%, respectively) compared to the $0 condition (59.3% returning the postcard and 10.5% providing a new telephone number). Second, respondents in the $20 incentive condition had significantly higher rates of postcard returns compared to those in the $10 condition (71.5% vs. 68.1%, respectively) but did not have significantly higher rates of new telephone number provision. The same pattern of results was observed for the percentage of respondents in each treatment condition who responded to the first mailing, with a greater proportion returning the postcard in the $10 and $20 conditions compared to the $0 condition, and those in the $20 condition returning the postcard at a significantly higher rate than those in the $10 condition. Similarly, there was no difference in the rate of new telephone number provision in the first mailing between respondents in the $10 and $20 conditions, but both groups responded to the first mailing with significantly more new telephone numbers than those in the $0 condition.

Third, the second mailing, during which families who had not responded within two months of the initial mailing were re-sent the request for contact update information, was successful in substantially increasing the initial postcard return rate for each incentive group. The second mailing was particularly effective for families in the $0 incentive condition, increasing the postcard return rate in this group by 38 percent (from 42.8% to 59.3%). Families in the $10 and $20 conditions returned the postcard from the second mailing at a significantly lower rate than those in the $0 condition (13.4% and 12.5% vs. 16.5%, respectively), but at rate that was still substantial – adding about 13 percentage points to the overall return rate for both groups. The significantly higher responsivity to the second mailing of families in the no incentive condition compared to those in the $10 and $20 conditions by approximately three percentage points may simply be due to the greater pool of families in the no incentive condition who were sent a second mailing, given their higher rate of nonresponse to the initial mailing. While there were no statistically significant differences in the rate of new telephone number provision across the three groups in response to the second mailing, it generated a substantial increase in new telephone numbers for each incentive group. Again, the second mailing was particularly effective for the $0 incentive group, raising the overall new telephone number rate by more than 54 percent.

Production outcomes

Examination of the 2011 production outcomes in the bottom panel of Table 2 – which also includes respondents in the “no mailing” treatment condition – indicates that the mailing had beneficial effects on field effort. Respondents in the no mailing condition had significantly poorer results on two of the four production outcomes compared to respondents in the mailing conditions. Tracking rates in this group were significantly higher compared to respondents in the $20 condition (24.4% vs. 20.6%). Compared to families in the $0 condition, those in the no mailing condition were significantly more likely to require a high number of calls to finalize their case.

4.2. Effects of Treatment Conditions on Contact Update Outcomes: Multivariate Results

Multivariate regression models were estimated to examine the effects of the treatment conditions on the contact update outcome measures described above. Logistic regression was used to estimate models for obtaining a postcard return and a new telephone number. Each of the models included the set of control variables described earlier.

Table 3 presents the results of the effects of the treatment conditions on contact update outcomes for the three groups that were sent the mailing. The results from the main effect models are consistent with the bivariate results in Table 2 in showing that respondents in the $10 and $20 conditions returned the postcard and provided a new telephone number at significantly greater rates than those in the $0 condition. Respondents in the $20 condition also returned the postcard at a significantly higher rate than those in the $10 condition; there were no significant differences between these groups in the provision of new telephone numbers (results not shown). Interestingly, the control variables had quite different effects on the outcomes as well. Likely movers, families requiring four or more calls to finalize the case, younger heads, and the low-income SEO oversample all had significantly lower rates of postcard returns. The opposite pattern was observed in the models predicting the provision of new telephone numbers, with these characteristics all significantly predicting higher rates, most likely due to the association of these socioeconomic attributes with more frequent changes in telephone numbers. Families from the immigrant refresher sample and those who were newly split off each had significantly lower rates of postcard returns.

Table 3.

Effects of treatment conditions on contact update outcomes for families sent a mailing: (n = 7,814)

	Contact update outcomes
	Returned postcard:		Obtained new telephone number:
	Main effect model OR	Interaction model OR	Main effect model OR	Interaction model OR
Treatment conditions
Mailing sent
$0 (reference)
$10	1.54**	1.11	1.35**	1.09
$20	1.76**	1.26	1.45**	1.13
Control variables
Likelihood of moving before 2011:
Probably or definitely	0.71**	0.57**	1.75**	1.03
None or uncertain (reference)
Missing	0.63**	0.63**	1.01	1.00
Family income
Lowest quintile	0.99	0.75	1.68**	1.78*
Highest 4 quintiles (reference)
Number of ’09 calls to finalize the case:
1 –3 (reference)
4 or more	0.48**	0.48**	1.17*	1.17*
Age of head of family:
Less than 35	0.62**	0.50**	1.33**	1.31
35 or older (reference)
Sample types:
SRC (reference)
SEO	0.53**	0.41**	1.39**	1.30
Original PSID sample (reference)
Immigrant sample	0.42**	0.42**	1.22	1.22
Non split-off family (reference)
Split-off family	0.63**	0.62**	0.86	0.86
Interaction terms
Likely movers
$10 incentive * Probably or definitely		1.24		1.61
$20 incentive * Probably or definitely		1.30		1.98**
Age of head of family:
$10 incentive * Less than 35		1.26		1.08
$20 incentive * Less than 35		1.36		0.95
Family income
$10 incentive * Lowest quintile		1.61*		0.88
$20 incentive * Lowest quintile		1.18		1.01
Sample type:
$10 incentive * SEO sample		1.29		1.12
$20 incentive * SEO sample		1.38		1.01
Mean of dependent variable	0.685		0.135

OR = odds ratio.

^**p < = 0.01,

^*p < = 0.05.

In order to test the differential impact of the incentives, a second set of multivariate models included simultaneous terms for interactions between each of the mailing conditions and characteristics of families, including: likely movers, young age of household head, very low family income, and membership in the original low-income SEO oversample. The models included the same control variables included in the earlier models. As shown in Table 3, families with very low income were more responsive to the incentive conditions than higher income families, with those in the $10 group 61 percent more likely to return the postcard than those in higher income quintiles. The financial incentives did not lead to a higher rate of postcard return for families as a function of likelihood of moving, age of household head, or sample membership. Results from the interaction model for provision of a new telephone number found that likely movers who were sent $20 were particularly responsive to the financial incentives compared to non-movers, with the odds of new telephone number provision higher by nearly 100 percent for those in the $20 condition. The financial incentives did not generate higher rates of new telephone number provision for families as a function of age of household head, family income, or sample membership.

4.3. Effects of Treatment Conditions on Production Outcomes: Multivariate Results

Logistic regression was used to examine the effects of the treatment conditions on production outcomes in 2011, including “tracking required,” “high calls,” and “completed an interview.” Poisson regression was used to estimate the effects of the total number of calls. Each model included the set of control variables included in Table 3. Because results from initial regression models found no discernible differences between the incentive amounts on production outcomes, the results presented in Table 4 are based on models that estimate the overall effect of the mailing, collapsed across all three mailing conditions ($0, $10, $20) compared to the no mailing condition, on each of the production outcomes. There was an overall significant effect of the mailing, regardless of incentive amount, compared to no mailing on three of the production outcomes, such that the mailing significantly reduced the odds of tracking, being in the high call group, and needing a high number of total calls to finalize the case.

Table 4.

Effects of treatment conditions on production outcomes for all families: (n = 8,690)

	Production outcomes in 2011
	Tracking required: OR	Total calls: Poisson b	High calls: OR	Completed interview: OR
Treatment conditions
Mailing sent($0, $10, $20)	0.82*	−0.06**	0.84*	1.29
No mailing sent (reference)
Control variables
Likelihood of moving before 2011:
Probably or definitely	1.28**	0.02**	1.05	1.28*
None or uncertain (reference)
Missing	1.25	−0.01	1.08	0.78
Family income
Lowest quintile	1.25**	−0.24**	0.76**	0.71**
Highest 4 quintiles (reference)
Number of ’9 calls to finalize the case:
1 – 3 (reference)
4 or more	2.48**	0.89**	4.66**	0.46**
Age of head of family:
Less than 35	1.58**	0.26**	1.45**	1.16
35 or older (reference)
Sample types:
SRC (reference)
SEO	1.61**	0.13**	1.27**	1.31**
Original PSID sample (reference)
Immigrant sample	1.41**	0.16**	1.63**	0.99
Non split-off family (reference)
Split-off family	1.44**	0.15**	1.36**	0.80
Mean of dependent variable	0.219	15.3	0.243	0.943

OR=odds ratio.

^**p < = 0.01,

^*p < = 0.05.

The control variables had strong effects on each of the production outcomes. Having four or more calls to finalize the case in 2009 was a significant and positive predictor of each indicator of high data collection effort in 2011. Young age of household head and being a member of the original low-income oversample, the immigrant refresher sample, or a split-off family were respondent characteristics that each predicted increased odds of tracking, needing high calls to complete the case, and a higher average number of total calls. As would be expected, being a likely mover significantly predicted higher odds of tracking and higher total calls. Low family income had mixed effects on production outcomes, increasing the odds of tracking and nonresponse, yet predicting fewer overall total calls on average, and reducing the odds of needing a high number of calls to finalize the case.

Several models were estimated to examine the differential impact of the incentive amounts by likelihood of moving, being in the lowest quintile of income, age of household head, and membership in the low-income SEO oversample. In contrast to the findings for contact update outcomes, no evidence was found for differential effects of the incentive amounts on any of the production outcomes by these family characteristics.

5. Discussion

The goals of the current study were to examine the effects of an incentive on cooperation with the request for between-wave contact information, to determine the optimal incentive amount, and to document the effectiveness of the contact update mailing used in PSID since its inception for data collection outcomes. An important caveat to note at the outset is that the experiment evaluated in this study was conducted in the PSID, which has a two-year period between interviews, making the generalization of results to surveys that have different lengths of time between interviews, such as commonly used annual surveys, uncertain. Four findings from this study are discussed below.

First, we found positive effects of the experimental manipulations of assignment to the mailing conditions on the contact update outcomes

The conditions that included a financial incentive yielded greater cooperation with the request for updated contact information than the no incentive condition. Furthermore, there was a higher rate of postcard return and new telephone number provision for those families receiving $20 compared to those receiving $10, but the magnitude of this difference was fairly small, and does not justify the provision of double the incentive amount for return of the contact information. Moreover, multivariate analyses documented that some socioeconomic characteristics of families – including being very low income and having a high likelihood of moving – were associated with elevated responsivity to the incentives.

The beneficial effects of the incentives on cooperation with the contact update request is consistent with other research identifying positive effects of incentives on survey response rates (e.g., Singer 1999; 2002). Nonetheless, more than half of the families in the no incentive group returned the postcard (59 percent). This high response rate among the no incentive group may reflect the loyalty and commitment of the families in the study, and possibly an expectation of receiving future incentives given the consistency of receiving them for past participation in the PSID. This hypothesis is consistent with other research demonstrating the persistence of the positive effects of incentives over waves of panel studies (Mack et al. 1998; Scherpenzeel et al. 2002). It is also worth noting that analyses examining the lagged effects on 2011 outcomes of the treatment conditions to which families were assigned in the first (2008) experiment found that those families assigned to the prepaid incentive condition had significantly higher rates of postcard returns in 2011 than those in the postpaid condition – despite the fact that postcard returns in 2009 were generally unaffected by whether the family was prepaid or postpaid in 2008 (see McGonagle et al. 2011). How long this effect persists will be a topic for additional research, as will the examination of whether in future waves there are lagged positive effects among families receiving $20 in the current experiment, as well as lagged negative effects among families who received no incentive.

Second, we demonstrated the overall utility of the between-wave contact update mailing, finding positive effects on key indicators of data collection effort

In multivariate models, families that were randomly assigned to be sent the mailing required less tracking and fewer telephone calls to finalize the case. We generated a basic estimate of the cost effectiveness of the mailing by calculating the approximate point at which it was beneficial in terms of its cost savings in number of calls. We first estimated the approximate cost of the mailing, the bulk of which was due to the incentive payments made to the families who provided updated contact information in the $10 and $20 incentive conditions (approximately $98,000). Across the families that were sent the mailing, the cost was $12.50 per family ($98,000/7,814 families). We then estimated the predicted reduction in calls as a consequence of the mailing compared to no mailing and found it was about 1 call. Thus, across the sample of families sent the mailing, there was a savings of approximately 7,814 calls (i.e., 1.0 calls × 7,814 families). While costs of calls vary, our estimate is that an average call during 2011 data collection costs less than $12.50, suggesting that this design was not cost effective in terms of reducing calls. However, it is likely that there are other potential cost savings that are more difficult to assess, including the resolution of cases earlier in the field period, reductions in tracking effort, the confidence instilled in interviewers who have updated information prior to making a call, and the goodwill generated by the mailing – all of which may be nontrivial factors that contribute to its net effectiveness. Future research devoted to developing cost modeling strategies to evaluate field effort would be very useful to survey practitioners who wish to evaluate interventions designed to improve data collection operations.

Third, there were no differential effects of the financial incentive conditions on production outcomes in multivariate models

Families who were sent the mailing had reduced field effort, regardless of the incentive amount, or whether or not they were promised any incentive. This suggests that it is the mailing per se – and not the particular incentive amount – that may be of most value in achieving positive effects on production outcomes, although it is not known at this time whether these incentive amounts will have lagged effects on future data collection outcomes. Worthy of future study is the identification of the exact mechanism through which the mailing affects production outcomes. It may simply be that the updated contact information makes it easier for interviewers to locate and contact families, resulting in lower rates of tracking and fewer calls. Additionally, the mailing itself may underscore the legitimacy and value of a survey, and evoke principles of social exchange and reciprocation – mechanisms that, it is speculated, also underlie the positive effects of other respondent materials, such as letters sent in advance of data collection (de Leeuw et al. 2007). Furthermore, for a panel survey with a lengthy time span between interviews, the mailing may provide a connection to the study from one wave to the next, and in doing so, heighten the effects of familiarity, which have well documented links to positive affect and other positive outcomes (Bornstein 1989; Lee 2001).

Fourth, certain socioeconomic and demographic characteristics of respondents were associated with an elevated responsivity to the incentives

The finding that low income families who received incentives had higher rates of postcard return compared to higher income families is consistent with other research identifying a greater impact of financial incentives on survey participation by those in poverty and with a low income (James 1997; Mack et al. 1998; Ryu et al. 2006). The current study also finds that incentives increase the odds that likely movers will provide a new telephone number. These results support the use of tailored protocols to avoid a potential source of nonresponse bias, such as determining the incentive amount based on a family’s income level, or targeting special incentives at individuals who report in a prior wave that they are likely to relocate in return for maintaining contact information. However, there was no evidence that these differential effects carried over to lower rates of tracking, calls to complete the case, or a higher response rate. In line with an economic exchange model (see Singer 2002), it may be that these differential effects are fleeting and dissipate once the “transaction” of providing the updated information and receiving the financial incentive has been completed.

In conclusion, the results from the current study, along with those from the first experiment, support the ongoing use of a contact update mailing to keep track of families between waves. The most effective approach is one that includes a re-mail to nonresponders. With regards to amount and type of incentive, the two studies together suggest that a postpaid incentive works as well for most panel members as one that is prepaid. In the current study, the $10 and $20 postpaid incentives yielded greater benefits for contact update outcomes than the no incentive condition. While the $20 postpaid incentive produced significantly higher postcard returns and new telephone numbers than the $10 incentive, the difference in effect sizes was small. Moreover, there were no differential effects of the incentive amounts on production outcomes. The results tentatively suggest that the amount of the incentive is less important than sending a mailing between waves as a way to maintain updated contact information as well as to sustain a connection with panel members. However, it is possible that this result is specific to the study design of the PSID and should be replicated in other studies. Moreover, research will be conducted to examine possible lagged effects of the incentive conditions on contact update and production outcomes in the next wave, in order to provide additional information about optimal incentive amounts in panel studies.