Abstract
Cell phone surveys have become increasingly popular and researchers have noted major challenges in conducting cost-effective surveys while achieving high response rates. Previous work has shown that calling strategies that maximize both respondent contact and completed interviews for landline surveys may not be the most cost-effective for cell phone surveys. For example, Montgomery, et al. (2011) found important differences between landline and cell samples for best times to call and declines in contact rates after repeated dialing. Using paradata from the 2010 and 2011 National Flu Surveys (sponsored by the Centers for Disease Control and Prevention), we investigate differences in calling outcomes between landline and cell surveys. Specifically, we predict respondent contact and interview completion using logistic regression models that examine the impact of calling on particular days of the week, certain times of the day, number of previous calls, outcomes of previous calls and length of time between calls. We discuss how these differences can be used to increase the likelihood of contacting cooperative respondents and completing interviews for both sample types.
Keywords: call patterns, calling rules, National Flu Surveys, landline surveys, cell phone surveys, logistic regression
1. Introduction
Landline telephone surveys have been heavily used in the survey research field for decades. As a consequence, researchers have had ample time to identify practices that maximize efficiency and response rates. This long history has allowed for considerable observation and experimentation, which have informed refinements of calling and scheduling patterns. Cell telephone surveys are much newer and are still gaining in popularity. Researchers have begun the process of optimizing cell calling methods, but there is much work to be done. Common sense tells us that Americans use cell phones in very different ways than they use landline phones, and this may have important consequences in terms of the best ways to contact respondents. Previous work suggests that cell phones are typically personal devices that respondents have available nearly all of the time. Carley-Baxter, Peytchev, and Black (2010) found that that the majority of cell phones have individual users – less than 15% of cell respondents reported sharing the cell phone they were contacted on with another person. Previous work also suggests that cell phones make respondents available for large portions of the day, with more than 80% of cell users reporting that they keep their cell phones turned on all day (Carley-Baxter, et al., 2010, ZuWallack, 2009).
There are almost certainly differences in usage patterns between landline phones and cell phones that could be exploited to improve survey participation rates. For cost reasons, unraveling these differences and maximizing contact rates and efficiency can be especially important for cell surveys. Cell surveys tend to be more expensive than landline surveys due to lower response rates (ZuWallack, 2009). A logical starting point for cell calling patterns is to adapt existing methods from landline calling, and it is likely that many survey organizations have taken this approach. However, using nearly identical rules for both sample types may not be an optimal strategy. We look to expand previous work and identify differences that can be used to tailor rules specifically for cell sample.
There is quite a bit of previous research on the best times to make call attempts for landline surveys. Landline contact rates tend to be higher on week nights (after 5:00 pm) than they are during weekday mornings and afternoons, when many respondents may be at work (Brick, et al., 1996, Massey, et al., 1996, Montgomery, et al., 2011, Stec, et al., 2004). Weekend calling often results in higher contact rates than weekday daytime calling as well (Massey, et al., 1996, Stec, et al., 2004). There is less previous research available on the best times to call for cell phone surveys, but early work indicates that contact rates are more consistent across dialing times for cell phone surveys than they are for landline surveys (Brick, et al., 2007, Montgomery, et al., 2011, Yuan, et al., 2005). Previous work (Brick, et al., 2007) also notes that refusals are more prevalent for cell sample than landline sample.
For landline surveys, there is also some previous work addressing the optimal amount of time to allow between call attempts to the same telephone number. Results are somewhat mixed, and this may be because surveys vary in field period length and also in how frequently the same numbers are attempted. Stokes and Greenberg (1990) found that longer delays between calls were associated with higher contact rates. Their analysis compared relatively short delays, ranging from within two hours to two or more days. Sangster and Meekins (2004) found mixed results based on length of delay between calls, but their results are a bit hard to evaluate. The measure they included in their models was simply an average number of days between attempts for each sample line across all calls, and they do not provide a sense of the distribution of delays present in their particular survey. We are interested in comparing a larger range of delays for landline sample as well as cell sample. To our knowledge, the optimal length for delays between cell phone call attempts has not been investigated.
To summarize our objectives, we are interested in identifying the conditions for calling cell phone sample that yield the highest rates of cooperative contact with respondents. We explore which call times are best when making the first call attempts to each telephone number and the optimal amount of time to allow between call attempts. The purpose of this research is to build on previous work and continue to refine cell calling procedures based on observations from large volumes of cell call attempts.
2. Methods
Data for this paper were drawn from the National Flu Surveys conducted between November, 2010 through March, 2012. These were national random-digit-dial (RDD) surveys sponsored by the Centers for Disease Control and Prevention (CDC). The objective of these surveys was to monitor influenza vaccination rates, and they were conducted seasonally in November and March. Across the four waves of data collection that occurred within this time span, call attempts were made for over 1,000,000 landline sample lines and over 700,000 cell lines. The data of interest for this paper mainly come from paradata about each call attempt and outcome. One notable exception is respondent time zone, which is a piece of information about each sample line obtained along with the telephone numbers. We do not include any actual questionnaire response data in our analyses. We use paradata from all telephone numbers that received call attempts, not just those where contact was made or interviews were completed.
The outcome of primary interest for this paper is cooperative contacts with households. This includes a variety of scenarios where a human respondent is reached on a residential/personal line and is cooperative. This excludes refusals and hang-ups, as well as contacts with businesses. It includes completed interviews, completed screeners (whether the respondent ends up being eligible for the survey or not), situations where the respondent is busy but agrees to a callback or appointment, etc. We focus on this outcome because cooperative contact is a necessary precursor to household screening and interview completion, but it occurs more frequently than these later outcomes, and it is less sensitive to particular survey requirements and procedures (making it more generalizable to other types of surveys).
The analysis plan for this work included a descriptive analysis as well as logistic models and event history models. In the descriptive analysis, we compare landline and cell phone samples on their rates of cooperative contact, non-cooperative contact (refusals and hang-ups), and business contact. We also compare the outcomes of the first call attempts (“first dials”) made to both sample types based on the day of the week and time of the day of the call. To assess the impact of varying time delays between calls (“lag times”), we use logistic models predicting cooperative contact on the second call attempt based on lag time since the first attempt. Since the optimal delay between calls may vary depending on the previous call outcome, we build separate models based on the specific result of the first call (e.g., non-contact, answering machine/voicemail, refusal, or hang-up). We also address the overall impact of lag time in event history models that predict cooperative contact over the entire set of calls made to each sample line. The independent variables for these models include the day of week and time of day for each call, the days of lag time between calls, previous call outcomes, the month of data collection (November, which is during the flu season, vs. March, which is after flu season), and time zone.
3. Results
We address three main issues in the results. First, we describe first dial outcomes, reviewing the overall distribution and then looking more specifically at types of contact that occur between landline and cell sample lines. Second, we examine variations in first dial contact based on the day and time of the call. Third, we describe the impact of lag times between calls in models predicting cooperative contact with households.
3.1 First Dial Outcomes
For the descriptive analysis, we first look at the overall distribution of outcomes for first dial attempts for landline and cell sample lines. As seen in Table 1, the majority of these attempts do not reach a human respondent, but human contact is more likely on cell phones than landlines. This is probably because many people often carry their cell phones with them, even when away from home. Non-contact outcomes (which include ring no answers, busy signals, disconnected numbers, etc.) are most common for both sample types, and are especially common for first dial attempts to landline sample lines (56.5%, vs. 41.5% for cell lines). The second most prevalent category of outcomes is answering machines and voicemail messages. These occur at a higher rate for cell lines than landline lines (35.4% vs. 25.1%). Cell first dial attempts reach human respondents 23.1% of the time, while landline first dials only make human contact 18.4% of the time.
Table 1.
Percentage of All First Dials Resulting in Various Outcomes
| Type of Outcome | Landline | Cell |
|---|---|---|
| Non-contact | 56.5% | 41.5% |
| Answering machine/Voicemail | 25.1% | 35.4% |
| Human Contact | 18.4% | 23.1% |
3.2 First Dial Contact Types
Next, we examine the specific type of human contact that is occurring for both sample types. Table 2 shows that the increased contact for cell sample includes both cooperative and non-cooperative contact. Completed interviews are slightly more likely to occur on first dial attempts for landline lines than cell lines (1.9% vs. 0.7%). Respondents agree to callback and appointment requests slightly more frequently for cell lines than landline lines (7.1% vs. 5.8%). In our experience, roughly 2% of all first dial attempts to cell lines result in the line being screened out for reasons specific to cell dialing, but we still categorize these as cooperative contacts. One example of this is the situation where the respondent is a minor who indicates that no adults share the phone. Another example is when the respondent does not meet the National Flu Survey requirement of being a cell-only or cell-mainly user (instead, he/she has a landline telephone that is likely to be answered when someone is at home). A small number of cooperative contacts for both sample types are categorized as “other contact”. This category falls under 0.5% of all first dial attempts and includes households that require language line services, ineligible/out of scope outcomes, and cases where the respondent prefers to call in to the 1–800 line or receive a letter about the study before participating.
Table 2.
Percentage of All First Dials Resulting in Various Types of Human Contact
| Contact Type | Landline | Cell | |
|---|---|---|---|
| Cooperative | Completed Interview | 1.9% | 0.7% |
| Callback or appointment | 5.8% | 7.1% | |
| Cell-specific screen out | n/a | 1.9% | |
| Other contact | 0.4% | 0.5% | |
| Non-cooperative | Refusal | 4.0% | 4.6% |
| Hang-up | 4.5% | 7.9% | |
| Business | Business | 1.8% | 0.5% |
| Cooperative | No contact on first dial, but called our 1–800 line afterwards | 0.1% | 0.6% |
Table 2 also shows non-cooperative contacts that occur on the first dial. Refusals and hang-ups tend to happen more frequently for cell sample lines than landline lines (4.6% vs. 4.0% and 7.9% vs. 4.5%, respectively). More landline attempts result in identification of a business phone line than cell attempts (1.8% vs. 0.5%). Lastly, Table 2 includes an interesting type of contact that occurs after the first dial attempt. More than a half of a percentage point of cell sample first dial attempts (0.6%) result in a non-contact, followed by a respondent calling in to our 1–800 line. Many of these returned calls occur fairly quickly after our attempts, which has staffing consequences for our inbound dial handling.
3.3 First Dial Call Times
With the overall trends of human contact established between sample types, we next investigate how these rates vary by the time of the call. We group together all types of cooperative contact on first dial attempts and compare the rates by day of week and time of day in Table 3. For landline sample lines, cooperative contact increases steadily throughout the day on weekdays, from 7.1% of all morning dials to 8.8% of night dials1. Landline refusals and hang-ups also steadily increase over the course of the day. They start out lower than cooperative contacts in the morning at 6.7% of all first dials, and then outpace the cooperative contacts at night at 9.9%. On the other hand, the rate of landline business contacts decreases over the course of the day, from 3.4% of all morning dials to 0.9% of night dials. The landline calling patterns on Fridays are very similar to those observed during the rest of the weekdays, although cooperative contact during the night drops from 8.8% to 7.6%.
Table 3.
Percentage of All First Dials Resulting in Cooperative Contacts, Refusals/Hang-ups, and Business Contacts by Day of Week and Time of Day
| Day of Week | Time of Day | Landline | Cell | ||||
|---|---|---|---|---|---|---|---|
| Cooperative Contact | Refusal/Hang-up | Business | Cooperative Contact | Refusal/Hang-up | Business | ||
| Weekdays (Monday – Thursday) | Morning | 7.1% | 6.7% | 3.4% | 9.5% | 11.1% | 0.6% |
| Afternoon | 7.6% | 7.3% | 3.2% | 10.6% | 12.1% | 0.6% | |
| Evening | 8.4% | 8.7% | 1.5% | 10.7% | 12.8% | 0.4% | |
| Night | 8.8% | 9.9% | 0.9% | 9.7% | 12.5% | 0.3% | |
| Friday | Morning | 7.5% | 6.7% | 3.7% | 9.4% | 10.9% | 0.5% |
| Afternoon | 7.0% | 7.7% | 3.1% | 10.3% | 13.0% | 0.6% | |
| Evening | 8.2% | 9.0% | 1.4% | 10.3% | 12.9% | 0.4% | |
| Night | 7.6% | 9.3% | 0.7% | 9.9% | 13.1% | 0.3% | |
| Saturday | Morning | 6.6% | 7.0% | 1.3% | 10.6% | 12.2% | 0.1% |
| Afternoon | 7.7% | 8.0% | 1.1% | 9.8% | 13.5% | 0.3% | |
| Evening | 7.5% | 8.3% | 0.9% | 10. 7% | 13.5% | 0.4% | |
| Night | 8.2% | 9.7% | 0.7% | 9.8% | 13.0% | 0.4% | |
| Sunday | Morning | 8.3% | 9.1% | 0.7% | Omitted2 | ||
| Afternoon | 7.6% | 8.7% | 0.8% | 9.3% | 13.7% | 0.2% | |
| Evening | 8.4% | 9.5% | 0.8% | 10.0% | 14.4% | 0.2% | |
| Night | 8.7% | 10.1% | 0.5% | 9.4% | 12.5% | 0.3% | |
Note that the results for Sunday morning first dials to cell sample lines are omitted here because the sample sizes were very small.
The weekday patterns for cell sample lines are a bit different. Cooperative contact rates with cell respondents are consistently higher than with landline respondents, and are highest during the afternoon and evening (at 10.6% and 10.7%, respectively). They tend to be lower during the morning and night (at 9.5% and 9.7%, respectively). Cell refusals occur at higher rates than landline refusals, and increase from morning to evening (11.1% to 12.8%). The cell refusal rate stays relatively elevated in the night shift (12.5%). The rate of business contacts is consistently lower for cell sample lines than landline sample lines, and it decreases slightly over the course of the day (from 0.6% to 0.3%). The cell calling patterns on Fridays are similar to those observed during the rest of the weekdays, although refusals are a bit more likely during the night than they are during nights on other weekdays (13.1% vs. 12.5%).
For calls to landline sample lines on weekends, Saturday mornings had less cooperative contact (6.6%) and Sunday mornings had more cooperative contact (8.3%) than weekday mornings (7.1%). In general, the cooperative contact rates and refusal rates are slightly higher on Sundays than Saturdays. For cell sample lines, cooperative contact rates are slightly lower overall on Sundays than they are on weekdays and Saturdays. Refusal rates are slightly higher on the weekend, and on Sundays in particular, than they are on the weekdays. Sunday afternoons and evenings have particularly high refusal rates with 13.7% and 14.4% of all first dials resulting in a refusal or hang-up.
3.4 Lag Times Between Calls
To assess the impact of varying lag times between calls, we turn to logistic models predicting cooperative contact on second dials. We limit each of these models to a single sample type (landline vs. cell) and to a single first dial outcome (non-contact, answering machines/voicemail, refusal, or hang-up). The lag time between the first and second call attempts is the predictor of particular interest to us here. Table 4 shows the odds ratios for the non-contact and answering machine/voicemail outcome models. We do not report statistical significance of the model coefficients here because the sample sizes are so large as to make significance less meaningful. After a non-contact outcome on the first dial for landline sample lines, calls made six or more days later yield less cooperative contact than calls made after shorter two to three day lags. The pattern for cell sample lines is different, with lags of four to five days and six or more days yielding more cooperative contact than shorter two to three day lags. There are no large differences based on lag times after answering machine/voicemail outcomes.
Table 4.
Logistic Model Output, Predicting Cooperative Contact on Second Dial After Non-Contacts or Answering Machine/Voicemail Outcomes on First Dial3
| Lag Time (Reference Category = 2–3 days) | Sample Lines with Non-Contact Outcome on First Dial | Sample Lines with Answering Machine/Voicemail Outcome on First Dial | ||||||
|---|---|---|---|---|---|---|---|---|
| Landline Model (N=302,817) | Cell Model (N=109,000) | Landline Model (N=267,824) | Cell Model (N=240,346) | |||||
| Odds Ratio | 95% CI | Odds Ratio | 95% CI | Odds Ratio | 95% CI | Odds Ratio | 95% CI | |
| 0–1 day | 1.05 | 0.96 – 1.16 | 0.91 | 0.81 – 1.02 | 1.02 | 0.97 – 1.08 | 0.97 | 0.92 – 1.02 |
| 4–5 days | 0.96 | 0.91 – 1.01 | 1.34 | 1.23 – 1.45 | 1.04 | 0.99 – 1.10 | 0.99 | 0.93 – 1.05 |
| 6+ days | 0.85 | 0.81 – 0.90 | 1.15 | 1.07 – 1.23 | 0.98 | 0.93 – 1.02 | 1.01 | 0.96 – 1.07 |
These models also included the following predictors, which are eliminated here due to space concerns: day of week/time of day combination, month of survey (November or March), time zone of respondent.
Table 5 shows similar output for additional logistic models predicting cooperative contact on the second dial, but this time after a first dial refusal or hang-up. The lag categories used here are slightly different because our existing scheduling rules dictate longer lags after non-cooperative contacts, meaning there were not enough data points to justify categories for lags of zero to one day after these outcomes. For landline sample, we find no substantial differences based on lag time between calls. For cell sample, we find that longer lags of six or more days after first dial refusals are associated with higher levels of cooperative contact on the second dial than lags of zero to three days. After hang-ups, both lags of four to five days and six or more days are associated with higher levels of cooperative contact on the second dial than lags of zero to three days.
Table 5.
Logistic Model Output, Predicting Cooperative Contact on Second Dial After Refusal or Hang-up on First Dial 4
| Lag Time (Reference Category = 0–3 days) | Sample Lines with Refusal Outcome on First Dial | Sample Lines with Hang-up Outcome on First Dial | ||||||
|---|---|---|---|---|---|---|---|---|
| Landline Model (N=36,604) | Cell Model (N=22,427) | Landline Model (N=47,812) | Cell Model (N=52,611) | |||||
| Odds Ratio | 95% CI | Odds Ratio | 95% CI | Odds Ratio | 95% CI | Odds Ratio | 95% CI | |
| 4–5 days | 1.01 | 0.91 – 1.13 | 1.09 | 0.93 – 1.28 | 0.97 | 0.90 – 1.05 | 1.15 | 1.05 – 1.25 |
| 6+ days | 1.05 | 0.93 – 1.18 | 1.33 | 1.10 – 1.60 | 0.96 | 0.90 – 1.03 | 1.13 | 1.04 – 1.21 |
These models also included the following predictors, which are eliminated here due to space concerns: day of week/time of day combination, month of survey (November or March), time zone of respondent.
We are also interested in the overall likelihood of cooperative contact occurring during the life of a sample line, and how this is affected by the delays between calls. To assess this for both landline and cell phone cases, we construct two separate models. These models need to account for many factors. As we describe above, the timing of each call is important. The length of the lag between calls is another factor under consideration, but the effect of the lag time may vary based on how long particular sample lines have been active in the data collection period. That is, lags of certain lengths may be more or less appropriate early in cases’ dialing histories as compared with later in their histories. To account for these constraints, we use two multivariate event history models, which use proportional hazards assumptions. These models include a time-varying covariate for days of lag time between calls. In addition to this and the predictor variables used previously in the logistic models predicting second dial cooperative contact, these models include other new variables to account for events that dictate longer lags per our scheduling rules (e.g., whether each sample line ever had a refusal or hang-up in its history).
Figure 1 shows the predicted survival curves from the event history models. (Full model output is shown in the Appendix.) Again, the dependent variable is cooperative contact with households. Sample lines that never achieve cooperative contact are included from their first dial until the end of the data collection period. Those that do achieve cooperative contact are included in the model from the time of their first call attempt until their first cooperative contact occurs. Survival curves are often used to display the risk of negative outcomes occurring, but we use it here to illustrate the occurrence of a desired outcome. In effect, each sample line is “at risk” for experiencing cooperative contact until the survey is concluded or until contact occurs.
Figure 1.
Survival Curves Showing the Likelihood of Not Making Cooperative Contact with a Household Over the Life of a Sample Line5
The survival curves in Figure 1 represent the hazard of continuing to be called as an active sample line. The lines in the lowest position on each panel of Figure 1 represent lags of zero days. The initial declines on these curves (closest to the Y axes) represent first dial attempts, which have lags of zero days because there was no previous attempt. This indicates that cooperative contact rates are high on first dials compared with later dials. This trend is stronger in the cell model than the landline model. The continuations of the curves for lags of zero days represent scenarios when multiple calls occur on the same day, later in the data collection period. This situation typically arises when we make a call attempt that does not result in a cooperative contact, but the respondent calls in to our 1–800 line later during the same day. These curves drop off sharply, showing that these situations typically end with cooperative contact and happen earlier in the calling history as opposed to later. It is not surprising that respondents who quickly call us back on our 1–800 line tend to be cooperative (as opposed to calling in to refuse or to report that we have reached a business). The steeper curves for the cell model indicate that this cooperative calling in behavior is more common and quicker for cell cases than landline cases.
The remaining curves in Figure 1 show the likelihood of remaining as an active sample line given longer lag times between calls. The majority of these show a slow decrease over time, which indicates that the cumulative amount of cooperative contact increases as the data collection period goes on. In general, these remaining curves are arranged in decreasing order from top to bottom. This shows that the shorter lags tend to be associated with more cooperative contact, while longer lags tend to be associated with less cooperative contact. This trend becomes more pronounced the longer the sample line remains active. For cell sample, lags of one day and two days are associated with more cooperative contact than those with longer lags, which are grouped together at the top of the figure. This suggests that lags of one to two days are better than longer lags for cell sample, but beyond the two day mark, there is little difference.
4. Discussion
The results shown here confirm previous indications that contacting respondents on cell phones can be challenging. Cell phone sample has higher overall contact rates than landline sample, but this contact includes more refusals and hang-ups than landline sample. To maximize efficiency, we see that weekday evenings are optimal times to make first dial attempts for both sample types. However, it may not be practical to schedule so many first dials during the same time period. Landline sample also performs well during weekday nights and Sunday evenings and nights. For cell sample, weekday afternoons are another good time to make first dials. Due to higher refusal rates, it may be best to avoid first dials to cell sample on the weekends, especially on Sundays.
In terms of lag times, we see that cell sample seems to “clear” faster and has a shorter window of opportunity. Shorter lag times seem more beneficial overall for both sample types, but there is less variation on this for cell sample. Longer lag times may be better after refusals, especially for cell sample lines. Longer cooling off periods may be particularly helpful for cell phone lines because they are more likely to be answered by the same respondent again (as noted by Brick, et al., 2007), in contrast to landline, where other household members may answer the phone on later call attempts.
In terms of next steps for our research, we plan to continue investigating additional interactions between particular outcomes and lag times, using event history models and latent class analysis. In particular, we would like to investigate refusals, hang-ups, business contacts, household screening, and household interview completion in more detail.
Appendix
Model Output From Multivariate Event History Models Predicting Cooperative Contact with Households
| Landline Model | Cell Model | |||||
|---|---|---|---|---|---|---|
| Hazard Ratio | Std. Error | Hazard Ratio | Std. Error | |||
| Novembera | 0.968 | 0.005 | *** | 1.006 | 0.005 | |
| Ever answering machineb | 0.536 | 0.003 | *** | 0.204 | 0.001 | *** |
| Ever refusedb | 0.815 | 0.006 | *** | 0.254 | 0.003 | *** |
| Ever hung upb | 0.834 | 0.006 | *** | 0.484 | 0.004 | *** |
| Ever called 1–800 numberb | 3.425 | 0.067 | *** | 2.701 | 0.037 | *** |
| 1 day lagc | 0.162 | 0.003 | *** | 0.029 | 0.000 | *** |
| 2 day lagc | 0.097 | 0.002 | *** | 0.018 | 0.000 | *** |
| 3 day lagc | 0.080 | 0.002 | *** | 0.013 | 0.000 | *** |
| 4 day lagc | 0.048 | 0.001 | *** | 0.008 | 0.000 | *** |
| 5–6 day lagc | 0.025 | 0.001 | *** | 0.005 | 0.000 | *** |
| 7–10 day lagc | 0.027 | 0.001 | *** | 0.003 | 0.000 | *** |
| 11 or more day lagc | 0.005 | 0.000 | *** | 0.001 | 0.000 | *** |
| M-Th morningd | 1.194 | 0.014 | *** | 1.163 | 0.014 | *** |
| M-Th afternoond | 0.875 | 0.006 | *** | 0.967 | 0.008 | *** |
| M-Th nightd | 1.083 | 0.007 | *** | 0.880 | 0.007 | *** |
| Fri morningd | 1.290 | 0.030 | *** | 1.514 | 0.033 | *** |
| Fri afternoond | 0.701 | 0.010 | *** | 0.903 | 0.012 | *** |
| Fri eveningd | 1.359 | 0.017 | *** | 1.060 | 0.012 | *** |
| Fri nightd | 1.398 | 0.017 | *** | 0.971 | 0.013 | * |
| Sat morningd | 1.306 | 0.017 | *** | 1.467 | 0.027 | *** |
| Sat afternoond | 1.266 | 0.014 | *** | 1.117 | 0.016 | *** |
| Sat eveningd | 0.813 | 0.016 | *** | 0.812 | 0.019 | *** |
| Sat nightd | 0.708 | 0.015 | *** | 0.676 | 0.016 | *** |
| Sun morningd | 1.370 | 0.025 | *** | 1.127 | 0.024 | *** |
| Sun afternoond | 0.817 | 0.009 | *** | 0.675 | 0.010 | *** |
| Sun eveningd | 1.162 | 0.015 | *** | 0.860 | 0.016 | *** |
| Sun nightd | 0.936 | 0.012 | *** | 0.699 | 0.012 | *** |
Month of data collection, with reference category of March.
Indicates the sample line ever had this outcome over all calls, with reference category of No.
Day of lag time since previous call, with reference category of zero days.
Day and time of call, with reference category of weekday (Monday through Thursday) evening.
p < 0.05
p < 0.01
p < 0.001
Footnotes
We define the times of day as follows: morning is 9:00 am through noon, afternoon is noon through 5:00 pm, evening is 5:00 pm through 7:00 pm, and night is 7:00 pm through 9:00 pm. All of these times are in local respondent time.
The full output for these models is shown in the Appendix. Predictors include the number of days of lag time between calls (shown here), as well as the month of data collection (November or March), whether each sample line ever had particular outcomes including reaching an answering machine, having a refusal, having a hang-up, or having the respondent call in to the 1–800 line, and day and time of the call.
References
- Brick JM, Allen B, Cunningham P, Maklan D. Outcomes of a Calling Protocol in a Telephone Survey. Proceedings of the Survey Research Methods Section of the American Statistical Association. 1996:142–249. [Google Scholar]
- Brick JM, Brick PD, Dipko S, Presser S, Tucker C, Yuan Y. Cell Phone Survey Feasibility in the U.S.: Sampling and Calling Cell Numbers Versus Landline Numbers. Public Opinion Quarterly. 2007;71(1):23–39. [Google Scholar]
- Carley-Baxter LR, Peytchev A, Black MC. Comparison of Cell Phone and Landline Surveys: A Design Perspective. Field Methods. 2010;22(1):3–15. [Google Scholar]
- Massey JT, Wolter C, Wan SC, Liu K. Optimum calling patterns for random digit dialed telephone surveys. Proceedings of the Section on Survey Research Methods, American Statistical Association. 1996:485–490. [Google Scholar]
- Montgomery R, Zeng W, Khare M, Wooten K. A Comparison of Cell and Landline Dialing Patterns. Paper presented at the 65th Annual Conference of the American Association for Public Opinion Research; Phoenix, AZ. 2011. [Google Scholar]
- Sangster RL, Meekins BJ. Modeling the Likelihood of Interviews and Refusals: Using Call History Data to Improve Efficiency of Effort in a National RDD Survey. Proceedings of the Survey Research Methods Section of the American Statistical Association. 2004:4311–4317. [Google Scholar]
- Stec JA, Lavrakas PJ, Shuttles CW. Gaining Efficiencies in Scheduling Callbacks in Large RDD National Surveys; Proceedings of the Survey Research Methods Section of the American Statistical Association; 2004. pp. 4430–4437. [Google Scholar]
- Stokes SL, Greenberg BS. A Priority System to Improve Callback Success in Telephone Surveys. Proceedings of the Survey Research Methods Section of the American Statistical Association. 1990:742–747. [Google Scholar]
- Yuan AY, Allen B, Brick JM, Dipko S, Presser S, Tucker C, Han D, Burns L, Galesic M. Surveying Households on Cell Phones - Results and Lessons. Proceedings of the Section on Survey Research Methods, American Association of Public Opinion Research. 2005:4021–4027. [Google Scholar]
- ZuWallack R. Piloting Data Collection via Cell Phones: Results, Experiences, and Lessons Learned. Field Methods. 2009;21:388–406. [Google Scholar]