Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jan 1.
Published in final edited form as: Int J Soc Res Methodol. 2020 Sep 29;24(2):181–202. doi: 10.1080/13645579.2020.1824626

Interaction Before and During the Survey Interview: Insights from Conversation Analysis

Nora Cate Schaeffer 1,2
PMCID: PMC8570387  NIHMSID: NIHMS1634499  PMID: 34744481

Abstract

Conversation analysts have described the formation of actions and the sequential organization of talk in a wide variety of contexts and offer resources that can be used to study interaction in the interview. Conversational practices are relevant before and during the survey interview: First, the process of recruiting sample members to become respondents provides a site in which an analysis of actions can be applied with a measurable outcome – the effects on participation. Second, once the interview and the task of measurement begins, conversational practices intersect with – and sometimes disrupt -- the “paradigmatic” question-answer sequence, complicating our notions of what it means to “measure” a construct in interaction. CA provides insights into interaction at both these tasks. This paper selectively reviews work that uses conversation analysis to understand the survey interview and offers some new applications (e.g., examining answers to yes-no questions).

Keywords: Interviewer-respondent interaction, Conversation analysis, Yes-no questions, Survey recruitment

Introduction

The work of designing and implementing a survey has been characterized as consisting of two broad dimensions: who is the survey about (i.e., representation) and what is the survey about (i.e., measurement) (Groves et al. 2009). Representation and measurement are two principal classes of error in the Total Survey Error (TSE) framework, which organizes sources of survey error. In interviewer-administered surveys, interviewers both recruit sample members and administer instruments. As a result, errors of representation and measurement originate in the interaction between the interviewer and sample member. By studying this interaction, we aim to both understand causes of error and identify ways to reduce it. Much of what we know about how sample members decide to participate or respondents answer questions has been learned by watching and listening to the interaction between interviewers and sample members. For example, researchers have examined how characteristics of the interviewer as an interactional partner influence the decision to participate (e.g., Oksenberg and Cannell 1988; Jackle et al. 2011; Groves et al. 2008; van der Vaart et al. 2005). Survey researchers have also long used methods of interaction behavior coding to study interaction during the interview to evaluate interviewers (Cannell, Lawson, and Hausser 1975) and survey questions (e.g., Oksenberg, Cannell, and Kalton 1991; van der Zouwen and Dijkstra 2002; Smyth and Olson 2018).

These studies of how a variable, such as the acoustic properties of the interviewer’s voice, are associated with an outcome, such as completing an interview, have made important contributions. For example, with respect to recruitment, findings about the impact of the interviewer’s voice suggest that there are few associations between acceptance of the request to be interviewed and the many operationalizations of “acoustic properties” that have been examined (e.g., Groves et al. 2008; van der Vaart et al. 2005), although if the pitch of the interviewer’s greeting is high compared to that of other interviewers, the odds of participation are lower (Schaeffer et al. 2018; see also Oksenberg and Cannell 1988, p. 266). Similarly, ratings of the qualities of the interviewer’s voice that might make them a more or less attractive interactional partner (such as “friendliness”) have not been found to be consistently associated with success in recruiting sample members (e.g., Oksenberg & Cannell 1988; see also van der Vaart et al. 2005). Studies of interaction have also contributed to our understanding of the process of measurement. For example, if a question projects a “yes” or “no” answer but provides the interviewer a set of ordered categories in which to record answers, the frequency of follow-up by interviewers (and presumably interviewer variance) is higher than with properly formed questions (e.g., Smit, Dijkstra, and van der Zouwen 1997; Smyth and Olson 2018).

To identify important variables (e.g., acoustic properties or the response format of a question) and decide how to operationalize them, researchers draw on theory, but they also use direct observations of interaction or reports about the interaction, say from interviewers. When researchers’ variables and operationalizations accurately capture the underlying structure and content of the interaction, findings -- including null findings – should be more decisive. An important resource for survey researchers trying to understand the components and mechanics of talk and interaction is provided by the methods and findings of conversation analysis.

Conversation analysis (CA) is an approach to studying talk in interaction that assumes that “shared methods of reasoning and action” result in an order to talk and interaction (Stivers and Sidnell 2013). CA provides analytic methods for describing that order and the core structures of talk. These core structures include turn design, turn constructional units and transition-relevant sites, recipient design, preference structure, repair, sequence organization, and action formation (e.g., Stivers and Sidnell 2013). Although the specific role of these structures in our examples is not discussed in detail, these structures ground much of the analysis summarized here.

The review extends a fruitful existing literature that applies CA methods to the analyses of the constrained interactions of the standardized survey interview (e.g., Suchman and Jordan 1990; Schaeffer 1991; Maynard and Schaeffer 1997; Maynard et al. 2002). CA methods can be used to describe, for example, what “rapport” might look like as an interactional phenomenon (Garbarski, Schaeffer, and Dykema 2016). These methods can also identify actions and sequential structures that can be located in the interview, so that their consequences can be studied. Because conversation analysis requires close examination of the details of interaction, the variables and measures developed in this way are empirically grounded.

Drawing for the most part on previous studies, this paper argues for the importance of CA for problems of interest to survey researchers by reviewing examples of past contributions and suggesting directions for the future. The discussion is organized according to the TSE structure of “representation” and “measurement.” I review some ways that CA has contributed to the study and the practice of surveys by examining respondent and interviewer interactions both before and during the interview.

The conversational practices identified by conversation analysis are relevant before and during the survey interview in different ways: Before the interview, the process of recruiting sample members to become respondents provides a site in which an analysis of actions and conversational practices can be applied with a measurable outcome – the effects on participation. During the interview, the task of measurement begins; conversational practices intersect with or disrupt the “paradigmatic” question-answer sequence (Schaeffer & Maynard 1996), complicating our notions of what it means to “measure” a “variable” using interaction. As in recruitment, some studies of interaction within the interview provide an outcome, such as reliability (e.g., summary in Schaeffer and Dykema 2011a).

Two themes underlie this review, which is from the perspective of a survey researcher:

  • What have survey researchers learned by using the methods and findings of conversation analysis that might have been harder to see without them?

  • Do things we have learned using CA help us with the projects of recruiting sample members and standardized measurement in large production surveys with large numbers of interviewers?

Most of the analyses, examples, and transcripts presented here are from two sources that my colleagues and I have studied, the Wisconsin Longitudinal Study (WLS) and the Continuous National Survey (CNS). Our studies of recruitment in the WLS used a case-control design in which we transcribed recruitment calls with 257 matched pairs of sample members who declined and accepted the request for an interview. The calls were made as part of a 2004 effort to re-interview members of the cohort that graduated from high school in Wisconsin in 1957, so all sample members were older adults at the time of the interviews. (See Schaeffer et al. 2013 and 2018 for details of that study design.) Additional studies of the WLS have examined interaction in several sections of the interview (e.g., Dykema et al. 2018; Garbarski, Schaeffer, and Dykema 2016; Schaeffer and Maynard 2008). The CNS consisted of interviews with a general population sample in the United States, sampled using a random digit dialing (RDD) procedure. This study was the basis for early studies of recruitment (e.g., Maynard and Schaeffer 1997), and a set of transcribed interviews was provided to participants in a workshop, “Interviewer-Respondent Interaction in the Standardized Survey Interview,” sponsored by the VU University Amsterdam (see Maynard et al. 2002). In addition, both our work and the other studies referred to here also analyzed other surveys, such as the Current Population Survey (e.g., Schaeffer and Maynard 2008).

Before the interview: Recruitment

This section illustrates two different types of contributions of CA to the study of recruiting sample members. In the first illustration, we examine the opening turns of the recruitment call, and in the second, we review two studies of the scripting of the interviewer’s “introduction.” The most detailed studies of interaction when sample members are recruited for interviews use phone interviews, because they are easy to record (e.g., Maynard and Schaeffer 1997, 2002; Smit & Dijkstra 1991; Dijkstra & Smit 2002), but some comparisons with in-person recruitment are also possible (Morton-Williams 1993; Campanelli, Sturgis, & Purdon 1997; Sturgis & Campanelli 1998).

The distinctive character of a CA approach to studies of recruitment can be illustrated with a contrast, provided by Groves and Couper’s (1996; 1998) path-breaking work about nonresponse in household interview surveys. In this work, Groves and Couper provided several detailed models of the processes involved in recruitment. Figure 1 shows their models that describe in detail the behavior of the interviewer and sample member during recruitment. Figure 1a outlines the decisions that the interviewer makes as she tailors her responses and maintains interaction. For example, she evaluates the householder’s reaction and makes decisions about what strategies are available for her to use in persuading the householder. Figure 1b outlines parallel decisions for the householder. These models include two useful features: First, they draw on the rich theoretical grounding of social psychology to develop theory about the decisions faced by each party. The models thus give an important picture of the way that scripts and expectations might enter the recruitment process and the points at which each actor must make decisions. A second important feature of these models is that there are two of them – one for each party to the interaction -- which provides for more detailed thinking about each of the actors.

Figure 1. Models of participants’ strategies during recruitment for the survey interview.

Figure 1.

Figure 1.

a: Interviewer strategies for tailoring and motivating interaction

b: Householder strategies for evaluating survey request (Groves and Couper 1996, p. 68).

Originally published in Groves, R. M., & Couper, M. P. (1996). Contact-Level Influences on Cooperation in Face-to-Face Surveys. Journal of Official Statistics, 12(1), 63-83. Used with permission of the authors.

The psychological approach underlying these models focuses attention on what happens inside each participant’s head. In contrast, a conversation analysis approach directs attention to what the participants do. An interactional approach brings the actors together into the same model, as illustrated by the interactional model of the recruitment call shown in Figure 2 (Schaeffer et al. 2013). The model was developed based on an extended conversation analysis of recruitment calls in several data sets (e.g., Maynard and Schaeffer 1997; Maynard, Freese, and Schaeffer 2010). These analyses identified actions (e.g., greeting, self-identification), properties of actions (e.g., “hello,” “hi,” “good afternoon, sir”), and sequences of actions (e.g., questions before or after the request for participation) in the recruitment call. As a result, in addition to encompassing both actors, the model suggests that there are different paths, or sequences of actions, through the call opening, some of which may end in completed interviews and others in declinations.

Figure 2. Interactional model of recruitment.

Figure 2.

Reprinted from Schaeffer, Nora Cate, Dana Garbarski, Jeremy Freese, and Douglas W. Maynard. 2013. “An Interactional Model of the Call for Survey Participation: Actions and Reactions in the Survey Recruitment Call. Public Opinion Quarterly 77(1): 323-351, by permission of Oxford University Press for the American Association for Public Opinion Research.

Illustration 1: Using the Identification/Recognition Sequence to Understand the Opening of the Recruitment Call

However, the analysis of the call opening just summarized was incomplete. Because individual features of each action, and even sequences of actions, occurred together as a package, some features were too highly correlated to allow accurate estimation of separate effects, and Schaeffer et al. (2013) had not sufficiently taken this into account. In a subsequent paper, we developed a stronger analysis of the call opening using the CA analysis of the “identification/recognition” sequence in telephone calls.

The “identification/recognition sequence,” is a tightly organized sequence of turns: The phone rings and the householder answers. In the interviewer’s first turn she identifies herself sufficiently that the answerer can tell who she is and may be able to form some guess about the purpose of the call. This identification/recognition sequence is an everyday occurrence and one of the classic sequences in conversation analysis (Schegloff 1979). The job of the caller in their first turn is to provide resources to help the call answerer identify the caller, as can be seen in these excerpts (quoted from Schegloff 1979, p. 37):1

Excerpt 1.

Identification/recognition sequence (Schegloff 1979, p. 37).

Excerpt 1a. Call opening 1 Excerpt 1b. Call opening 2
Phone (Rings) Phone (Rings)
Answerer Hello? Answerer Hello?
Caller Hello, Charles. (0.2) This is Yolk. Caller What’s going on out there? I understand you got a robbery (0.8)
Answerer Oh. Hello, Yolk. Answerer Uh yes. Who’s this speaking, please?
Caller WGN
Answerer WGN?
Caller Yes, sir
Answerer Well, this is the robber. The so-called robber, I guess.

Excerpt 1a illustrates the basic mechanics of this sequence: The caller begins with “Hello, Charles” and provides a pause for the Answerer to speak. The 0.2 second silence of the Answerer suggests that “Hello, Charles” did not let the Answerer recognize the caller, and so the caller identifies himself as Yolk. The answerer then communicates a change of state and recognition, with “Oh. Hello, Yolk.” If the caller does not identify themselves in the first turn, then the person who receives the call predictably raises the issue of identification in the next turn, as can be seen in Excerpt 1b: In this example, the caller launches into their agenda and asks what’s going on -- is there a robbery in progress -- without identifying themselves. The Answerer – the would-be robber as it turns out – asks for an identification in their first turn, suggesting that this response is automatic enough to be enacted even when more pressing matters are unfolding.

Returning to the survey recruitment call, we can see the same sequence in Excerpt 2.

Excerpt 2.

Canonical introduction, Wisconsin Longitudinal Study, acceptance, phone

Actor Turn Transcript Action
INT 1 (ring) (ring) Summons
SM 2 Hello. Response to summons/Greeting
INT 3 Hello. Greeting
3 My name’s (FF) (L). Self-identification
3 I’m calling from the University of Wisconsin Madison for the Wisconsin Longitudinal Study. Institutional Identification Purpose of call
3 May I please speak to (FF) (L)? Request to speak to sample member
SM 4 This is he. Identity confirmation

Excerpt 2 illustrates a “canonical” construction of the interviewer’s first turn (Schaeffer et al. 2018), which includes a greeting, self-identification, institutional identification, and request to speak to the sample member – a full identification turn. The Wisconsin Longitudinal Study (WLS), from which this transcript comes, used a “flexible” introduction: A suggested script – basically the canonical introduction – appeared on the screen, but interviewers were not required to follow it exactly.

Because interviewers were allowed to improvise, many used an “efficient” first turn, which includes only the greeting and request to speak to the sample member as shown in Excerpt 3. According to their supervisor (personal communication), interviewers who used the efficient turn construction did so in an effort to keep the person who answered on the phone. Interviewers believed that the person answering the phone acted as gatekeeper and might block access to the sample member by hanging up if the interviewer revealed the purpose of the call too soon; the goal of the efficient introduction was to keep the person who answered on the phone longer.

Excerpt 3.

Efficient introduction, Wisconsin Longitudinal Study, declination, phone

Actor Turn Transcript Action
INT 1 (ring) Summons
SM 2 Hello. Response to summons/Greeting
INT 3 Hello. Greeting
May I please speak to (FFF) (L)? Request to speak
SM 4 uh Token
Who is this? Wh- question
INT 5 I’m calling from the University of Wisconsin Survey Center at the University of Wisconsin Madison about the Wisconsin Longitudinal Study. Institutional identification Purpose of call
SM 6 Never mind. Declination
INT 7 I’m sorry.
SM 8 (hangs up)

However, an analysis of the impact of the structure of the interviewer’s first turn (Schaeffer et al. 2018) found that the canonical opening is a strong predictor of participation and the efficient opening is associated with declinations. Schegloff’s discussion of the identification/recognition sequence provided the infrastructure for that investigation: his analysis directed our attention to the entirety of the interviewer’s initial turn and grounded hypotheses about the consequences of disrupting the sequence. The efficient introduction in Excerpt 3 illustrates these consequences. In an efficient introduction, the interviewer fails to initiate reciprocity by identifying themselves in the opening turn, and the initiative for establishing the interviewer’s identity passes to the householder. Like Schegloff’s would-be robber, the sample member must then “repair” the interviewer’s botched identification/recognition sequence. When the sample member must initiate a repair because the interviewer failed to identify themselves, the footing of the participants changes.

The clear implication for practice is that the interviewer should provide a full identification turn in their first turn, probably using “hello” as a greeting (Schaeffer et al. 2018). Although these results, once described, may seem “obvious,” there is no comparable prior study, and the focus on the interviewer’s first turn and analysis using the identification/recognition sequence are grounded in CA. A second implication is that important details of how that turn is constructed – the greeting, how the interviewer’s name is presented, and so forth – require experiments using other populations and other survey designs. For example, our analysis of features of the actions in the call also found that, for the study population in the WLS, if the interviewer’s greeting was “hello” the outcome was more likely to be successful than if she said “hi,” a conclusion that should be examined in formal experiments with different populations (Schaeffer et al. 2013). Thus, the analysis of actions in the call opening (e.g., Figure 2) begins to identify candidate treatments for subsequent experiments using an action-based vocabulary.

The analysis of the sequence of actions and an examination of the efficient call opening also provided evidence to refine an earlier finding that if the person speaking with the interviewer asked a question, that signaled a higher propensity to complete the interview (Groves and Couper 1996, pp. 78, 82). First, with respect to location, if the sample member asked a question before the interviewer asked for participation, the odds of participating were lower. Second, with respect to content, questions about how long the interview would take were associated with greatly increased odds of participation, but “wh-” type questions (e.g., “who,” “what,” etc.) were associated with greatly decreased odds of participation. (See Schaeffer et al. 2013.) The data were too sparse to give this issue a full empirical analysis in the efficient turn, but questions like “Who is this?” (e.g., turn 4 in Excerpt 3) that appear before the interviewer has the opportunity to request participation are associated with a decreased likelihood of participation (Schaeffer et al. 2018).

Illustration 2: Analyzing “Scripted” and “Flexible” Introductions

Our analysis of actions in the call opening can be used to reconsider the implications of two influential experiments that compared scripted versus unscripted (called here, “flexible”) introductions. These experiments – one small and face-to-face (Morton-Williams 1993) and the other large and using phone (Houtkoop-Steenstra and van den Bergh 2002) – examined the impact of scripting an introduction on participation. Both studies concluded that the interviewer is more successful in obtaining participation “without a script.” Under the influence of these findings, studies like the WLS adopted a “flexible” introduction, and, although there is no inventory of practice at contemporary survey shops to consult in this matter, at least two prominent university-based research centers (personal communication) prefer to use a “flexible” introduction.

Nevertheless, it is not obvious what it means “not to follow the script” during recruitment or how having a script or not actually affects interaction. We have already seen that the interviewer’s first turn in the contact is quite constrained by the demands of identification/recognition, and also that interviewers’ spontaneous improvisations in the sequence and placement of actions in the opening resulted in the less, not more, successful efficient introduction. In addition, other analyses of the WLS appear to contradict the results of these two studies: In the WLS, when interviewers followed the suggested wording in the script for two key actions that occur early in the call -- asking if the sample member had received a letter or describing the study – the sample member was more, not less, likely to participate (Schaeffer et al. 2013, p. 334).

Both experiments authorized interviewers to be “flexible,” “conversational,” or “not follow the script,” but neither study specified what that might mean. Studies of interaction suggest that different possible meanings of “not following the script” include (1) adding or deleting actions, (2) changing the order of actions, (3) adding, deleting, paraphrasing or otherwise improvising the words used during an action, or (4) altering other features of actions (using techniques 1-3), for example, to increase or decrease formality (e.g., “hi” versus “hello” in a greeting). (The interviewer’s cadence or other acoustic properties of their voice could influence participation, but do not represent a change to the “script.”)

Panel A in Table 1 reproduces the script that Morton-Williams (1993) gave the interviewers to use when the door was answered, and the requirements for that introduction are listed in Table 2. Morton-Williams speculated that the results suggested that the script was simply “not…very good” (p. 84): The long introduction was often interrupted, and interviewers focused on following the script rather than responding to householders’ concerns. The best interviewers in the unscripted condition “developed,” and then repeatedly used, an opening like that in Panel B. This shorter introduction loads key information that the householder needs to assess the situation -- a full self- and institutional identification, the purpose of the contact and how the address was obtained -- into a reasonably-sized first turn.2 Compared to the standard introduction, interviewers deleted some actions and words. It appears that the flexible interviewers made a better script, and one an interviewer might usually be able to deliver before an interruption. Morton-Williams does not report about the interaction that followed this introduction, but it is possible that the end of this shorter introduction provided a place for the householder to make an acknowledgement or ask a question, and thus occasioned a more “interactive” beginning to the recruitment process.

Table 1.

Scripted and “unscripted” introductions

A. Scripted introduction for face-to-face interview (Morton-Williams 1993, p. 83)
Good/morning/afternoon/evening: This is … (address)… isn’t it? I am… (your name)…from Social and Community Planning Research. Here is my identity card. We are an independent social research organization and we are approaching a cross section of the public to find out how people feel about a whole range of social issues. This address was selected from the electoral register and I need your help first with a few questions to find whom I should interview.
B. “Unscripted” introduction for face-to-face interview (Morton-Williams 1993, p. 82)
Hello. This is ADDRESS? My name is NAME, and I work for Social and Community Planning Research which is an independent survey research organization. We’re doing a survey in this area, and they have given me addresses from the electoral roll….
C. Sample medium-length standardized introduction for phone interview (Houtkoop-Steenstra and van der Bergh 2002, p. 207)
Good morning, this is [interviewer’s name] speaking, from [company’s name] in [city]. We are conducting a nation-wide research about reading magazines, watching television, shopping, and some current affairs. Could I please first verify the phone number? Is your number…? Then I would like to have an interview with someone from your household about television and magazines and so on. In order to decide whom I should ask the question, I would like to know how many people live in your household for at least 4 days a week. This, of course, includes yourself. Then I would like to have an interview with….

Table 2.

Elements in introductions in two experiments

Morton-Williams (1993, p. 77)a Houtkoop-Steenstra and van der Bergh (2002, pp. 207-208)
Standardized Introduction

Conversational Short Medium Long

Greeting x x x
Interviewer’s name 2 x x x x
Company’s name 3 x x x x
Show identity card 4
Company’s authority 5, 8 x
Research topic 6 x x x x
Reason for calling 6 x
Second mentioning of topic 9 x x x x
Respondent’s importance x x
Example of question x
Length of interview 11 x
Confidentiality 10 x
Respondent selection 7 x x x x
Check phone number 1 x x x x
Request to participate x x x
Number of Dutch words NA Varies 81 108 164
a

The elements provided by Morton-Williams are mapped as closely as possible onto those given by Houtkoop-Steenstra and van der Bergh. The numbers show in the order in which they are listed. Items 8-10 are not expected to be covered before the respondent selection.

The second influential experiment about whether or not to script an introduction was conducted in a phone survey. It compared a “conversational” introduction that provided the interviewer an agenda with three scripted introductions of different lengths (Houtkoop-Steenstra and van der Bergh 2002). The medium-length script in Panel C of Table 1 is longer than Morton-Williams’ script, and the descriptions of the introductions, summarized in Table 2, are a bit puzzling. For example, the agenda for the “conversational” introduction did not include a greeting or request to participate, but these actions are not really optional – the interviewer must greet the person who answers the phone and must ask for participation in the survey. So, the conversational and “short” introductions included the same actions, but could have differed in the order or wording of the actions. The authors concluded (although the contrasts in the analysis are complicated) that, when contrasted with all the standardized introductions together, the conversational introduction was more likely to result in an interview.

Unfortunately, the study did not include recordings or a manipulation check, so it is unknown what the interviewers in the “conversational” or other conditions actually did. It is possible, for example, that interviewers in the “conversational” introduction simply shortened their introduction, as the interviewers in Morton-Williams’ study did, moved the verification of the phone number to a later position, or interjected a comment of some sort. It is also possible that, like the householders in Morton-Williams’ study, the householders listening to the standardized introductions interrupted the long recitations. The analyses of the WLS suggests that such interruptions occur at predictable locations, that after a short first turn the interviewer must be prepared to respond to an action by the person who answered the phone, and that the interviewer’s responding action is likely to be more successful when the interviewer uses a (good) script.

Discussion: Studying Interaction During Recruitment

In discussing these two examples, I have tried to show that a CA analysis of actions improves our understanding of the action structure of the call opening and what a “conversational” introduction could mean. In their descriptions of the “initial introduction” in face-to-face interviews, both Morton-Williams (1993, p. 79) and Campanelli, Sturgis, and Purdon (1997, p. 4-15) present evidence that self and institutional identification are accomplished in almost all the interviews these investigators examined, usually in the first turn (Morton-Williams 1993, p. 79). Morton-Williams herself concluded that her results do not provide grounds for deciding against providing a written introduction for interviewers (1993, p. 84), and her comments suggest that the flexible interviewers used their authority to compose a better first-turn “script,” which they then followed. The analysis of WLS data that found a positive effect on participation when flexible interviewers followed a script for specific actions can be seen as complementary: The WLS analysis examined scripting not for the contact as a whole, but for specific actions, and thus considers the execution of those actions, not their placement. The analysis of the WLS gives us reason to suspect that once an action begins, following the script – if it is a good script -- makes the interviewer sound not robotic, but competent and coherent.

Morton-Williams’ discussion of the problems with the longer introductions suggests that longer introductions may fail less because interviewers use scripted words than because they use a scripted sequence of actions. For an interviewer to be successful, the interviewer’s next action probably needs to be responsive to the householder in ways that a script with multiple actions cannot anticipate. A script that presents the actions of the call opening in a specific order presents the initial contact to the interviewer as a list of actions to be executed in order regardless of the expressed needs or concerns of the sample member and prevents the interviewer from presenting themselves as a responsive interactional partner.

A recent experiment about persuading sample members to participate in a survey illustrates interviewers apparently resisting a “script” that may require them to be unresponsive. Ongena & Haan (2016) trained interviewers in their experimental conditions to use either a “personal” or a “formal” style when persuading sample members to participate. The researchers used interaction coding to check whether interviewers implemented their experimental training, and so they were able to determine what arguments the interviewers actually used. The researchers observed that interviewers were more successful using arguments that were assigned to the other condition. Although the authors speculate that interviewers formulated their arguments in a more fresh and spontaneous way when using arguments they were not trained in, it is also possible that interviewers violated experimental training in order to use arguments more responsively, and this responsiveness was persuasive.

Taken together with the other results just discussed, we can speculate that interviewers sound robotic not when they use a particular cadence or “sing song” voice, but when they have a “tin ear,” and proceed from action to action, regardless of the responses of the householder or sample member. An interactional approach to interviewer training might prepare interviewers to recognize different actions of sample members, quickly select a responsive action, and deliver the selected action in a way that displays competence. In some ways this proposal envisions a generalized version of the training developed by Groves and McGonagle (2001), which gave interviewers practice in quickly deploying persuasive responses, but our research suggests that avoiding refusals begins in the interviewer’s first turn.

During the interview: Measurement: Following up answers to yes-no questions

Recruitment contacts are likely to have a variety of paths toward an exit or an interview, but once the interview begins – and the householder, answerer, or sample member becomes a respondent -- the interviewer’s control of the agenda and the structure of the paradigmatic question-answer-acknowledgment sequence (Schaeffer & Maynard 1996) constrain the interaction, at least until the respondent’s answer forces those constraints to loosen. Some of the challenges within the interview result from the tension between the practices of conversation and those of standardized measurement (see models in Schaeffer & Dykema 2011a; Dykema, Schaeffer, Garbarski, and Hout 2018). The practices of standardization were developed to reduce random error and increase reliability. When interviewers follow the rules of standardization, they all behave in the same way – they are standardized. When interviewers are standardized, answers differ because the true values of the respondents differ, not because their interviewers behave in different ways. But the practices of standardization differ in important ways from those of conversation, and, as Suchman and Jordan (1990) pointed out, standardization limits the interactional resources participants can use to manage routine issues of talk.

The core technique of various versions of standardization is reading the question as worded (Fowler & Mangione 1995; see also Brenner 1981, 1982). Despite training and supervision, interviewers sometimes reintroduce conversational resources on their own (e.g., Haan, Ongena, & Huiskes 2013). Some variants of standardization aim to restore limited conversational resources after the interviewer has read the question as worded and the respondent has given an answer. “Person-oriented” interviewing focuses on the acknowledgment action and supports the respondent’s motivation by authorizing the interviewer to reflect something the respondent has said (e.g., “I understand what your moving to this house meant for you”) (Dijkstra 1987: 320; Dijkstra & van der Zouwen 1987). “Conversational” interviewing is focused on follow-up after an uncodable answer, request for clarification, or other signs of “trouble” and allows the interviewer to diagnose and remedy problems in comprehension (“Well, what kind of work does she do?”) (Schober & Conrad 1997:586; Conrad & Schober 2000). Event-history calendar interviewing, which aims to support recall, is more a departure from standardization than a variant because it may forgo scripted questions entirely (“So you started working for Acme around a year and a half ago?”) (Belli, Shay, & Stafford 2001:55).

Unlike interviewers, respondents are at liberty to use conversational resources, and they regularly add conversational elements to codable answers, including pauses, tokens, uncertainty markers, and elaborations (e.g., “yes, because I see him every day”) (e.g., Hak 2002; Schaeffer and Maynard 2008; Schaeffer, Dykema, & Garbarski 2016; Schaeffer et al., forthcoming). Respondents also sometimes provide reports – talk that is relevant to the question but does not match the response format projected by the question – instead of a codable answer (Schaeffer and Maynard 1996, 2008). When respondents depart from the paradigmatic question-answer sequence in these ways, interviewers are more likely to depart from the rules of standardization. A complicating factor for interviewers is that standardized training does not always provide clear techniques for determining when an answer is codable or for following up on the answers respondents actually provide. In some cases, repairing the candidate answer might seem to require conversational resources that standardization places off limits (examples in Dijkstra and van der Zouwen 1982; Moore and Maynard 2002).

Somehow survey researchers have managed to train standardized interviewers without specifying what a “codable” answer is (e.g., Fowler and Mangione 1990). A reframing of General Interviewer Training that is still under development defines a codable answer this way: a codable answer comes after the question and any response categories have been read, answers the question, and matches the projected response format, the format on the screen, or selects a response category (Schaeffer et al., forthcoming 2019). The easiest codable answer to recognize is one that exactly matches the response format projected by the question. For example, a yes-no question projects “yes” or “no” as an answer, so “yes” or “no” by itself is “properly formatted” (Schaeffer & Dykema 2011b; Dykema et al. 2018) or “type conforming” (Raymond 2003) and easily recognized as codable. Interviewers can be trained to recognize a codable answer as one that includes a unique portion of a response category (the “kernel” of a codable answer) or matches the format on the screen. So an answer of, “I guess I’d have to say ‘yes’ probably” would usually be codable as “yes,” as the best answer the respondent has to offer, even though “yes” is nested within conversational elements that express uncertainty.

The discussion that follows describes how the conversational practices of respondents pose challenges for interviewers and those who must train them, using yes-no questions to illustrate these challenges. The common yes-no response format is used for a variety of questions of different types – occurrence (“Did you see a doctor about your own health?”), threshold (“Would you say you were very happy?”), and description or classification (“Are you currently married?”). Respondents have many ways of resisting the request for “yes” or “no” with a report, of indicating “yes” or “no” in conversational ways, and of augmenting an answer with information about certainty, comprehension problems, and so forth. The examples that follow become increasingly challenging to resolve, and, although there is no specific advice about how interviewers should follow up, because there is no research comparing the alternatives, the discussion addresses issues that must be decided by those devising rules of interviewing.

Codable answers: Repetitional answers

One conversational practice for answering “yes-no” questions repeats part of the question with appropriate grammatical transformation – for example, a question asks “Do you…?” or “Are you…?” and a respondent replies, “I do” or “I am.” Even without “yes” or “no,” the interviewer may still find that interpreting such answers is straightforward. Thus, in Excerpt 4, “I…am” is probably heard as “yes,” even without the emphatic “sure”:

Excerpt 4.

Repetitional answers, Continuous National Survey, Letters and Sciences Survey Center, phone

Actor Turn Transcript
INT 1 And are you a registered voter?
R 2 I sure am.

Heritage and Raymond (2012) argue that a “repetitional” answer like this one repeats part of the question in a way that “confirms” (that is, strengthens) rather than “affirms” (that is, simply agrees to) the proposition in the question. By confirming, the respondent exerts agency in claiming epistemic entitlement to the knowledge asked about and resisting the constraints of the question. The CA discussion suggests that by accepting “I sure am” as a synonym for “yes,” the interviewer shows interactional competence and also may support the respondent’s agency, and thus their engagement and motivation.

Codable answers and state uncertainty

One common complication faced by respondents, and subsequently by interviewers, is state uncertainty – when the respondent is unsure about their “true value” (Schaeffer and Thomson 1992). There are many ways an answer may be marked to show that the respondent’s underlying state has some uncertainty attached to it, as shown in the answers to the two knowledge questions in Excerpt 5.

Excerpt 5.

Uncertain answers followed by automatic coding, Continuous National Survey, Letters and Sciences Survey Center, phone

Actor Turn Transcript
INT 1 And do the police and the FBI use the census to keep track of trouble makers?
R 2 I doubt it
INT 3 And how about to locate illegal aliens. Is the census used for that?
R 4 Not that I know of

The responses “I doubt it” (i.e., the respondent doubts that the Census is used in that way) and “Not that I know of” (i.e., the respondent does not know of a case in which this has happened) do not suggest that the respondent does not understand the terms or meaning of the question. As conversational answers, each of these answers “leans toward” “no,” but is marked as somewhat uncertain or based on imperfect knowledge. Each answer thus offers the interviewer two paths to a substantive answer: either follow up so that the respondent’s answer can be heard to be an explicit “no” (e.g., “So would you say ‘yes’ or ‘no’?”) or accept an implied “uncertain ‘no,’” that is, engage in “immediate coding” (Hak 2002). The first path is more standardized, but the respondent may experience the pursuit of repair as needlessly belaboring the obvious or as displaying that the interviewer does not recognize “no” when she hears it. This particular interviewer did not follow up either answer and “coded” each answer as “no,” thus displaying that she heard the answer as an “uncertain ‘no,’” rather than “I don’t know” (Hak 2002, p. 458). The interviewer’s acceptance trains the respondent about how much latitude is available when answering. By focusing on the implied “no,” rather than the expressed uncertainty, this interviewer displays that she heard the respondent, understands common conversational practices, and is a competent conversational partner; this approach may be motivating to the respondent.3

Task uncertainty, reports, and transformative answers

Neither repetitional answers nor declarations of uncertainty are properly formatted answers to a yes-no question, and so they are special kinds of “reports” (Drew 1984; Schaeffer & Maynard 2008; Schaeffer & Maynard 1996). Repetitional answers and answers like “I doubt it” or “not that I know of” notify the interviewer that a “yes” or “no” can be pursued if she thinks it relevant. A different sort of trouble – task uncertainty -- is faced by a respondent who has an answer that may not fit the assumptions of the question or the offered categories (Schaeffer & Thomson 1992). The resulting reports commonly take a form like that in Excerpt 6, but this respondent is unusually articulate about the source of the problem:

Excerpt 6.

Report that targets a classification, definition, or problem with the goal of the question, Wisconsin Longitudinal Study, phone

Actor Turn Transcript
INT 1 Do you have a signed and witnessed will?
R 2 I have an estate planned.
INT 3 Okay. Um.
R 4 I don’t know if that, I I don’t know how to answer that question. I don’t have a will per se but I do have…
INT 5 Okay, but okay. So you do not have a signed and witnessed will, but you do have ah
R 6 Yeah. Uh. Everything’s in the estate.

The report (“I have an estate planned”) in the initial answer position in this example (Turn 2) displays task uncertainty, which the respondent later confirms in the elaboration, that they “don’t know how to answer that question.” The statements that respondents provide in reports commonly illuminate an ambiguity in the question or uncertainty about how to apply a concept in the question to the respondent’s own situation. In this case, the report points to the intent of the question: Does the researcher want to know if there is a plan for transfer of property after death -- of which a “will” may simply be the most common -- or specifically if there a particular mechanism for this transfer, that is, a “will.”

The component of the question targeted by the report in Excerpt 6 can be heard as a conceptual or definitional issue. In Excerpt 7, the answer transforms a question that asks whether the respondent has “made plans” into a question about whether the process has begun or how much time has been spent making plans. The answer reveals another ambiguity in the question: Is it asking about a process, whether such plans have ever been completed, or whether a plan is now in place. The answer makes clear that for those who have not completed their plans, the task of deciding whether plans have been “made” requires locating some threshold between “yes” and “no.”

Excerpt 7.

Report that targets a threshold or classification, Wisconsin Longitudinal Study, phone

Actor Turn Transcript
INT 1 Now I am going to ask you some qua or ask some questions about the later years in life. Have you made plans about the types of medical treatment you want or don’t want if you become seriously ill in the future?
R 2 Um, briefly.
INT 3 Okay. So, would you like me to put yes or no for that?
R 4 Um, I guess, yes cause somewhat.

Like the repetitional answers and expressions of state uncertainty, reports resist the response format projected by the question. These reports are like the “transformative” answers described by Stivers and Hayashi (2010). A report may be aimed at a specific term or component of a question or may aim to transform the focus, bias, or presupposition of a question (Stivers & Hayashi 2010). Compared to the range of contexts examined by Stivers and Hayashi, the survey interview is quite specialized. Because the opportunities for negotiation are limited, transformations of the question’s “agenda” may be less common than in other talk. Nevertheless, reports and transformative answers provide clues about respondents’ problems in answering questions.

Reports that target thresholds

Excerpt 7 illustrated that even when questions are about events, a respondent may consult an underlying continuum to decide whether an event has occurred. In cases that are more explicitly evaluative, respondents who are near a possible threshold between “yes” and “no” may implicate that threshold in their answer. In Excerpt 8, for example, the answer “not very good” (turn 2) transforms the question into one about how well the respondent hears, and so exposes that the question does not specify how well the respondent must be able to hear in order to say “yes” – for example, even a little or very well?

Excerpt 8.

Report that targets a threshold, Wisconsin Longitudinal Study, phone, female interviewer and male respondent

Actor Turn Transcript
INT 1 Without a hearing aid and while in a group conversation with at least three other people, have you been able to hear what is said?
(1.7 seconds)
R 2 Not very good.
INT 3 Would you say no?
(0.6 seconds)
R 4 No.

The answer in Excerpt 9 (turn 2) transforms the question into one about how much of the time the respondent can hear, and may also suggest that it is not clear to the respondent whether “most of the time” is sufficient to say “yes”:

Excerpt 9.

Report that targets a threshold, Wisconsin Longitudinal Study, phone, female interviewer, male respondent

Actor Turn Transcript
INT 1 Without a hearing aid and while in a group conversation with at least three other people have you been able to hear what is said?
R 2 Most of the time.
INT 3 {O} Is that a yes then or…
R 4 {O} Yes.
INT 5 {O} Okay.

In both Excerpt 8 and Excerpt 9, the respondent characterizes their hearing as closer to one end of a continuum than to the other, and the interviewer’s follow-up offers an interpretation that responds to the direction that the respondent’s report suggests. These examples suggest that yes-no questions that refer to an underlying continuum would benefit from making the threshold to say “yes” explicit.

The question in Excerpt 10 does have such an explicit threshold: “completely.” The repetitional answer in this case takes a form different from those we examined earlier. Instead of “they do,” the respondent says, “They understand me” (turn 2). The omission of “completely” could be a simple shortening for convenience or show that the respondent forgot the exact wording of the question. Or the omission could signal a range of things -- that “they” understand the respondent “well enough” even if not “completely,” that “well enough” is “completely” for practical purposes, that the respondent has no way of knowing whether their understanding is “complete,” and so on. In the vocabulary of Heritage and Raymond, the respondent exerts agency to “confirm” that he is understood; in the vocabulary of Stivers and Hayashi, the respondent has transformed the question. But the answer leaves ambiguous whether the best selection is “yes” or “no.” The interviewer’s leading follow-up however, suggests that they heard the respondent’s answer as “yes.”

Excerpt 10.

Report that targets a threshold, Wisconsin Longitudinal Study, phone

Actor Turn Transcript
INT 1 Have people who do not know you understood you completely when you speak?
R 2 They understand me.
INT 3 I should put yes, then, is that correct?
R 4 Yes.

Responding to reports: Alignment, affiliation, cooperation, and follow up

Reports of many kinds, including those that could be classified as transformative answers, can be seen as “uncooperative.” Garbarski et al. (2018) examined whether either repetitional answers or other reports predicted the respondent’s willingness to give a saliva sample some time later; they found no relationship, suggesting that these answers do not index “cooperativeness,” at least for later requests for participation. However, reports and other transformative answers either (1) do not align with the question because they resist its terms or presuppositions or (2) disaffiliate because they resist what the interviewer is trying to accomplish (Stivers & Hayashi 2010, p. 20). Nevertheless, by proposing an answer to a (more answerable and implied) transformed question, some reports both locate and solve a possible “problem,” and so cooperate with the larger project of measurement. The implied transformed questions -- “How much have you worked on making plans…?” or “How well do you hear…?” or “How much of the time can you hear…?” or “Do they understand you…?” – may be easier to answer for respondents at some locations on the continua of making plans, hearing, or being understandable. It is then up to the interviewer to either decide that the answer to the “transformed” question can be mapped onto the question that was asked or to follow up.

All these reports that lack a “yes” or “no” kernel require a decision about whether to follow up, and, if so, how. (Although this question is acute in standardized interviewing, it is potentially just as relevant in other research interviewing.) The excerpts just reviewed illustrate a range of responses by the interviewer. The automatic coding in Excerpt 5 and the “response-sensitive” follow-up (which the client requested for the WLS interviews) shown in Excerpts 6, 8, 9, and 10, treat the direction indicated by the respondent’s answer as likely to be the best that the respondent has to offer. The interviewers’ responses display everyday interactional competence in recognizing the polarity (yes or no) implied by the answers, increase efficiency by obtaining a sufficient answer quickly, and put the interviewer in the position of proposing the upshot of the respondent’s report, for example, “Would you say no?” in Excerpt 8. The proposals in these follow-up actions recognize that although the respondent has epistemic authority over their own knowledge, knowledge of what is a “good enough” answer resides, from the respondent’s point of view, with the interviewer.

These excerpts can be used to refine our understanding of when and what follow-up is required. When the question requires that the respondent consult an underlying continuum, and the report targets a threshold – as in Excerpts 8, 9, and 10 – alternative follow-up actions represent different approaches to standardization: responsive (“Would you say no?”), responsive but confirming (“So you would say yes, is that correct?”), or balanced (“Would you say yes or no”). For Excerpt 10, the alternatives could also include follow-ups that target the threshold and the ambiguity in the respondent’s answer: “The question asks if ‘they understand you completely’?” or “Would you say ‘they understand you completely’?” or “Would you say ‘they understand you completely’ or not?” or repeating the entire question. (Note, however, that these targeted follow-ups may be heard as emphasizing “completely” in a way that changes how the respondent understands the threshold for “yes” in an unanticipated way.)

When reports target a concept or definition rather than a location on a continuum, the answer may not “lean” toward “yes” or “no,” as we saw in Excerpt 7. In Excerpt 7, the question may be intended to be about a categorical state of affairs, like the question in Excerpt 6, and it is difficult to pinpoint part of the question to target. The respondent’s answer in Excerpt 6 includes a clear description of the respondent’s situation and a clear statement of task uncertainty, “I don’t know how to answer that question.” Neither a responsive nor a balanced follow-up is likely to be useful in Excerpt 6, because they cannot resolve task uncertainty of this sort. In both Excerpts 6 and 7, the respondent might expect that it is up to the interviewer to draw an upshot or plot a course, that the epistemic burden has shifted to the interviewer, who turns out to be an agent with limited authority. It is also not clear whether providing definitions, as in “conversational” interviewing, would address the issues displayed in these excerpts.

Follow-up as repair

Compared to codable answers that are comprised of just a “kernel,” answers marked as uncertain and uncodable answers have more varied origins and forms. We commonly evaluate choices among follow-up techniques in terms of whether all interviewers can be trained to recognize the situations in which to use the technique and to use the technique in the same way, that is, we focus on operational considerations and the benefits of standardization. There are possible unexplored costs, however. Follow-up is an other-initiated “repair” of the respondent’s answer, an answer that might seem to be fine as it is from the respondent’s point of view. In Excerpt 11, the respondent’s pause, delay, and report “just reading glasses” might suggest that she is uncertain whether only prescription glasses should be counted as “glasses or contact lenses.” The interviewer’s announcement at turn 3 that she is “just” going to reread the question, notifies that respondent that her answer must be repaired, and in the recording of the transition to that reading, at turn 4, one can hear “friendly awkwardness.” Although the respondent’s report identifies a possible issue with the survey question, the interviewer’s rereading is not targeted, and so does not help the respondent locate what problem the interviewer sees in her answer. The pause and delay before “no” in turn 6 suggest that even after hearing the question again she may still have trouble selecting her answer.

This respondent remained engaged and friendly in the rest of this brief exchange, and we did not look for evidence in this particular interview that this repair had subsequent negative consequences, but this repair occurs in the first section of an interview that typically lasted more than an hour. It is worth considering whether many such repairs might eventually reduce the respondent’s engagement or motivation, two aspects of rapport (Garbarski, Schaeffer, & Dykema 2016).

Excerpt 11.

Report followed by rereading of the question, Wisconsin Longitudinal Study, phone

Actor Turn Transcript
INT 1 During the past four weeks, have you been able to see well enough to read ordinary newsprint without glasses or contact lenses?
R 2 Ah. Just reading glasses.
INT 3 Okay. I’m just going to reread that question.
R 4 Oh. Okay.
INT 5 Uh. During the past four weeks, have you been able to see well enough to read ordinary newsprint without glasses or contact lenses?
R 6 Ah. No.
INT 7 Okay.

Our illustrations also draw attention to the form of the interviewer’s follow-up question. The responsive or unbalanced follow-ups in these excerpts qualify as “leading probes” using Fowler and Mangione’s (1990, p. 40) definition that a probe is leading if it can be answered “yes” or “no.” Moore & Maynard (2002, p. 303-304) note that in about half the cases in which they observed an interviewer use a “directive” follow-up, like those in Excerpts 8 through 10, the respondent’s answer was “formally” rather than “substantively” inadequate; that is, the report suggested a possible direction (see also Ongena and Dijkstra 2005; Molenaar and Smit 1996). As we have noted, an unbalanced follow-up may be “responsive” to the answer; an unbalanced follow-up action may allow the interviewer to balance the requirements of standardization with the conversational preference for self-correction in repair. Although these unbalanced follow-ups are commonly “yes-no interrogatives” like those in Excerpts 8 and 9, the “yes-no declarative” form in Excerpt 10 followed by a request for confirmation is also used (Seuren & Huiskes 2017). The three follow-up questions in these examples each display a different technique for offering a properly formatted “yes” or “no” as a reformulation of what the respondent has just said. However, although the follow-up questions recognize the respondent’s “epistemic authority” over their answer, a respondent who disagreed with the proposal would then be in a position of needing to repair the answer offered by the interviewer.

Discussion: Measurement in interaction

Underlying the above discussion of answers to yes-no questions and how interviewers follow up those answers are concerns about the impact of standardization on responsiveness, engagement, motivation, and interactional expressions of rapport. Although experiments about follow-up techniques are challenging to design and execute adequately, they are needed to determine which innovations – for example, the various methods for following up described above -- can be incorporated into training, implemented in field situations, and improve the balance among various sources of error and the various goals of measurement (e.g., Sayles, Belli, Serrano 2010; Dijkstra 1987; Conrad, this issue; Conrad and Schober 2000; Schober and Conrad 1997).

Ideally, experimentation would assess the impact on reliability as well as motivation, comprehension, and memory. Innovations are likely to be most useful if they consider the details of how respondents actually answer questions and how interviewers actually follow up, as suggested by the excerpts reviewed here. (Similarly, a reconsideration of how to read – and write – survey questions might well begin by considering interviewers’ improvisations as they read (Haan, Ongena & Huiskes 2013).) Both codable and uncodable answers include a variety of expressions of uncertainty, lack of knowledge, in addition to evidence of conceptual and other issues that respondents face. Interviewers sometimes confront conflicting pressures from the rules of standardization, habits of ordinary conversational and interactional competence, and the challenge of supporting the respondent’s engagement and motivation. Developments in interviewing technique should consider the ways that we currently rely on the interviewer’s interactional competence (as in “immediate coding”), the many goals of measurement, and the interactional pressures on the interviewer.

Conclusion

Conversational practices are ubiquitous in all interactions that become data for researchers, both in recruitment participants and in research interviewing of any sort. Conversational practices appear in our approaches to sample members, how respondents answer, how interviewers understand those answers, and how interviewers follow up. Nevertheless the interaction that takes place before the interview differs from that during the interview in significant ways, and so the discussions in this paper drew on different threads or traditions within CA. During recruitment, the placement and execution of actions is particularly significant. For example, the consequences of using an efficient introduction suggest that there are costs to abbreviating the identification/recognition sequence. In predicting participation, researchers probably need to attend to both the content of questions (e.g., length of interview or something else) and their placement (e.g., before or after a request to participate). Once the interview begins, the types of relevant actions are more constrained. During the interview, the details of questions, answers and acknowledgments, and negotiating the sufficiency of “answers” are central. The extended discussion of answers to yes-no questions illustrates that rules of interviewing eventually encounter conversational practices in respondents’ answers. We do not know enough about precisely which follow-up practices best support the reliability and validity of the resulting data as well the motivation and engagement of the respondent. Clearly this is an area for research.

The examples considered in this paper obviously do not exhaust the resources in conversation analysis from which survey researchers could benefit. Nor are the benefits of such analyses restricted to survey researchers: Although survey researchers have particularly strong requirements for a “codable” answer, researchers who work with less rigid forms of interviewing also want to understand the ways in which their questions and practices influence interviewees and to believe that they understand what the person they interview “means” by an answer.

Analysis of survey interviews also offers benefits to conversation analysts. Because efforts to recruit survey respondents have a measurable outcome, studies of recruitment provide an opportunity for studying the consequences of the sequencing and features of actions. Once the interview begins, surveys produce many instances of sequences using the same question in which to observe variation in responses and what follows, and the quality of the resulting measurement serves as a criterion. Like the collaborations with cognitive psychologists (e.g., Sirken et al. 1999), those between survey researchers and conversation analysts offer discoveries for both disciplines.

Acknowledgments

The work about nonresponse using the Wisconsin Longitudinal Study (WLS) summarized here was supported by a grant from the National Science Foundation [SES-1230069] Additional support for this research was provided by the University of Wisconsin - Madison Office of the Vice Chancellor for Research and Graduate Education with funding from the Wisconsin Alumni Research Foundation. Other support for the construction of the original data file, analysis, and collection of the data was received from the National Science Foundation [SES-0550705] to Douglas W. Maynard. The research about interaction using the WLS was supported by National Institute on Aging grant to Schaeffer under grant P01 AG 21079 to Robert M. Hauser. Other support was provided by the Wisconsin Center for Demography and Ecology (National Institute of Child Health and Human Development Center Grant [R24 HD047873]), Wisconsin Center for Demography of Health and Aging (National Institute on Aging Center Grant [P30 AG017266, by the William H. Sewell Bascom Professorship, and by the University of Wisconsin Survey Center (UWSC).

Some of the research reported here uses data from the Wisconsin Longitudinal Study (WLS) of the University of Wisconsin-Madison. Since 1991, the WLS has been supported principally by the National Institute on Aging [AG-9775, AG-21079, AG-033285, and AG-041868], with additional support from the Vilas Estate Trust, the National Science Foundation, the Spencer Foundation, and the Graduate School of the University of Wisconsin-Madison. Since 1992, data have been collected by the University of Wisconsin Survey Center. A public use file of data from the Wisconsin Longitudinal Study is available from the Wisconsin Longitudinal Study, University of Wisconsin-Madison, 1180 Observatory Drive, Madison, Wisconsin 53706 and at http://www.ssc.wisc.edu/wlsresearch/data/.

Biography

Nora Cate Schaeffer is Sewell Bascom Professor of Sociology and the Faculty Director of the University of Wisconsin Survey Center at the University of Wisconsin-Madison. Her current research focuses on interaction when the sample member is recruited and during the interview and on instrument design issues. Her research has appeared in Public Opinion Quarterly, Journal of Official Statistics, Journal of the American Statistical Association, Sociological Methodology, and Journal of Survey Statistics and Methodology. She is a Fellow of the American Statistical Association and of the Midwest Association for Public Opinion Research

Footnotes

1

Most of the examples and excerpts approximate the source transcripts and their terminology (e.g., “answerer,” “householder,” or “sample member” – which are not interchangeable), but some punctuation is added for readability and some detail is omitted. Because the discussion here does not require the details of CA transcription conventions, the excerpts use conventional transcription. In a few of the survey interview transcripts, INT is the interviewer, SM is a sample member, and R is a respondent; parentheses indicate pauses (although some details about pauses were omitted), and the length in seconds is included when it is available; {O} indicates overlap.

2

The verification was required to be completed first, even though Morton-Williams noted that interviewers disliked beginning this way, and, in practice, interviewers placed the verification later or omitted it in about half of the recorded interviews (1993, p. 78-79). The confirmation of the phone number reached is also commonly placed in this location in contemporary phone surveys, and we can speculate that this placement disrupts the identification/recognition sequence, is intrusive in the same way that greeting the sample member by name before introducing oneself is, and reflects the researcher’s priorities and not the respondent’s.

3

A third path, when an interviewer hears an answer as expressing “I don’t know,” rather than an “uncertain ‘no,’” would be to code the answers as “don’t know,” particularly because the facts these questions ask about may be unfamiliar to the respondent. A “don’t know” category is available for all questions, but most rules for standardized interviewing require that an answer be followed up before a “don’t know” answer is coded. The practice of following up is consistent with research that suggests that respondents may answer reliably even when they are not certain and with analysts’ preferences for “substantive” answers (e.g., Schuman and Presser 1981; Krosnick 2002).

References

  1. Belli RF, Shay WL, & Stafford FP (2001). Event History Calendars and Question List Surveys: A Direct Comparison of Interviewing Methods. Public Opinion Quarterly, 65(1), 45–74. [DOI] [PubMed] [Google Scholar]
  2. Brenner M (1981). Aspects of Conversational Structure in the Research Interview. In Werth P (Ed.), Conversation and Discourse (pp. 19–40). London: Croom Helm. [Google Scholar]
  3. Brenner M (1982). Response-effects of ‘Role-Restricted’ Characteristics of the Interviewer. In Dijkstra W & van der Zouwen J (Eds.), Response Behaviour in the Survey-interview (pp. 131–165). London: Academic Press. [Google Scholar]
  4. Campanelli P, Sturgis P, & Purdon S (1997). Can You Hear Me Knocking: An Investigation into The Impact of Interviewers on Survey Response Rates. London, England: The Survey Methods Centre at SCPR, Social and Community Planning Research. [Google Scholar]
  5. Cannell CF, Lawson SA, & Hausser DL (1975). A Technique for Evaluating Interviewer Performance. Ann Arbor, MI: Survey Research Center of the Institute for Social Research, University of Michigan. [Google Scholar]
  6. Conrad FG, & Schober MF (2000). Clarifying Question Meaning in a Household Telephone Survey. Public Opinion Quarterly, 64, 1–28. [DOI] [PubMed] [Google Scholar]
  7. Dijkstra W (1987). Interviewing Style and Respondent Behavior: An Experimental Study of the Survey Interview. Sociological Methods and Research, 16(2), 309–334. [Google Scholar]
  8. Dijkstra W, & Smit JH (2002). Persuading Reluctant Recipients in Telephone Surveys. In Groves RM, Dillman DA, Eltinge JL, & Little RJA (Eds.), Survey Nonresponse (pp. 121–134). New York: John Wiley & Sons. [Google Scholar]
  9. Dijkstra W, & van der Zouwen J (Eds.). (1982). Response Behaviour in the Survey-interview. London: Academic Press. [Google Scholar]
  10. Dijkstra W, & van der Zouwen J (1987). Styles of Interviewing and the Social Context of the Survey-Interview (sic). In Hippler H-J, Schwarz N, & Sudman S (Eds.), Social Information Processing and Survey Methodology (pp. 200–211). New York: Springer-Verlag. [Google Scholar]
  11. Drew P (1984). Speakers’ reportings in invitation sequences. In Atkinson JM & Heritage J (Eds.), Structures of Social Action: Studies in Conversation Analysis (pp. 129–151). Cambridge: Cambridge University Press. [Google Scholar]
  12. Dykema J, Schaeffer NC, Garbarski D, & Hout M (2018, forthcoming). The Role of Question Characteristics in Designing and Evaluating Survey Questions. In Beatty P, Collins D, Kaye L, Padilla J, Willis G, & Wilmot A (Eds.), Advances in Questionnaire Design, Development, Evaluation, and Testing, (pp. 449–470). Hoboken, NJ: Wiley. [Google Scholar]
  13. Fowler J, Floyd J, & Mangione TW (1990). Standardized Survey Interviewing: Minimizing Interviewer-Related Error. Newbury Park: Sage. [Google Scholar]
  14. Garbarski D, Schaeffer NC, & Dykema J (2016). Interviewing Practices, Conversational Practices, and Rapport: Responsiveness and Engagement in the Survey Interview. Sociological Methodology. doi: 10.1177/0081175016637890 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Garbarski D, Schaeffer NC, Dykema J, Baber A, & Min BH (2018). Rapport in the Standardized Survey Interview: Responsiveness, Engagement, and Future Participation. Unpublished working paper. Loyola University Chicago. [Google Scholar]
  16. Groves RM, & Couper MP (1996). Contact-Level Influences on Cooperation in Face-to-Face Surveys. Journal of Official Statistics, 12(1), 63–83. [Google Scholar]
  17. Groves RM, & Couper MP (1998). Nonresponse in Household Interview Surveys. New York: John Wiley and Sons. [Google Scholar]
  18. Groves RM, Fowler FJ Jr, Couper MP, Lepkowski JM, Singer E, & Tourangeau R (2009). Survey Methodology, Second Edition. Hoboken, New Jersey: Wiley. [Google Scholar]
  19. Groves RM, O’Hare BC, Gould-Smith D, Benkí JR, & Maher P (2008). Telephone Interviewer Voice Characteristics and the Survey Participation Decision. In Lepkowski JM, Tucker C, Brick JM, de Leeuw ED, Japec L, Lavrakas PJ, Link MW, & Sangster RL (Eds.), Advances in Telephone Survey Methodology (pp. 385–400). New Jersey: John Wiley & Sons. [Google Scholar]
  20. Groves RM, & McGonagle KA (2001). A Theory-Guided Interviewer Training Protocol Regarding Survey Participation. Journal of Official Statistics, 17(2), 249–266. [Google Scholar]
  21. Haan M, Ongena Y, & Huiskes M (2013). Interviewers’ Questions: Rewording Not Always a Bad Thing. In Winker P, Menold N, & Porst R (Eds.), Interviewers’ Deviations in Surveys: Impact, Reasons, Detection and Prevention (pp. 173–194). [Google Scholar]
  22. Hak T (2002). How interviewers make coding decisions. In Maynard DW, Houtkoop-Steenstra H, Schaeffer NC, & van der Zouwen J (Eds.), Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview (pp. 449–470). New York: Wiley. [Google Scholar]
  23. Heritage J, & Raymond G (2012). Navigating Epistemic Landscapes: Acquiescence, Agency, and Resistance in responses to Polar Questions. In de Ruiter JP (Ed.), Questions: Formal, Functional, and interactional perspectives (pp. 172–192). Cambridge: Cambridge University Press. [Google Scholar]
  24. Houtkoop-Steenstra H, & van den Bergh H (2002). Effects of Introductions in Large-Scale Telephone Survey Interviews. In Maynard DW, Houtkoop-Steenstra H, Schaeffer NC, & van der Zouwen J (Eds.), Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview, (pp. 205–218). [Google Scholar]
  25. Jackle A, Lynn P, Sinibaldi J, & Tipping S (2011). The Effect of Interviewer Personality, Skills, and Attitudes on Respondent Co-Operation with Face-to-Face Surveys. Institute for Social and Economic Research Working Paper No. 2011–14. [Google Scholar]
  26. Krosnick JA (2002). The causes of no-opinion responses to atttitude measures in surveys: They are rarely what they appear to be. In Groves RM, Dillman DA, Eltinge JL, & Little RJA (Eds.), Survey Nonresponse (pp. 87–100). New York: John Wiley & Sons. [Google Scholar]
  27. Maynard DW, Freese J, & Schaeffer NC (2010). Calling for Participation: Requests, Blocking Moves, and Rational (Inter)action in Survey Introductions. American Sociological Review, 75(5), 791–814. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Maynard DW, Houtkoop-Steenstra H, Schaeffer NC, & van der Zouwen J (Eds.). (2002). Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview. New York: John Wiley and Sons. [Google Scholar]
  29. Maynard DW, & Schaeffer NC (1997). Keeping the Gate: Declinations of the Request to Participate in a Telephone Survey Interview. Sociological Methods & Research, 26(1), 34–79. [Google Scholar]
  30. Maynard DW, & Schaeffer NC (2002). Opening and Closing the Gate: The Work of Optimism in Recruiting Survey Respondents. In Maynard DW, Houtkoop-Steenstra H, Schaeffer NC, & van der Zouwen J (Eds.), Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview (pp. 179–204). New York: Wiley. [Google Scholar]
  31. Molenaar NJ, & Smit JH (1996). Asking and answering yes/no-questions in survey interviews: a conversational approach. Quality & Quantity, 30(2), 115. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=sih&AN=9972114&loginpage=Login.asp&site=ehost-live [Google Scholar]
  32. Moore RJ, & Maynard DW (2002). Achieving Understanding in the Standardized Survey Interview: Repair Sequences. In Maynard DW, Houtkoop-Steenstra H, Schaeffer NC, & van der Zouwen J (Eds.), Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview (pp. 281–312). New York: Wiley. [Google Scholar]
  33. Morton-Williams J (1993). Interviewer Approaches. Aldershot, England: Dartmouth Publishing. [Google Scholar]
  34. Oksenberg L. & Cannell CF (1988). Observation of Behavior in Telephone Interviews. In Groves RM, Biemer PP, Lyberg LE, Massey JT, Nicholls WL II, & Waksberg J (Eds.), Telephone Survey Methodology (pp. 476–496). New York: John Wiley & Sons. [Google Scholar]
  35. Ongena Y, & Dijkstra W (2005). Effects of Suggestive and Non-suggestive Probing on Different Types of Inadequate Answers. Paper presented at the American Association for Public Opinion Research, Miami, FL, May. [Google Scholar]
  36. Ongena Y, & Haan M (2016). Using Interviewer-respondent Interaction Coding as a Manipulation Check on Interviewer Behavior in Persuading CATI Respondents. Survey Practice, 9, 1–7. [Google Scholar]
  37. Raymond G (2003). Grammar and Social Organization: Yes/No Interrogatives and the Structure of Responding. American Sociological Review, 68(6), 939–967. [Google Scholar]
  38. Sayles H, Belli RF, & Serrano E (2010). Interviewer Variance between Event History Calendar and Conventional Questionnaire Interviews. Public Opinion Quarterly, 74(1), 140–153. [Google Scholar]
  39. Schaeffer NC (1991). Conversation with a Purpose--Or Conversation? Interaction in the Standardized Interview. In Biemer PP, Groves RM, Lyberg LE, Mathiowetz NA, & Sudman S (Eds.), Measurement Errors in Surveys (pp. 367–392). New York: John Wiley & Sons. [Google Scholar]
  40. Schaeffer NC, & Dykema J (2011a). Response 1 to Fowler’s Chapter: Coding the Behavior of Interviewers and Respondents to Evaluate Survey Questions. In Madans J, Miller K, Maitland A, & Willis G (Eds.), Question Evaluation Methods: Contributing to the Science of Data Quality (pp. 23–39). Hoboken, NJ: John Wiley & Sons, Inc. [Google Scholar]
  41. Schaeffer NC, & Dykema J (2011b). Questions for Surveys: Current Trends and Future Directions. Public Opinion Quarterly, 75(5), 909–961. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Schaeffer NC, Dykema J & Garbarski D (2016). “Conversational Practices and Standardization: How Respondents Answer Survey Questions.” Paper presented at the annual meetings of the American Association for Public Opinion Research, Austin, TX: May. [Google Scholar]
  43. Schaeffer NC, Dykema J, Coombs SM, Schultz RK, Holland L, and Hudson M (forthcoming, 2019). “General Interviewer Training: Developing Evidence-Based Practices for Standardized Interviewing.” Paper to be presented at Interviewers and Their Effects from a Total Survey Error Perspective Workshop, University of Nebraska – Lincoln, 26–28 February. [Google Scholar]
  44. Schaeffer NC, Garbarski D, Freese J, & Maynard DW (2013). An Interactional Model of the Call for Participation in the Survey Interview: Actions and Reactions in the Survey Recruitment Call. Public Opinion Quarterly, 77(1), 323–351. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Schaeffer NC, & Maynard DW (1996). From Paradigm to Prototype and Back Again: Interactive Aspects of Cognitive Processing in Survey Interviews. In Schwarz N & Sudman S (Eds.), Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research (pp. 65–88). San Francisco: Jossey-Bass. [Google Scholar]
  46. Schaeffer NC, & Maynard DW (2008). The Contemporary Standardized Survey Interview for Social Research. In Conrad FG & Schober MF (Eds.), Envisioning the survey interview of the future (pp. 31–57). Hoboken, NJ: Wiley. [Google Scholar]
  47. Schaeffer NC, Min BH, Purnell T, Garbarski D, & Dykema J (2018). Greeting and Response: Predicting Participation from the Call Opening. Journal of Survey Statistics and Methodology, 6, 1: 122–148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Schaeffer NC, & Thomson E (1992). The Discovery of Grounded Uncertainty: Developing Standardized Questions About Strength of Fertility Motivation. In Marsden PV (Ed.), Sociological Methodology 1992 (Vol. 22, pp. 37–82). Oxford: Basil Blackwell. [Google Scholar]
  49. Schegloff EA (1979). Identification and Recognition in Telephone Openings. In Psathas G (Ed.), Everyday Language: Studies in Ethnomethodology (pp. 23–78). New York: Irvington. [Google Scholar]
  50. Schober MF, & Conrad FG (1997). Does Conversational Interviewing Reduce Survey Measurement Error? Public Opinion Quarterly, 61(4), 576–602. [Google Scholar]
  51. Schuman H, & Presser S (1981). Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording and Context. Orlando, FL: Academic Press. [Google Scholar]
  52. Seuren LM, & Huiskes M (2017). Confirmation or Elaboration: What Do Yes/No Declaratives Want? Research on Language and Social Interaction, 50(2), 188–205. [Google Scholar]
  53. Sirken MG, Herrmann DJ, Schechter S, Schwarz N, Tanur JM, & Tourangeau R (Eds.). (1999). Cognition and Survey Research. New York: John Wiley and Sons. [Google Scholar]
  54. Smit JH, & Dijkstra W (1991). Persuasion Strategies for Reducing Refusal Rates in Telephone Surveys. BMS, Bulletin de Methodologie Sociologique, 33, 3–19. [Google Scholar]
  55. Smit JH, Dijkstra W, & Van der Zouwen J (1997). Suggestive Interviewer Behaviour in Surveys: An Experimental Study. Journal of Official Statistics, 13(1), 19–28. [Google Scholar]
  56. Smyth JD, & Olson K (2019). The Effects of Mismatches between Survey Question Stems and Response Options on Data Quality and Responses. Journal of Survey Statistics and Methodology, 7(1), 34–65. [Google Scholar]
  57. Stivers T, & Hayashi M (2010). Transformative Answers: One Way to Resist a Question’s Constraints. Language in Society, 39(1), 1–25. [Google Scholar]
  58. Stivers T, & Sidnell J (2013). Introduction. In Sidnell J & Stivers T (Eds.), The Handbook of Conversation Analysis (pp. 1–8). Chichester, West Sussex: Blackwell Publishing Limited. [Google Scholar]
  59. Sturgis P, & Campanelli P (1998). The Scope for Reducing Refusals in Household Surveys: An Investigation Based on Transcripts of Tape-Recorded Doorstep Interactions. Journal of the Market Research Society, 40(2), 121–139. [Google Scholar]
  60. Suchman L, & Jordan B (1990). Interactional Troubles in Face-to-Face Survey Interviews. Journal of the American Statistical Association, 85(409), 232–253. [Google Scholar]
  61. van der Vaart W, Ongena Y, Hoogendoom A, & Dijkstra W (2005). Do Interviewers’ Voice Characteristics Influence Cooperation Rates in Telephone Surveys? International Journal of Public Opinion Research, 18(4), 488–499. [Google Scholar]
  62. van der Zouwen J, & Dijkstra W (2002). Testing Questionnaires Using Interaction Coding. In Maynard DW, Houtkoop-Steenstra H, Schaeffer NC, & van der Zouwen J (Eds.), Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview (pp. 427–448). New York: Wiley. [Google Scholar]

RESOURCES