Emotional Risks to Respondents in Survey Research: Some Empirical Evidence

Abstract

Some survey research has documented distress in respondents with pre-existing emotional vulnerabilities, suggesting the possibility of harm. In this study, respondents were interviewed about a personally distressing event; mood, stress, and emotional reactions were assessed. Two days later, respondents participated in interventions to either enhance or alleviate the effects of the initial interview. Results indicated that distressing interviews increased stress and negative mood, although no adverse events occurred. Between the interviews, moods returned to baseline. Respondents who again discussed a distressing event reported moods more negative than those who discussed a neutral or a positive event. This study provides evidence that, among nonvulnerable survey respondents, interviews on distressing topics can result in negative moods and stress, but they do not harm respondents.

The question of potential risks to respondents who participate in survey interviews is one that continues to be asked by institutional review boards (IRBs) and the regulators of federally sponsored research in the United States. As the research community has become more sensitive to human subject protections, one specific area of concern has been the potential harm in asking individuals to report on emotionally distressing topics, especially in the context of no benefit for participants. Of concern are psychological risks including depression, altered self-concept, increased anxiety, decreased confidence in others, guilt, shame, fear, embarrassment, boredom, frustration, receiving information about oneself that is unpleasant, and inconvenience (Hermeren, 1983; Sieber, 2000). Alternately, some researchers have reported benefits to respondents in survey research that had not been previously considered. These benefits include the opportunity to discuss the event, access to resources, new insight, feelings of well-being, improvements in health, and the potential to help others (Newman et al., 1997; Newman, Risch, & Kassam-Adams, 2006). Recent studies, cited below, have begun to articulate emotional distress as well as subjective benefits experienced by respondents in survey research.

In this paper, we report a study designed to describe emotional reactions in a general population sample when discussing distressing topics during telephone interviews. The main goal was to document the extent to which emotional reactions occurred, as well as to provide information on the intensity, duration, and impact of these survey-induced emotional reactions on the subject’s life. A follow-up interview was intended to test a method to alleviate emotional reactions that could occur as a result of surveys on sensitive topics.

The available evidence has examined several distressing topics, including terrorism (Boscarino et al., 2004; Galea et al., 2005); sexual and physical violence among adults (Black et al., 2006; Johnson & Benight, 2003; Masho, Odor, & Adera, 2005; Newman, Walker, & Gefland, 1999; Walker et al., 1997) and adolescents (Finkelhor, Ormrod, & Turner, 2007a, 2007b; Hanson et al., 2006a, 2006b); intimate partner violence (Henderson et al., 2005; Seedat, Stein, & Forde, 2005; Zink et al., 2005); traumatic injuries (Kassam-Adams & Newman, 2005; Ruzek & Zatzick, 2000); and bereavement (Dyregrov, 2004; Hauksdottir et al., 2006; Kreicbergs et al., 2004; Takesaka, Crowley, & Casarett, 2004). This body of research demonstrates that some respondents are vulnerable to the experience of emotional distress, typically those with pre-existing depression or PTSD. The distress seems to be minimal for most, though, and few require referral for clinical follow-up.

Benefit has also been reported by many participants in the studies above. A significant proportion of individuals in surveys of sexual and physical abuse (Walker et al., 1997), child abuse (Newman et al., 1999), and domestic violence (Johnson & Benight, 2003) reported some benefit of their participation. A large majority of parents and children interviewed about injuries reported feeling good about helping others (Kassam-Adams & Newman, 2002, 2005); and most of the individuals interviewed about terrorism (Boscarino et al., 2004), as well as bereaved parents (Dyregrov, 2004), reported that participation was a positive experience.

Several studies have compared the effects of surveys on traumatic events to those assessing less emotional content (e.g., personality, values). In a group of individuals who were receiving outpatient treatment for PTSD, Ferrier-Auerbach, Erbes, and Polusny (2009) found that those who completed the surveys on traumatic events reported more sadness, although they did not differ from those who completed the nontrauma surveys in their ratings of perceived gain from participation. Other researchers contrasted surveys on trauma and sexuality with cognitive measures, reporting that those who completed the surveys on trauma and sexuality acknowledged slightly greater negative emotion (Yeater et al., 2012). Both groups also rated the distress associated with examples of normal life stressors, and both groups reported the normal life stressors to be associated with more distress than participation in the study. A similar study compared questions on SAT/GPA scores, body image, emotional abuse, and sexual abuse, and found that these four types of items were associated with similar (minimal) levels of distress in college students (Cromer et al., 2006). However, the trauma questions were rated as being more important, with a more favorable cost-benefit ratio. Even those who found the items most distressing felt the research was important. Another line of research has explored the impacts of disclosure of traumatic or stressful events through either writing or discussion (e.g., Baddeley & Pennebaker, 2011). A large meta-analysis of studies of this type reported that disclosure in experimental settings is beneficial to psychological health, physical health, and overall functioning (Frattaroli, 2006).

In summary, the literature to date does report negative emotional reactions in some survey research respondents, but also some potential benefits for respondents. However, the literature thus far has not provided information on the frequency, intensity, and impacts of the negative emotional reactions, nor do we have a good understanding of steps that can be taken to alleviate emotional reactions experienced by survey subjects. The studies described above typically had one or more traditional protections in place—e.g., informing respondents of sensitive topics; respondents could refuse to answer; respondents could call a number to reach a counselor. Additional procedures to identify and alleviate emotional harms, either as a part of the protocol for all respondents or as a separate component for those who demonstrate a need or are in a vulnerable group, are needed.

Methods

Subjects

Respondents were 395 adult (age 18+) community-dwelling residents in the Chicago metropolitan area, recruited via calls to a random digit dialed (RDD) telephone sample from February–May 2011. Respondents were selected from the household using a modified version of the Trodahl-Carter-Bryant respondent selection method (Bryant, 1975). Only individuals able to complete the interview procedures in English were included.

The response rate for the initial interview was 13.1% (calculated using the American Association of Public Opinion Research’s Standard Definitions response rate formula 3; AAPOR, 2011). The initial interview refusal rate was 55.6%. A total of 316 respondents subsequently completed the second interview, for a successful follow-up rate of 80%.

Procedures

Respondents were asked to participate individually by phone on two separate occasions, separated by approximately two days. On the first day, immediately following verbal consent and eligibility screening, they were asked to provide demographic data, and completed measures of current mood and stress. Respondents were then asked to discuss an upsetting personal experience with the interviewer. Afterwards, they rated their post-interview mood, stress, and their reactions to participation in the study. The eligibility screening required an average of 8.5 minutes (SD = 4.1; range = 4–47), and the first-day interview averaged 40.2 minutes in length (SD = 10.9; range = 18–97).

When respondents were re-contacted two days later, their mood and stress levels were again assessed, and they completed measures of impacts of the discussion of the upsetting event in the time between the two interviews. They were then randomized to one of three manipulations (described below): positive mood induction, neutral/distraction, or another discussion of the upsetting event. Afterwards, respondents again rated their mood, completed the emotion subscale of the RRPQ, and were offered support/referral information. Individuals were monitored for psychological distress throughout (i.e., interviewers were sensitive to respondent reactions and asked participants how they were doing at several points). This second interview averaged 27.0 minutes in length (SD = 6.3; range = 17–67).

The Survey Research Laboratory at the University of Illinois at Chicago conducted all interviews, using Computer-Assisted Telephone Interviewing. An experienced field coordinator had overall responsibility for training interviewers and supervising data collection. Interviewers received training and practice on techniques for establishing rapport, answering questions, and maintaining respondent cooperation. They received training on general human subject protections as well as study-specific training on the assessment and management of emotional distress. Because of the concern about respondent distress, interviews incorporated items to check with the subject on how they were feeling (e.g., are you doing OK, do you need a break before continuing), as well as emergency safety scripts that were accessed if a person reported plans for suicide or appeared extremely upset at any point in the phone contacts. The safety scripts allowed interviewers to further assess the respondent’s situation, to decide if emergency intervention was required or not. Mock interviews were employed to evaluate interviewer performance during training, and interview monitoring was used to randomly evaluate interviewer performance during actual interviews. Interviewers received a total of 20 hours of training for this study.

Respondents were paid $30 for completing the first interview, and an additional $15 for completing the second interview. All study protocols were approved by the University of Illinois at Chicago Institutional Review Board. Waivers of documentation of consent and alterations for both eligibility screening and for the main study were granted by the IRB, to allow verbal consenting on the telephone. We used a shortened consent process that included information on what participants would be asked to do, potential risks, voluntariness, and contact information for the PI and for the IRB. Interviewers also used FAQs to answer any additional questions posed by potential respondents.

ELIGIBILITY SCREENING AND MEASURES

With the help of a consultant who has expertise in PTSD, we developed screening procedures to exclude individuals at risk for emotional harm. Potential respondents were ineligible for any of the following reasons: (1) experience in the past three months of the unexpected death of someone very close to them (N = 114); (2) a psychiatric inpatient admission in the past three months (N = 2); (3) exposure to a traumatic event in the past three months (N = 28); (4) a lifetime diagnosis of PTSD (N = 46), measured using the PTSD Symptom Scale Interview (Foa et al., 1993); and (5) current depression (BDI>9) or a positive response to the suicide item (N = 75), measured with the Beck Depression Inventory—Short Form (Beck & Beck, 1972). In addition, three individuals were ineligible because interviewers accessed the safety scripts due to something the respondent said or did during the eligibility screening. Overall, a total of 268 individuals were found not eligible for participation. A weekly conference with a consultant who has expertise in human subject protections occurred during data collection, as an additional protection for respondents.

INITIAL INTERVIEW MOOD MANIPULATIONS

Respondents were asked to select the specific content to be discussed with the interviewer (using procedures adapted from Labott et al., 2001):

Next, I am going to ask you to talk about a personally upsetting event that happened to you. Think of a personally upsetting experience that happened in the past few years, where you experienced strong emotions and in which at least one other person was involved. Be sure it is an event that you are willing to talk with me about.

They were then given the following instructions:

Now I’d like you to put yourself back into the situation as if it just happened. It’s important to get back into the event now, as if it’s happening again. Then describe the event to me in detail as if you were talking about it with a friend or relative. Include as many details as you can so that I can understand what was going on and how you felt. The more information you provide, the better I’ll be able to understand what happened. After you tell me about it, I will ask you some questions to be sure I understand all the details of what was going on.

Respondents were allowed to tell their story. Subsequently, the interviewer asked questions to elicit more information and their specific reactions in this situation. The interview was complete when the respondent had been discussing the distressing event for a minimum of 12 minutes, and they had also completed a set of questions concerning when the event happened, who was present and how they reacted, the emotions and thoughts experienced by the participant, and what he/ she did.

SECOND INTERVIEW MOOD MANIPULATIONS

Respondents were randomized to one of three interventions: (1) Discuss intervention: Respondents were asked further questions about the event they discussed initially, (2) Neutral/Distract: Respondents were interviewed about their opinions about smoking bans in restaurants, and (3) Positive: Respondents discussed a recent event that made them feel good or happy, using procedures similar to the initial interview. These discussions lasted a minimum of six minutes.

HYPOTHESES

From pre- to post-interview, we expected that self-reported moods would become more negative and that stress would increase, although we did not expect that participants would be harmed by these fluctuations in mood and stress. After two days, when respondents were recontacted, it was expected that moods and stress would have returned to baseline, although no research has yet looked specifically at this question. We anticipated that those with more negative moods and more stress, and those who had stronger emotional reactions initially (after the initial interview), would report greater post-survey impacts (in the time between the two interviews). At the second contact, when respondents were interviewed again, we expected that those who again discussed the emotional event would demonstrate greater negative mood, stress, and other emotional reactions, compared to those who did not, although we again did not expect them to be harmed. We also anticipated that individuals assigned to positive and neutral mood interventions would report improved moods. We expected that individuals asked to again discuss an emotional event during the second interview would be more likely to request referral or support, compared to other respondents.

Measures

All measures and the timing when they were administered are shown in Table 1.

TABLE 1.

Timing of Study Measurements.

	Screening	Initial Interview		Second Interview
		Pre- Discussion of Event – Time 1a	Post- Discussion of Event – Time 1b	Pre- Intervention – Time 2a	Post- Intervention– Time 2b
Depression	X
PTSD	X
Recent trauma, loss, psychiatric admission	X
Mood		X	X	X	X
Stress		X	X	X	X
Demographics		X
Crying			X
Reactions to Research Participation Questionnaire			X		X
Post-survey impacts				X
Current/continued upset and referral			X		X

DEMOGRAPHIC INFORMATION

Participants reported their age, ethnic background, marital status, occupation, and amount of education at baseline.

MOOD

The Profile of Mood States short form (Shacham, 1983) consists of 37 mood adjectives; each is rated on a 5-point scale. Scores are summed for each of the six subscales, and a Total Mood Disturbance (TMD) Score that reflects the individual’s current mood is calculated by summing scores on the subscales of Depression, Confusion, Tension, Anger, and Fatigue, and subtracting the Vigor score. Higher scores indicate greater mood disturbance. Across the four administrations, the alpha reliability coefficients for the depression, tension, anger, fatigue, and vigor subscales all ranged from 0.85–0.93. Alpha coefficients for the confusion subscale ranged from 0.57–0.68 across administrations.

STRESS

Seven adjectives composing the stress subscale of the Arousal Adjective Check List (AACL) were rated on a 4-point scale from “not at all” to “very much” (e.g., stirred up, relaxed, tense) before and after each of the two interviews (Bohlin & Kjellberg, 1973; Kjellberg & Bohlin, 1974). Scores ranged from 0–21, with higher scores indicating greater subjective stress (i.e., negative high arousal). Alpha coefficients ranged between 0.82–0.86 across the four administrations.

CRYING

After the initial interview, interviewers rated if they had been aware that the respondent had cried during the interview or not. Respondents were also asked if they cried during the interview or if they were close to tears.

CURRENT UPSET AND INTEREST IN REFERRAL OR MENTAL HEALTH SUPPORT

With items adapted from work with individuals who experienced effects of terrorism in New York City (Galea et al., 2005), we asked about respondents’ level of upset and interest in further assistance with emotional issues. Specifically, at the conclusion of each interview we asked if the interview questions were upsetting and if respondents were still upset. All respondents were offered contact information for a national crisis hotline, a phone call from a psychologist (within the next few days or right away), or a list of community resources. The list of resources contained phone numbers and websites for a variety of issues, e.g., domestic violence, drug abuse, and child abuse. Respondents were offered these options prior to ending the contact with the participant after each interview.

CONTINUED UPSET

Using items adapted from Surtees et al. (2006), respondents were asked to rate (1) how upset they were by the initial interview (impact), and (2) how much they felt over it now (adaptation); each on a 4-point scale. They were also asked how many hours it took them to “get over” the upset from the initial interview. These items were administered at the beginning of the phone call on the second day.

REACTIONS TO RESEARCH PARTICIPATION QUESTIONNAIRE (RRPQ)

Developed by Newman et al. (2001) to measure a respondent’s opinion about the experience of study participation, the RRPQ consists of one item regarding motivation and 23 on which individuals rate their experience. Factor analysis has yielded five factors: Personal Satisfaction, Personal Benefits, Emotional Reactions, Perceived Drawbacks, and Global Evaluation. The full RRPQ was administered at the end of the first day’s interview. The alpha coefficients for the subscales were as follows: Personal Satisfaction (0.57), Personal Benefits (0.79), Emotional Reactions (0.82), Perceived Drawbacks (0.72), and Global Evaluation (0.72). The Emotional Reactions scale was also administered at the end of the second interview (alpha = 0.77). The Emotional Reactions and Perceived Drawbacks scales were coded such that higher scores represented more negative reactions, and the Personal Satisfaction, Personal Benefits, and Global Evaluation scales were coded such that higher scores reflected more positive reactions.

IMPACT OF EVENT SCALE – REVISED (IES)

Developed by Horowitz, Wilner, and Alvarez (1979), the IES is a 15-item measure that was designed to measure responses to stressful life events. Cluster analyses yielded two subscales, i.e., Intrusion and Avoidance. The Intrusion subscale contains items that refer to intrusive thoughts or images, while the Avoidance subscale contains items reflecting avoidance of reminders or thoughts about the event. Each item is rated in terms of the frequency with which it occurs. In this study, the IES items were used to assess the extent to which the interview was a stressful event that resulted in frequent intrusive thoughts or efforts to avoid. Specifically, respondents were asked to use this scale to rate the effects of the interview, not of the original event itself. The internal consistency (i.e., coefficient alpha) of the subscales in our sample was acceptable (intrusion = 0.78; avoidance = 0.75). In addition, individuals were asked two items to assess more general impacts. Specifically, they were asked to rate on 4-point scales how positive or negative was participating in the initial interview, and also their overall rating of the impact of participation.

Results

Characteristics of the final sample of respondents (n = 395) are presented in Table 2. These respondents were predominately female (53.9%), with an average age of 52 years (SD = 17.6) and 14.8 years of education (SD = 2.6). Approximately half were non-Hispanic white (52.7%); a third were African American (34.9%); and 7.6% were Hispanic. A plurality were married (45.1%); 31.4% were single and 21.0% were separated/divorced or widowed.

TABLE 2.

Sample Characteristics.

	(N)	Percent
Race/Ethnicity
White	(208)	52.7
African American	(138)	34.9
Hispanic	(30)	7.6
Other	(21)	5.3
Gender
Male	(182)	46.1
Female	(213)	53.9
Marital Status
Married	(178)	45.1
Single	(124)	31.4
Separated/Divorced/Wiowed	(83)	21.0
Other	(10)	2.5

	(N)	Mean	(SD)	(range)
Age (in years)	(394)	52.0	(17.6)	(18–93)
Education (in years)	(394)	14.8	(2.6)	(8–19)

Attrition Analysis

For those completing both interviews, the average time interval between interviews was slightly more than two days: 53.7 hours (SD = 17.8; range = 17.2–166.2). Bivariate attrition analyses (not shown) were conducted to determine if respondents completing both interviews (n = 316) differed from those completing only the first interview (n = 79). These analyses revealed no differences by gender, education, marital status, stress, and mood disturbance (measured at the end of the first interview – Time 1b). Younger respondents, and those representing minority race/ethnic groups, though, were less likely to complete both interviews, compared to older and white respondents. A logistic regression model (not shown) that examined attrition simultaneously across all items identified status as African American as the only characteristic independently associated with a lower likelihood of completing both interviews (OR = 0.49, CI = 0.26, 0.90, p < .01).

Discussion of Distressing Topics

As part of the initial interview, respondents spent an average of 15.4 minutes (range = 12.0–51.6) discussing the participant’s chosen distressing topic (this does not include the time spent answering the study’s measures and scales). For the second interviews, the topic corresponded to the respondent’s randomly assigned condition (i.e., distressing event again, neutral event, positive event). An average of 6.8 minutes (range = 6.0–21.9) was spent in discussion of the assigned topic during the second interview.

Interviewers recorded a verbatim description of the respondent’s distressing event at the beginning of the initial interview. The contents of the distressing events were available for 394/395 interviews. These data were coded using the qualitative data analysis software Atlas. ti version 6.2.28 (www.atlasti.com). Table 3 shows the categories of events discussed by respondents, as well as several examples within each category. Of note, 35% of the interviews were about deaths, with medical crises and conflicts as the next most frequent categories. A review of the examples provided in Table 3 demonstrates that the topics of discussion were truly distressing and important events for our respondents, with very few of the discussions focused on mundane issues.

TABLE 3.

Content of Distressing Events Reported by Respondents.

Category	Examples (quotes from respondents)	N	%
Death	Father died; co-worker’s suicide; lost child in pregnancy; best friend was killed in a murder	140	35.0
Medical crisis or significant medical problem	Finding out sister had breast cancer; son was paralyzed; finding out I had lung cancer	59	15.0
Conflict/argument	Had a falling out with mother-in-law; marital problems	44	11.2
Major problem at work/ firing	Job loss; overbearing supervisor; forced resignation as a teacher	37	9.4
Relationship end	Domestic partner left after 10 years; going through divorce	21	5.3
Violence/crime victim/ threat	Domestic violence with husband; witnessed shooting of a young person; physically abused	17	4.3
Accident	Victim of a hit-and-run accident; hit by a car	15	3.8
Interpersonal stress	Son’s medical questions; sister moved away	13	3.3
Financial stress	Insurance going up 25%; home foreclosure	9	2.3
Legal problems	Arrested in 2007; blamed for possession of a firearm	7	1.8
Other frustrations	Three friends smashed up brand-new car; forced to evacuate home by the police	7	1.8
Personal stressors, other	Coming out to people about being gay; telling my Mom I was pregnant	5	1.3
Problems with living situation/eviction	Apartment building burned; bad living situation; eviction	5	1.3
Natural disaster	Earthquake in Haiti; hurricane in Cancun	2	0.5
Racism	Victim of racial profiling; dealt with a racist professor	2	0.5
Unable to categorize	Went through a bomb scare; daughter had trouble in high school; parking at the garage	16	4.1
Total		399*

^*Note that 394 items were coded; 5 items were coded in 2 categories.

Changes in Mood and Stress During First Interview

We next employed pairwise t-tests to examine our hypothesis that moods and stress would increase from the beginning to the end of the initial interview (Table 4). These analyses indicate that the mean values of both the mood disturbance (TMD) and stress (AACL) measures increased significantly between the start and finish of the first interview (Time 1a to 1b). The analyses confirmed our expectation that asking respondents to discuss negative personal experiences would increase their negative affect relative to baseline levels. Additional support for the idea that respondents were emotionally impacted by the discussion of the upsetting event comes from the items that assessed crying by the respondent during the initial interview. Interviewers indicated that eighteen respondents (4.6%) were perceived as crying during the interview. Thirty-one of the respondents themselves (7.8%) said they cried, and 59 (14.9%) said they were close to tears.

TABLE 4.

Pairwise T-tests: Mood and Stress from Pre-post Initial Interview.

	Mood Disturbance (TMD)	Stress (AACL)
Time 1a
Mean (SD)	0.60 (2.19)	4.50 (4.05)
Time 1b
Mean (SD)	0.96 (2.60)	5.63 (4.73)
Difference
Mean (SD)	−0.36 (1.85)	−1.13 (3.52)
t-value (df)	−3.71*** (359)	−6.31*** (387)

^***p<.001

Changes in Mood and Stress Between Interviews

Pairwise t-tests were also conducted to examine our hypothesis that elevated negative affect would begin to recover from the end of interview 1 to the beginning of interview 2 (an average of two days later). Table 5 indicates that the mean values of both mood disturbance and stress at the start of the second interview (i.e., Time 2a) were significantly lower than they had been at the conclusion of the first interview (Time 1b). In fact, both mood disturbance and stress at Time 2a were lower than at baseline (Time 1a).

TABLE 5.

Pairwise T-tests: Mood and Stress from End of Interview 1 to Beginning of Interview 2.

	Mood Disturbance (TMD)	Stress (AACL)
Time 1b
Mean (SD)	0.91 (2.54)	5.51 (4.80)
Time 2a
Mean (SD)	0.08 (1.99)	3.78 (3.86)
Difference
Mean (SD)	0.83 (2.03)	1.73 (3.87)
t-value (df)	7.01*** (292)	7.91*** (311)

^***p<.001

Post-Survey Impacts Overall, and on Avoidance and Intrusion

Participants were asked to rate the overall impact of their participation in the initial interview as well as its valence. A majority of the respondents, 68.4%, rated the initial interview as having no effect (N = 96) or little effect (N = 120), while only 31% said it had a moderate (N = 85) or extreme (N = 13) effect on them (and two didn’t know). They were also asked to rate the valence of the impact of the initial interview. None of the respondents rated their participation as negative, and 22 (7%) rated the interview as “both positive and negative but more negative.” An overwhelming majority, 90%, rated their participation as either positive (N = 189) or “both, but more positive than negative” (N = 96), with nine people not responding.

We next employed multiple linear regression (see Table 6) to investigate the degree to which individuals with more negative moods and greater stress might report more negative post-survey impacts (at Time 2a) on the Avoidance and Intrusion subscales (IES). Columns 1 and 2 report regression results for the avoidance measure, which was assessed prior to the second interview (Time 2a). Stress and mood disturbance, measured after the initial interview (Time 1b), were not associated with subsequent Avoidance. These measures were also not associated with Intrusion (columns 3 and 4 of Table 6). We next examined the question of whether or not individuals with stronger emotional reactions would report greater post-survey impacts. The models in Table 6 also provide evidence with which to evaluate this question, as the five subscales of the Reactions to Research Participation Questionnaire (RRPQ), assessed at Time 1b, were additionally included as predictors. Of these, one was found to be associated with Avoidance: the Emotional Reactions subscale. Stronger emotional reactions during the research were associated with increased avoidance of the interview content in the time between the two interviews. Two of the RRPQ subscales were also predictive of Intrusion. As with Avoidance, stronger emotional reactions were associated with increased intrusiveness; greater perceptions of personal benefits were also associated with increased intrusiveness of the initial interview.

TABLE 6.

Regression Models of Post-survey Impacts.

	Avoidance (Time 2a)		Intrusion (Time 2a)
	Unstandardized Coefficients	(Standard Errors)	Unstandardized Coefficients	(Standard Errors)
Race/Ethnicity (ref=white)
African American	−0.27	(0.44)	−0.30	(0.43)
Hispanic	−0.32	(0.84)	−0.45	(0.79)
Other	0.14	(0.96)	0.09	(0.92)
Gender (ref=male)
Female	−0.04	(0.37)	−1.24***	(0.36)
Marital Status (ref=married)
Single	0.28	(0.42)	0.49	(0.41)
Separated/Divorced/Widowed	0.23	(0.53)	0.65	(0.51)
Age (in years)	0.01	(0.01)	0.02	(0.01)
Education (in years)	−0.19	(0.07)	−0.09	(0.07)
Mood Disturbance (TMD) – Time 1b	0.05	(0.11)	0.05	(0.11)
Stress (ACCL) – Time 1b	0.11	(0.07)	0.10	(0.06)
Participation (RRPQ) – Time 1b	0.03	(0.15)	−0.01	(0.15)
Benefits (RRPQ) – Time 1b	0.10	(0.08)	0.24***	(0.07)
Emotional Reaction (RRPQ) – Time 1b	0.21***	(0.06)	0.26***	(0.05)
Drawbacks (RRPQ) – Time 1b	0.15	(0.08)	−0.03	(0.08)
Global (RRPQ) – Time 1b	0.06	(0.12)	0.04	(0.12)
Adjusted R-square (N)	0.17 (281)		0.26 (285)

^*p<.05

^**p<.01

^***p<.001

Recovery Time

Additional analyses examined the self-reported amount of time respondents indicated was necessary to “get over” the upset associated with discussing the distressing experience during the first interview. At the beginning of the second interview (Time 2a), respondents reported that the time necessary to recover from the initial interview ranged from 0 to 72 hours. The average number of hours was 5.82 (SD = 11.72). The median number of hours needed was 0.5, and the modal number was 0. When examined in a multiple linear regression model (not shown), none of the following variables were found to be predictive of the reported amount of time necessary to “get over” the first interview: gender, race/ethnicity, age, education, mood disturbance, or stress (these latter two variables measured at Time 1b). Recovery time did systematically vary by marital status, however, as persons who were separated, widowed, or divorced required longer recovery times than did those who were currently married (B = 4.87, SE = 2.29, p < .05).

Intervention Effects

The effects of the intervention on mood disturbance (TMD), stress (AACL), and emotional experience at Time 2b were next examined with multiple linear regression. Compared to respondents asked to recall an upsetting event, those asked to discuss a neutral or a positive experience subsequently reported significantly lower mood disturbance scores (Time 2b; see columns 1 and 2 of Table 7). This effect was independent of demographic characteristics and mood disturbance scores at the beginning of the interview. An additional model (not shown) that contrasted the positive and neutral conditions did not find them to significantly differ. In examining the effects of the intervention on stress (AACL; columns 3 and 4), respondents in the positive condition, but not the neutral condition, reported significantly less stress than those assigned to the distressing condition. The evidence thus suggests that those who again discussed the emotional event reported greater negative mood than those who were assigned to the positive and neutral interventions, and greater stress than those assigned to the positive intervention.

TABLE 7.

Regression Models Examining Mood, Stress, and Emotional Reactions; Second Interview.

	Mood Disturbance (Time 2b)		Stress (Time 2b)		Emotional Reaction –RRPQ (Time 2b)
	Coefficients	(Standard Errors)	Coefficients	(Standard Errors)	Coefficients	(Standard Errors)
Race/Ethnicity (ref=white)
African American	−0.48**	(0.17)	−0.49	(0.24)	0.12	(0.42)
Hispanic	−0.33	(0.31)	−0.54	(0.44)	−0.42	(0.76)
Other	0.18	(0.37)	0.33	(0.53)	−0.71	(0.86)
Gender (ref=male)
Female	0.04	(0.15)	0.20	(0.21)	−0.26	(0.36)
Marital Status (ref=married)
Single	−0.08	(0.17)	0.12	(0.24)	0.49	(0.41)
Separated/Div/Widowed	−0.18	(0.21)	−0.04	(0.30)	0.03	(0.51)
Age (years)	0.00	(0.01)	−0.01	(0.01)	0.03**	(0.01)
Education (years)	−0.02	(0.03)	−0.001	(0.04)	−0.19**	(0.07)
Intervention (ref=distressing)
Neutral	−0.49**	(0.17)	−0.17	(0.24)	−0.97**	(0.41)
Positive	−0.85***	(0.17)	−0.69**	(0.25)	−0.70	(0.41)
Mood Disturbance (TMD) – Time 2a	0.70***	(0.04)	—	—	—	—
Stress (AACL) – Time 2a	—	—	0.79***	(0.03)	—	—
Emotional Reaction (RRPQ) – Time 1b	—	—	—	—	0.36***	(0.05)
Adjusted R-square (N)	0.59 (299)		0.76 (309)		0.23 (308)

^*p<.05

^**p<.01

^***p<.001

Note: All coefficients are unstandardized.

Columns 5 and 6 present findings for a model that examines the effects of the intervention on the RRPQ Emotional Reactions subscale (measured at Time 2b). Here, respondents in the neutral condition reported emotional reactions significantly lower than those reported in the distressing condition. Respondents in the positive condition did not vary significantly from those in the distressing condition in terms of emotional reactions.

Use of Safety Scripts

One major concern was that participation in this research would place respondents at risk of harm due to negative emotions induced by participation. To evaluate this, we reviewed data for respondents for whom interviewers accessed our safety scripts, i.e., items designed to assess their safety and provide help if warranted. Respondents were asked these questions if they provided specific answers to items (e.g., BDI suicide item) or if they seemed highly distressed to interviewers. Only 1.5% of the sample (n = 6) were directed through these safety items. A review of the cases indicated that two respondents were upset (one by a PTSD screening question, and the other during the initial discussion of the distressing event). Both of these individuals were contacted by the PI, reported no longer being upset, and wished to continue with the research. Of the four others, two misunderstood a BDI question, one seemed to be under the influence of a substance, and one was misdirected to these items due to interviewer error (the interviewer felt the respondent would not be eligible and skipped out of the interview early to be able to offer referrals to the respondent); these four respondents did not continue with the research. After screening with the safety items, none of these six individuals required immediate intervention or a call to 911 for help. In addition, after the initial interview, all participants were asked if they were “OK to continue” with the measures. No one accepted an offer to wait prior to going on with the post-interview measures, and only one participant said he was too upset to continue participating at that moment. (He was contacted by the PI and acknowledged strong emotions at the time, but he went on to complete the study.)

Referral Requests

Another way to evaluate potential harms was to ask respondents about their interest in a referral or support— this occurred after each interview. Table 8 presents the frequency with which various resources were requested by respondents. Here it can be seen that no respondents requested an immediate call from the PI to address issues of emotional distress. After the initial interview, ten individuals (2.6%) did request a call within the next few days. When contacted, four people wanted additional help with the issue they had discussed (but denied additional distress due to the survey itself); all were provided with support and offered referrals for ongoing mental health treatment. (There was also one individual who was determined to be ineligible during the screener and requested a phone call—he was treated the same way.) The remaining six denied any needs (3), misunderstood (1), did not return the call after several attempts (1), or were interested in more information about the study (1). A larger proportion of individuals, 78 people (19.7%), requested the PI’s phone number, but only four called. Two of these requested help with psychosocial issues and were provided with referrals for psychological treatment. The other two had complaints about the timing of the call for the second interview. Respondents were also offered the number for a crisis hotline, and 48 (12.2%) were interested in this number at Time 1b. Finally, 96 individuals (24.3%) were interested in receiving a list of community resources. Overall, of the 395 people who completed the initial interview, 274 (69.4%) reported no need for any resources.

TABLE 8.

Resources Requested by Survey Respondents.

	Initial Interview (n=395)		Second Interview (n=316)		Distressing Condition (n=113)		Neutral Condition (n=103)		Positive Condition (n=100)
	n	%	n	%	n	%	n	%	n	%
PI – immediate call	(0)	0.0	(0)	0.0	0	0.0	0	0.0	0	0.0
PI call in next few days	(10)	2.6	(3)	0.9	2	1.8	1	1.0	0	0.0
PI phone number	(78)	19.7	(14)	4.4	8	7.1	2	1.9	4	4.0
Hotline number	(48)	12.2	(16)	5.1	5	4.4	6	5.8	5	5.0
Resources list	(96)	24.3	(23)	8.4	11	9.7	6	5.8	6	6.0
Declined all resources	(274)	69.4	(279)	88.3	98	86.7	92	89.3	89	89.0
Total resources provided	(232)	—	(53)	—	24	—	14	—	15	—

Note: Respondents could request multiple resources.

After the second interview, respondents were again asked if they wanted any of these referrals/support. Table 8 shows that there were slightly more requests for resources, including a call from the PI, the PI’s phone number, and the resources list, from individuals in the Distress condition. Multiple resources could be requested by the same individual, however, and the total number of persons in each condition requesting any resources were 14 (12.4%) in the distressing condition, 11 in the neutral condition (10.3%), and 11 in the positive condition (11.0%). These differences were not significant.

Discussion

The main purpose of this study was to determine the frequency and severity of emotional reactions in response to a distressing survey in a sample of adults who were selected to be a non-at-risk population for emotional distress. Our results show that moods became significantly more negative and stress levels increased from pre- to post-interview. However, the negative moods and stress that were elicited by the initial interview returned to baseline levels within two days, and most respondents reported they recovered from the interview almost immediately. In addition, stress and mood disturbance at the end of the initial interview did not predict willingness to continue participation in the second interview. We view these changes in mood and stress levels as typical reactions to events in an individual’s daily life, rather than adverse events that warrant concern.

Further, a majority of respondents rated the initial interview as having only a minimal effect on them, and most rated the impact of the interview as positive. Additional evidence for the lack of harm due to participation comes from the data on requests for help. That is, there were no adverse events, i.e., no respondents stopped participating for emotional reasons, and none required immediate intervention (calls to 911 or to PI) for harms associated with the research. Further, of the respondents who requested a call from the PI in the following few days, none were still distressed due to the study when contacted, although several did want additional support to address the issue they had discussed in the study. The majority of respondents were not interested in any of the options for referral or support that we offered to them.

We were also concerned about impacts that might occur between the two interviews. It was our expectation that induced negative moods and stress during the initial interview would result in additional impacts on the individuals during the time between the two interviews. Results, however, indicated that neither stress nor mood disturbance levels post-interview were associated with avoidance of and intrusiveness of the material in the two days following. However, stronger emotional reactions (RRPQ subscale) during the interview were associated with both Avoidance and Intrusion, and the perception of benefit was associated with intrusive thoughts of the interview in the two days following it. These findings seem to suggest that there is some impact of the discussion of a distressing topic in a survey interview that lasts beyond the interview itself, but the impacts are more a result of general emotional reactions and perceptions of benefit rather than specific changes in aspects of mood (e.g., anger, depression) or stress levels.

An additional issue addressed in this study was the feasibility of developing brief interventions that can be used to ameliorate negative moods elicited by a survey. We were able to show that brief discussions of positive or neutral topics improved moods significantly (and stress levels followed the same pattern). These data provide preliminary results on brief and novel interventions that can be utilized to aid researchers in their efforts to protect human subjects, in addition to more traditional strategies (e.g., hotline phone numbers, calls from a counselor).

Best Practices

There were several important strengths of this study. One was that each respondent chose to discuss an event that was personally distressing to him/herself, rather than responding to a standardized event chosen by the researchers. The content analysis above indicated that respondents took these interviews seriously, and chose topics that appear to be objectively distressing, e.g., deaths, serious medical problems, violence. While the discussion of these events often resulted in strong emotional reactions (noted in their mood scores or reports of crying), it did not result in harm to the individual. Further, the use of a prospective, pre-post design and the use of a random community sample of adults lend confidence to our findings and their generalizability.

It is important to address several limitations of this research. We were unable, for example, to follow up with the 20% of the eligible sample who did not complete the second interview. Although we consider the 80% successful follow-up rate to be excellent, it is possible that some of those who did not participate in the second interview were unavailable due to negative reactions to the first interview. Our attrition analyses, however, found no differences in mood and stress scores at the end of the first interview between those who did and did not complete the second interview. Another possible limitation is that the neutral condition, which asked respondents about their opinions regarding smoking bans in restaurants, may not have been neutral for all respondents. In particular, those who were smokers may have held strong opinions about this topic and felt they were negatively affected by the ban. Also, those believing they are adversely affected by secondhand smoke (e.g., persons with asthma or other chronic health conditions) may have had strong positive feelings about the smoking ban. Hence, the neutral condition may have had variable effects on some respondents.

Finally, while our extensive eligibility screening was designed to protect respondents in this first study of intentionally induced emotions in surveys, it limits the generalizability of our findings as our sample was not representative of a “normal” population. From an ethics perspective, our approach to the design of this study was quite paternalistic. That is, based on the literature, we decided who was vulnerable to harm and excluded those individuals from participating in the study. Another, and perhaps more appropriate approach, would be to explain potential risks to respondents and let them decide if it is in their best interest to participate. Our concern going into this project was that we did not have clear evidence regarding potential risks and benefits to provide to respondents during the consent process. Now, however, our data, in conjunction with studies by others (e.g., Cromer et al., 2006; Yeater et al., 2012), provides us with more information on risks and benefits that we can communicate to enable potential respondents to make an informed decision about their participation, rather than us declaring them ineligible if they meet any of a list of criteria.

Research Agenda

Surveys with the most potential for emotional harms to respondents typically involve at-risk populations. For example, to study PTSD one must survey individuals with PTSD, and to study a parent’s reaction to the death of a child, researchers must study parents who have lost children. Many surveys reviewed above have already been done with at-risk populations, but we have little data on the impacts of these on the respondents. Given the findings reported here with non-at-risk adults, we encourage further work to continue to refine interventions for alleviating distress in at-risk individuals who may experience it. Ultimately, researchers should have tools to study distressing issues as well as confidence that they are adequately protecting their respondents in the process.

Another question relates to the role of benefits in research of this type. As noted above, many respondents do find benefit in surveys with potentially distressing content, and one might expect that an interview (such as the one in the present study) might provide greater benefit to respondents, as it is more personal than a paper-and-pencil survey. In the present study, perceived benefit was associated with intrusiveness of the interview in the two days following. The implications of this are not yet clear. Future research should be designed to provide more information on perceived benefits in studies of this type, and also to delineate their role in decision making when potential respondents are asked for consent to participate.

Educational Implications

These results provide strong support for the idea that nonvulnerable respondents can discuss distressing topics during survey interviews, experience negative changes in their moods and increased stress, yet remain unharmed by this experience. In addition, they recover quite quickly. In terms of risk to human subjects, then, we can be comforted that research of this type, if done with nonvulnerable respondents, poses little risk to them. From these data, we can conclude that surveys of distressing topics, with adequate protections in place to manage distress if it occurs, should be considered minimal risk research for non-at-risk populations. Education of IRB members about research results of this type will be useful to their future assessment of risks posed by surveys on sensitive topics. Certainly the survey topic, the population, and the specific protections in place will need to be considered to make appropriate risk determinations for specific studies.

Biographies

Susan Labott is Professor and Director of Health Psychology in the Department of Psychiatry, University of Illinois at Chicago. She currently serves as Chair of the Social/Behavioral IRB at the University of Illinois at Chicago. For many years, the IRB has been making determinations of risks that subjects may be subjected to, but with little empirical information to guide those determinations. She also has done studies on emotional expression and its implications for research participants.

Timothy Johnson is Professor and Director, Survey Research Laboratory, University of Illinois at Chicago. He serves as Vice-Chair on the Social/Behavioral IRB at UIC. He has over 20 years of experience with survey research and currently directs the Survey Research Laboratory at UIC. He has a keen interest in the intersection of ethical issues with survey research. Drs. Labott and Johnson shared responsibility for project implementation and preparation of this manuscript.

Michael Fendrich is Professor and Director, Center for Applied Behavioral Health Research, Helen Bader School of Social Welfare, University of Wisconsin–Milwaukee. He is a former IRB chair, and has studied human subject protection issues in sensitive research. His role in this project was as an independent consultant to monitor human subject safety and protections.

Norah Feeny is Professor and Director, PTSD Treatment and Research Program, Department of Psychology, Case Western Reserve University. Her role in this project was to aid in the development of the exclusion criteria and also to monitor human subject protections during the data collection period.