Crowdsourcing as a Method for Collecting Data Pertaining to the Effects of Alcohol on Perceptions of Partner Aggression

Abstract

Background:

Efforts designed to investigate the effects of recent alcohol use on the perception of intimate partner aggression have been stultified by significant financial and logistical barriers which warrant the development of supplemental research methods that may result in more prolific investigation of the phenomenon.

Objectives:

The current study explored the viability of using online crowdsourcing to assess the effects of recent alcohol use on the perception of partner aggression.

Method:

Mechanical Turk was used to recruit a convenience sample of 60 males who were asked to provide information on their own use of partner aggression, their most recent episode of alcohol use, and their perception of the behaviors and characters depicted in a written partner aggression vignette. Data were evaluated using five separate hierarchical multiple regression models predicting participant perception.

Results:

Analyses revealed that 35% of the sample had used partner aggression in the past year and that 22% of the sample had consumed alcohol in the past day. Nonviolent participants perceived the aggressor and the behavior more negatively than partner violent participants. Some indicators revealed that recent alcohol use was associated with more positive perceptions of partner aggression.

Conclusion:

Expected associations among prior partner aggression, recent alcohol use, and perception of partner aggression vignettes were observed. Crowdsourcing may represent a source for data evaluating the effects of recent alcohol use on perceptions of aggression. Methodological refinement will benefit research and, ultimately, clinical prevention and intervention.

Prior research suggests that alcohol may affect the ability to perceive acts of physical and sexual intimate partner aggression (IPA),¹ such that intoxicated participants respond more favorably to aggressive acts or individuals than participants who have not consumed alcohol.² Research designed to examine the effects of recent alcohol use on perceptions of IPA has been hindered by the complexity of experimental designs, including recruitment difficulties, participant burden, high costs, the time-consuming nature of the work, and ethical restrictions to healthy, moderate drinkers.^2,3 These factors limit the generalizability and feasibility of conducting sufficient research to examine the effects of proximal alcohol use on perceptions of IPA. Alternative methodologies must be evaluated to supplement and extend sparse research. The current study involved recruiting a pilot sample to evaluate the use of an online crowdsourcing option to assess the effects of recent alcohol use on the perception of IPA depicted in a partner violence vignette.

Prior Research

The effects of alcohol on perceptions of IPA have been most consistently explained through the direct, psychopharmacological effects of alcohol that impair optimal social information processing and higher-order executive cognitive functioning.⁴ Prior research provides support for proximal etiological models of alcohol effects, showing that participants who have recently consumed alcohol perceive violent scenarios differently than participants who have not recently consumed alcohol.⁵ Proximal alcohol use in experimental research most often refers to the period of time immediately following peak absorption and before a significant amount of alcohol is metabolized, between 20 and 60 minutes post consumption.⁶

Research using written, audio, and visual vignette paradigms has shown that, when controlling for potentially spurious factors, assignment to an alcohol condition rather than a control condition results in more favorable attitudes toward IPA and IPA perpetrators depicted in stories involving partner aggressive behavior.^7,8,9 Similar relationships between alcohol condition and ratings of video violence depicted in simulated eyewitness studies have been detected with control participants more likely to unilaterally attribute guilt to the male character than intoxicated participants.² It remains difficult to generalize results when so few studies have been conducted in this area.

There are a number of factors that limit the prolific expansion of research involving the proximal effects of alcohol on perceptions of IPA. From a practical perspective, alcohol administration studies incur substantial costs with participant compensation allocated for periods of detoxification, funds required for third-party transportation to and from the laboratory, laboratory space, laboratory equipment, and advertising. Alcohol administration studies also require a significant time commitment for recruiting and lengthy sessions that may extend across several hours of participation and detoxification. Researchers may experience rejection or delays in the approval of protocols involving alcohol administration. Further, alcohol administration research has been criticized for its limited ecological validity related to the artificial nature of the environment in which research is conducted.¹⁰ Alternative research methods, such as conventional surveys and prospective methods are insufficient to evaluate proximal influences of alcohol on perceptions of aggression and face similar problems with participant burden, high costs, and attrition.^11,12 Online crowdsourcing may provide a viable alternative source of data to supplement, but not replace, controlled alcohol administration research.

Crowdsourcing

Amazon’s Mechanical Turk (MTurk) is a crowdsourcing market that connects a diverse population of “workers” with “requesters” who require human participants to complete a wide variety of online tasks. The popularity of MTurk among social scientists is to be partially attributed to the rapidity of recruitment and nominal compensation required to conduct research.¹³ Evaluation reveals that MTurk produces reliable data and provides access to a more diverse population than available in college or geographically isolated samples.¹⁴ The use of MTurk in the social sciences has grown rapidly over the past decade, including studies on aggression and addiction.^15–18 We are aware of no prior research that has attempted to use MTurk to assess the acute effects of alcohol use on the perception of interpersonal or intimate partner aggression.

The Current Study

The primary aim of the current study was to provide an initial evaluation of the use of an online crowdsourcing option to assess the effects of recent alcohol use on the perception of male-to-female IPA. This involved the recruitment of a small pilot sample of male participants who were asked to provide data pertaining to their most recent alcohol use and to respond to questions assessing their perception of IPA and a male IPA perpetrator in a written vignette depicting an act of male-to-female partner aggression. We hypothesized that 1) crowdsourcing would produce a sample in which a viable percentage of participants had recently consumed alcohol, and 2) analysis of MTurk data would partially replicate laboratory findings such that a) MTurk data would reflect established group differences between previously partner violent (PV) and nonviolent (NV) participants on a vignette measure assessing perceptions of relationship aggression with greater acceptance of aggressive behavior and the perpetrator among PV than NV participants and b) participants who reported recent alcohol use in the past day would be more accepting of aggressive behavior and the perpetrator than participants who had not consumed alcohol in the past day. Further, multiple threshold models posit that various dispositional and situational factors interact to produce event-level responses.⁹ An extension of multiple threshold models suggests that acute alcohol use (i.e., situational factor) would be most closely associated with perceptions of aggressive behavior among individuals who are already predisposed to aggression (i.e., dispositional factor).⁹ Thus, exploratory analyses were used to determine if PV participants who had recently used alcohol approved of aggression more than NV participants who had recently consumed alcohol.

Method

Participants

The current methodological manuscript is part of a larger pilot study intended to evaluate correlates of IPA using an online sample.¹⁹ Data were collected online between March 16, 2017 and March 21, 2017. Eighty-two males responded to an online survey posted to Amazon’s Mechanical Turk. Eligibility criteria for the current study required that participants be English speakers, male, in a relationship, and over the age of 21. Participants were excluded from analyses if they reported that they had been single, divorced, or widowed for the year prior to study completion (n = 18) or if they failed to respond to recent alcohol or past-year IPA questions (n = 4), resulting in a final sample of 60 males. Data provided by females were omitted from the current analyses as the primary aim was to assess the effects of alcohol on perception of male-to-female IPA and female responses to vignettes depicting male IPA are traditionally used to assess perception of risk rather than characteristics of aggression or the aggressor.²⁰

Procedure

Participants were allotted one hour to complete the online study with an average completion time of approximately 21.5 minutes. Consistent with the high end of the standard pay structure of MTurk, participants received $1.00 compensation. Crowdsourcing produced rapid results with all data collected and compensated within a four day period.

MTurk Participants were first directed to an informed consent page and then to a series of measures that included questions about demographic data, alcohol use, partner aggression, and a written vignette all administered through SurveyMonkey. Upon completion of all study measures, participants were instructed to enter a six digit number that could then be used to confirm participation and grant compensation through MTurk. This research was approved by the appropriate Institutional Review Board.

Measures

Alcohol use.

The Alcohol Use disorder Identification Test (AUDIT) is a psychometrically sound, 10-item inventory that assesses alcohol use behaviors and problematic drinking.²¹ The first item of the AUDIT asks participants to describe the frequency of their alcohol use over the past year with options ranging from never to four or more times a week. For the purposes of the current study, this item was used to control for alcohol use frequency in all analyses.

Acute alcohol use.

Each participant was instructed to respond to an item assessing his most recent alcohol use. Participants were given several response options, including currently drinking, within the last day, within the last week, within the last month, within the last year, more than a year ago, and never. Participants were classified as recent alcohol users if they had consumed alcohol within the past day. Alternatively, males who denied alcohol use within the past day were classified as participants who had not recently consumed alcohol.¹²

Partner aggression.

The 20-item Shortened Revised Conflict Tactics Scale (CTS2-S) includes 2-item scales assessing the perpetration of physical assault (α = .82), injury (α = .91), and sexual coercion (α = .75) toward a relationship partner in the past year.²² The brief inventory is consistent with the full Revised Conflict Tactics Scale (CTS2).²³ A composite score for all three subscales was used to dichotomize participants as PV if they endorsed any item and NV if they denied any use of physical assault, injury, or sexual coercion over the past year (α = .94).

Perception of partner aggression.

Participants were exposed to a written vignette depicting a partner violence incident.²⁴ Briefly, the adapted vignette depicted a boyfriend who slapped his partner and threatened her against contact with other men after she had spoken with a male friend. Participants were asked to respond to a short series of questions on a Likert scale with scores ranging from 1 (low / little) to 7 (high / very much). Questions assessing perception of aggression included, “Did [the boyfriend] have the right to behave as he did?”, “How likeable is [the boyfriend]?”, “If you had seen this, how likely would you have been to alert the authorities?”, “How responsible was [the boyfriend] for the incident?”, and “How aggressive was [the boyfriend]?” The last three items were reverse coded such that higher scores indicate higher positive attitudes toward IPA and the male aggressor. Participants were randomly assigned to receive one of three versions of the vignette, each with a different pair of characters (i.e., Rob and Elizabeth; Mohammed and Elizabeth; Mohammed and Khalifa). Outcomes of interest were not associated with vignette condition and results were pooled for the current study.

Analyses

We first report on the results of a chi square analysis used to determine if PV and NV participants were equally likely to report recent alcohol use as well as to determine if participants who had recently consumed alcohol and participants who had not recently consumed alcohol differed in terms of alcohol use frequency. We then conducted a series of hierarchical multiple regression analyses in which alcohol use frequency from the AUDIT was entered as a control variable in block 1, the main effects of recent alcohol use and partner aggression were entered simultaneously in block 2, and the interaction between recent alcohol use and partner aggression was entered in block 3 to predict participant ratings of the vignette. Outcomes included perceptions of partner aggression and a partner aggressive male as depicted in the vignette, including the five items that appear in the IPA vignette described previously. The significance level was set at alpha = .05. Analyses were conducted using IBM Statistical Package for the Social Sciences, version 24 (IBM, Armonk, NY, USA).

Results

Descriptive characteristics

Participants were, on average, 42.3 (standard deviation [SD] = 13.5) years old. Most participants identified as Caucasian (n = 39; 65.0%), Asian (n = 10; 16.7%), or Native American (n = 5; 8.3%), had obtained a college degree (n = 41; 68.3%), and reported an annual income in excess of $25,000 per year (n = 43; 71.7%). Participants had been married for an average of 9.6 (SD = 11.6) years. Twenty-one (35.0%) participants reported past year partner aggression and thirteen (21.7%) reported that they had consumed alcohol in the past day. A comparable number of PV (n = 8, 21.5%) and NV (n = 5, 23.8%) participants reported alcohol use in the past day (χ²(1) = .09, p = .77). Participants who had consumed alcohol endorsed more frequent drinking than participants who had not consumed alcohol on the day of the study (χ²(4) = 26.27, p < .001).

Primary Analyses

The first hierarchical multiple regression model focused on whether the male character had the right to behave aggressively. Alcohol use frequency accounted for only 0.1% of the variability in ratings the male character’s behavior (R² = .001, F_1,58 = 0.06, p = .804). Inclusion of the recent alcohol use and partner aggression effects in block 2 accounted for an additional 42.6% of variance in character ratings (ΔR² = .43, F_2,56 = 20.78, p < .001). The addition of the interaction term between recent alcohol use and partner aggression accounted for only an additional 2.5% of variance accounted for (ΔR² = .03, F_1,55 = 2.59, p = .11).

The partner aggression X recent alcohol use interaction term approached significance, justifying exploratory analyses. Follow-up simple slopes analyses revealed that ratings were significantly associated with recent alcohol use among PV participants (b = 1.459, SE = .60, p = .02, d = 1.25) but not NV (b = 0.31, SE = .51, p = .55, d = 0.24) participants. Evaluating main effects, results indicated that PV participants thought more strongly than NV participants that the male character had the right to behave aggressively. Participants who had recently consumed alcohol also thought more strongly than participants who had not recently consumed alcohol that the male character had the right to behave as he did. Models and effects for all primary analyses are displayed in Table 1.

Table 1.

Effects of recent alcohol use and partner aggression on perceptions of aggression depicted in a written vignette.

	Block 2						Block 3
Model	b	SE	p	95% Confidence Interval	d		b	SE	p	95% Confidence Interval	d
Did he have the right to behave as he did?
Recent Alcohol Use	0.77	0.43	.08	−0.09, 1.63	0.57		0.31	0.51	.55	−0.71,1.33	0.22
Partner Aggression	1.89	0.30	.001	1.29, 2.49	1.72		1.62	0.34	<.001	0.94, 2.30	1.38
X Recent Alcohol Use							1.15	0.71	.11	−0.27, 2.57
Would you have alerted the authorities?a
Recent Alcohol Use	0.73	0.66	.28	−0.59, 2.05	0.41		0.34	0.80	.68	−1.27, 1.95	0.19
Partner Aggression	1.04	0.47	.03	0.10, 1.98	0.59		0.82	0.54	.13	−0.26, 1.89	0.46
X Recent Alcohol Use							0.97	1.12	.39	−1.27, 3.21
How likeable is he?
Recent Alcohol Use	−0.13	0.45	.78	−1.03, 0.78	−0.09		0.29	0.55	.60	−0.80, 1.38	0.20
Partner Aggression	1.97	0.32	<.001	1.33, 2.62	1.73		2.21	0.36	<.001	1.48, 2.94	2.13
X Recent Alcohol Use							−1.03	0.76	.18	−2.57, 0.50
How responsible was he?a
Recent Alcohol Use	0.37	0.63	.56	−0.89, 1.63	0.22		0.07	0.76	.93	−1.46, 1.60	0.04
Partner Aggression	0.97	0.45	.03	0.07, 1.87	0.59		0.80	0.51	.12	−0.22, 1.82	0.50
X Recent Alcohol Use							0.74	1.07	.49	−1.40, 2.88
How aggressive was he?a
Recent Alcohol Use	0.19	0.42	.64	−0.65, 1.03	0.16		0.40	0.51	.43	−0.61, 1.42	0.34
Partner Aggression	1.24	0.30	<.001	0.64, 1.84	1.19		1.36	0.34	<.001	0.68, 2.03	1.35
X Recent Alcohol Uses							−0.52	0.71	.47	−1.94, 0.90

Note. Data were modeled using hierarchical multiple regression, outcome ratings range from one to seven, b = unstandardized coefficient, SE = standard error, p = p-value, d = Cohen’s d effect size. All models control for alcohol use frequency in block 1. Recent Alcohol Use (0 = no alcohol used in the last 24 hours, 1 = alcohol used in the last 24 hours); Partner Aggression (0 = no partner violence perpetration in the past year, 1 = partner violence perpetration in the past year).

^aItems were reverse coded so that higher scores would reflect more positive perceptions of aggressive behaviors or the aggressor.

PV participants were significantly less likely to indicate that they would alert the authorities than NV participants. A small but non-significant effect indicated a trend toward not alerting the authorities among participants who had consumed alcohol on the day of participation relative to participants who had not. The interaction failed to reach significance. Alcohol frequency accounted for a significant amount of variance in block 1 (R² = .10, F_1,58 = 6.09, p = .02) whereas block 2 approached significance (ΔR² = .09, F_2,56 = 3.01, p = .06), and block 3 did not (ΔR² = .01, F_1,55 = 0.75, p = .39).

PV participants indicated that they like the male character more [block 1: R² = .01, F_1,58 = 0.85, p = .36), block 2: ΔR² = .40, F_2,56 = 18.86, p < .001, block 3: ΔR² = .02, F_1,55 = 1.84, p = .18], saw the male character as less responsible for the incident [block 1: R² = .02, F_1,58 = 1.26, p = .27), block 2: ΔR² = .08, F_2,56 = 2.51, p = .09, block 3: ΔR² < .01, F_1,55 = 0.48, p = .49], and perceived the male character’s behavior as less aggressive than NV participants [block 1: R² < .01, F_1,58 = 0.02, p = .89), block 2: ΔR² = .24, F_2,56 = 8.85, p < .001, block 3: ΔR² = .01, F_1,55 = 0.54, p = .47]. The main effect of recent alcohol use and the interaction failed to reach significance for all three models.

Discussion

The current pilot study was undertaken to provide an initial evaluation of the viability of using MTurk to collect data pertaining to the effects of alcohol on perceptions of partner aggression. In support of our first hypothesis, the crowdsourcing methodology allowed us to capture a sample in which approximately 20% of the participants had recently consumed alcohol. Particularly in larger samples, a subsample of 20% would allow for reliable analyses of group differences.²⁵ In support of the hypothesis that MTurk data would reflect established group differences between PV and NV participants on perceptions of aggression, analyses revealed that PV participants expectedly perceived the male aggressor and his behavior more positively than NV participants. We detected only partial support for our final hypothesis, that participants who had recently used alcohol would be more accepting of aggression than participants who had not recently consumed alcohol. Trends in the data suggested that participants who had recently consumed alcohol were more likely to believe that the male character had the right to behave aggressively and were less likely to alert the authorities than participants who had not recently consumed alcohol. The crowdsourcing methodology proved promising as it revealed that 20% of the current sample had recently consumed alcohol and analyses resulted in findings that partially replicated laboratory results on the relationships between perceptions of IPA and both recent alcohol use as well as prior partner aggression.^7,8

We found only partial support for direct effects of acute alcohol use on perceptions of partner aggression. The current study employed a third person vignette and required participants to report on their perception of aggression in the absence salient provocation. Alcohol has been shown to enhance positive affect during prosocial interaction.²⁶ Previous research supports the theory of alcohol myopia in which alcohol restricts perception and processing to only the most salient cues, such as enhancing positive affect during prosocial interactions and eliciting greater aggression and justification for aggression following anger provocation.^4,27 In the current vignette, the female partner displayed no aggressive behavior. Future research may determine if the effects of alcohol on positive perceptions of male aggression are more pronounced when a female character initiates or responds with aggression.

Consistent with previous research, exploratory analyses revealed some evidence to suggest that acute alcohol consumption had a greater effect on PV than NV men, such that alcohol use was associated with a stronger perception that the male character had the right to use aggression among PV participants. Multiple threshold models posit that multiple factors influence event-level responding, including substances, external provocation, or physiological discomfort.²⁸ These models further predict that individuals with a greater predisposition to aggressive responding, such as previously partner violent participants, would perceive the male partner’s aggression more positively than individuals with a nonaggressive disposition, particularly following recent alcohol use. Thus, the current data partially conform to the expectations of multiple threshold models.

Research Implications

The use of online crowdsourcing was advantageous to the current investigation in many respects. This method resulted in a full round of participant identification, recruitment, and data collection within four days. The project incurred nominal costs with $1.00 paid to each participant and an additional 45–70% fee paid to MTurk. No budget was allocated for advertising, alcohol, transportation, or infrastructure. Online recruitment further mitigates many of the criticisms of laboratory-based aggression research, including the artificial context and novel nature of direct observation by permitting participants to consume alcohol as they naturally would and to submit responses from a familiar setting.² Of considerable importance, ethical considerations that limit laboratory samples to healthy, moderate social drinkers do not apply to online recruitment methods.^2,3 Thus, data gathered from online crowdsourcing may provide greater insight into high risk drinkers as well as a more generalizable account of the influence of acute alcohol use on perceptions of partner violence.

Further, thirty-five percent of the current sample reported past-year partner aggression with only four participants omitted from analyses due to missing data on partner aggression items. This is comparable to rates of current-relationship male-to-female partner aggression reported across small, community samples.²⁹ Only four otherwise eligible participants failed to provide complete responses to partner aggression or alcohol use items. Thus, anonymous crowdsourcing may represent a viable option for collecting candid data pertaining to socially undesirable behaviors, such as partner aggression and substance use.

Nevertheless, online naturalistic studies should be seen as a supplement to, rather than a replacement for, laboratory-based research in further exploring the effects of acute alcohol use on the perception of IPA. Online studies inherently lack the precision of laboratory-based research to administer precise amounts of alcohol, allow for proper absorption time, and control for confounds. Self-administration prior to participation in online research may be precipitated by and subject to any number of individual differences, such as temporary affective instability or social interaction and distraction, that could influence subjective responding and cannot possibly be fully anticipated or controlled for. Though this lack of control limits our understanding of the direct effects of alcohol on perception of IPA, this methodology allows for a unique degree of ecological validity in that alcohol use may more directly reflect use under the real-world conditions in which participants will observe or participate in genuine aggressive situations. Further, the proliferation of laboratory-based alcohol administration studies remains slow due to various practical impediments. Online crowdsourcing represents a supplemental methodological option to contribute proximal effects data and mitigate the sparsity of laboratory-based alcohol administration research.^3,6

Limitations and future research

The current pilot study utilized secondary data analysis of a sample recruited for other purposes. As such, the current sample was small and underpowered to detect effects of interest. We have reported effect sizes in addition to significance testing to provide greater insight despite small cell sizes. The current results were mixed, suggesting that MTurk may be a useful platform for collecting data pertaining to the effects of recent alcohol use on the perception of IPA. Replication is necessary. Further, researchers are advised to include items assessing individual height and weight as well as the amount of alcohol consumed and the period of time during which it was consumed in order to accurately estimate the participant’s blood alcohol concentration during study participation. This will result in greater standardization of the “alcohol condition” and a more nuanced depiction of the effects of alcohol on perception at various levels of intoxication. These data will still rely upon self-report, which is inferior to objective quantification of alcohol administration in the laboratory but is a more reliable indicator of same-day alcohol use than it would be for distal use.³⁰ Further, the effects of social desirability are diminished under conditions of anonymity and online reporting.^31,32 It is not possible to administer placebo beverages to online samples to control for alcohol’s expectancy effects. Reviews of the literature call into question the validity of placebo beverages in deceiving participants.^33,34 Online research need not make explicit to participants the associations of interest between acute alcohol use and perceptions of aggression prior to participation. Further, data suggest that MTurk workers are a diverse population but a degree of selection and sampling bias should be expected when attempting to generalize results detected among individuals who volunteer to regularly complete online assignments to those who do not. Recent refinements in crowdsourcing, such as TurkPrime, a variant of MTurk, are more versatile for collecting data and administering behavioral measures.

Conclusions

Laboratory-based experimental research remains the strongest methodology for offering insight into the relationship between acute alcohol use and perceptions of IPA but this literature is impeded by significant financial and logistical barriers. The current study offers initial support for the use of online crowdsourcing as a supplemental, rapid, and inexpensive method of data collection that yields diverse samples with a viable amount of acute alcohol use to conduct meaningful analyses into alcohol-related perceptions of partner aggression. Trends in the current data suggested that, consistent with previous research, partner violent participants perceived aggressive behaviors within the vignette more positively and held more positive opinions toward the male perpetrator than nonviolent participants. Similarly, some indicators suggested that participants who had recently consumed alcohol tended to have more favorable views toward partner aggressive behavior than participants who had not recently consumed alcohol. Further methodological refinement will benefit future research efforts to evaluate the effects of alcohol on the perception of partner aggression as well as other alcohol-related outcomes of interest.