Improving measurement of harms from others’ drinking: A key informant study on type and severity of harm

Abstract

Aims

Some types of harms experienced because of others’ drinking (AHTO) may produce greater negative effects than other harms. However, AHTO survey items were developed to measure type, not severity, of harm. We aimed to compare the perceived severity of a comprehensive list of AHTO items to assess consistency in subjective ratings of severity, facilitate a more nuanced analysis and identify strategies to improve measurement of AHTO in epidemiological surveys.

Methods

Thirty-six leaders of national alcohol surveys (conducted between 1997 and 2016) from 23 countries rated the typical severity of negative effects on the victim of each of 48 types of AHTO using a scale from zero (no negative effect) to 10 (very severe negative effect). The survey leaders were also asked to provide open-ended feedback about each harm and the severity-rating task generally.

Results

Of 48 harm items, five were classified as extreme severity (mean rating ≥8), 17 as high (≥6 <8), 25 as moderate (≥4 <6), and one as low (≤4). We used two-way random effects models to estimate absolute agreement intraclass correlation coefficients (AA-ICC) and consistency of agreement intraclass correlation coefficients (CA-ICC). Results showed that there was fair to excellent absolute agreement and consistency of agreement among “experts’’ ratings of the severity of harms from others’ drinking (single measures CA-ICC = 0.414, single measures AA-ICC = 0.325; average CA-ICC = 0.940, average AA-ICC = 0.914). Harms to children, and harms causing physical, financial, practical, or severe emotional impacts were rated most severe.

Conclusions

When designing new AHTO surveys and conducting analyses of existing data, researchers should pay close attention to harms with high perceived severity to identify effective ways to prevent severe AHTO and reduce the negative health and social impacts of AHTO. In-depth analyses of specific sub-sets of harms and qualitative interviews with victims of severe AHTO may prove useful.

Introduction

Alcohol use can result in a range of health and social problems for people other than the drinker (Greenfield et al., 2009; Laslett et al., 2011; Rehm et al., 2017; Rehm et al., 2009; Room et al., 2010; WHO, 2018). Epidemiological surveys have broadly aimed to measure the range of alcohol’s harms to others in numerous societies (Callinan et al., 2017; Callinan et al., 2016; Wilsnack, Greenfield, & Bloomfield, 2018). Also, epidemiological surveys have measured AHTO using a variety of question designs and varying levels of detail. For example, some surveys have included an extensive list of items that are divided into different life domains (such as harms related to the workplace, household, public spaces, and known heavy drinkers; e.g., Callinan et al. (2016), Wilsnack et al. (2018)). Other surveys have included a series of questions about the circumstances of smaller sub-sets of items (e.g., Callinan et al. (2017)), and some include items that measure how frequently harms occurred (e.g., Callinan et al. (2016)). AHTO surveys suggest some factors are related to the severity of harm experienced from others’ drinking – for example, the type and number of harms from others’ drinking differs by gender and the closeness of relationship to harmful drinkers (Laslett, Room, Waleewong, Stanesby, & Callinan, 2019; Stanesby et al., 2018). However, most AHTO items have been developed to measure type, not severity, of harm (Karriker-Jaffe, Room, Giesbrecht, & Greenfield, 2018; Room, Laslett, & Jiang, 2016; Wilkinson et al., 2009).

Identifying the scope and size of different AHTO behaviours is needed to identify the greatest public health needs and guide allocation of resources and policy responses to the most pressing areas. A vital part of this endeavour is quantifying the level of impact of the different types of AHTO on victims (i.e., the product of the severity of negative effects experienced and frequency of their occurrence). The current measures of harms from others’ drinking have allowed researchers to measure the types of harms people have experienced but not the impact of these harms on the victims. Assessing the relative severity of different types of harms from other people’s drinking would enable improved analyses of data collected from epidemiological surveys of AHTO and help to identify areas that require greater public health resources and policy responses. Severity-ranked scores would also make it possible for researchers to calculate summary measures of extent of harm from others’ drinking and to meaningfully categorize harm items and different levels of severity. Severity ratings might also enable deeper interpretation and discussion of findings from population surveys of alcohol’s harms to others.

Conceptual and methodological papers have discussed in the detail the purpose, strengths, weaknesses and potential of epidemiological surveys for researching and understanding AHTO (Dawson & Room, 2000; Karriker-Jaffe et al., 2018; Room et al., 2010; Room et al., 2016; Wilsnack et al., 2018) but measurement strategies that address severity have not been fully developed. An initial step in this direction is to seek key informant input from alcohol research and policy experts in ranking the severity of AHTO. A survey of the opinions of alcohol research and policy experts from numerous countries and cultural backgrounds can be used to identify novel and effective strategies to improve AHTO survey research and ultimately improve understandings of the burden of AHTO in societies.

Aims

Using quantitative and qualitative data collected from a survey of alcohol research and policy experts from numerous countries, this study aims to:

1. derive preliminary estimates of the perceived severity of negative effect on the victim for each type of AHTO;

2. assess inter-rater consistency and variation in ratings of severity of AHTO;

3. compare and rank the perceived severity of different types of AHTO; and

4. identify ways to improve the epidemiological survey research of AHTO.

Methods

Design and sample

Leaders of alcohol surveys conducted as part of the multinational Gender, Alcohol, and Culture: An International Study (GENACIS) and Gender and Alcohol’s Harms to Others (GENAHTO) studies in diverse countries around the world (Greenfield, Bloomfield, & Wilsnack, 2017; Wilsnack, 2017; Wilsnack et al., 2018) were invited to participate in a survey of experts in alcohol research and policy (i.e., “key informants”). Key informants were asked to complete a questionnaire in which they rated the typical severity of negative effects on the victim different types of AHTO. They were also asked to provide feedback about each harm item and the severity-rating task generally. Key informants were told that all participants would be acknowledged in the final paper submitted for publication.

Because pilot work with the severity rating task by research team members in several countries suggested possible gender differences in severity ratings, key informants were asked to invite one other person from their country (someone of the opposite sex to them but not necessarily a researcher) to complete the questionnaire. The aim was to collect one completed questionnaire from a male and one from a female from each country.

While no formal written and signed consent was obtained, the key informants were study directors or their (opposite-sex) colleagues and thus part of the broad study team. The study was explained in detail to invited key informants and they were informed of the intention to publish results from the obtained data obtained. We relied on implicit consent, given the low sensitivity of the responses requested. Those who wished to participate returned completed questionnaires; there were no negative consequences for key informants who chose not to participate.

A total of 40 key informants (25 invited directly and 15 invited indirectly via key informants) from 25 countries returned completed questionnaires. Of the key informants who returned completed questionnaires, one was excluded from analyses because a key informant of each sex had already returned a completed questionnaire for their country; a male and a female were both excluded because they returned a single shared set of ratings for their country that they completed together; and one was excluded because he/she did not rate the severity of a singular instance of each harm item. The final person excluded also factored in the prevalence of each harm in his/her country to his/her ratings.

Thus, the final sample comprized 36 key informants (23 invited directly and 13 invited indirectly via invitation from the key informant that was contacted directly) from 23 countries (Argentina, Australia, Brazil, Canada, Costa Rica, Czech Republic, Denmark, Germany, India, Ireland, Israel, Italy, Netherlands, New Zealand, Sri Lanka, Sweden, Switzerland, Thailand, Uganda, United Kingdom, United States of America, Uruguay, Viet Nam). Thirteen countries had two respondents, six had one male respondent, and four had one female. When two people responded from a country, their responses were averaged before analysis.

Instruments and measures

Informants were given the following instructions:

“Using your knowledge relating to alcohol and alcohol-related harms to others within the context of your country, please rate on a scale from 0 (no negative effect on the victim) to 10 (very severe negative effect on the victim) how severe each harm on the following pages would be considered in your country. Importantly, we are asking you to rate how severe are the negative effects for the victim of each particular harm. We are not asking you to rate how severe of a problem each particular harm is in your country but rather the experience of the typical victim of each harm.

Please make your rating taking into consideration the norms, expectancies, culture, etc. of most people in your country (i.e., how you believe the severity of each harm is perceived or experienced by victims of these harms in your country).

The items were initially developed to measure type of harms and so some may vary greatly in possible severity. Importantly, we acknowledge that the severity of negative effects for victims of each particular harm item may vary depending on the specific circumstances of the harm. For example, a minor traffic accident would be perceived as less severe than a fatal car accident, and harms may vary depending on whether the person harmed is male or female or a younger child vs. an older child. For some of the harm items below the circumstances are relatively vague, and thus the task of rating severity may be ambiguous for these items. For such items, we request that you provide a rating of how severe for the impact on the victim that reflects a level of the harm that is somewhere between the “least severe” and “most severe” version of the harm -- the severity most often associated with this harm in your country. And remember, you are rating the harm that would be experienced by victims of the drinker (not by the drinker).

To better understand how harms are interpreted in different countries, we are also particularly interested in any comments you have about the specific harm items, the harm item severity-rating tasks, and your experience while participating in this study. Please provide any comments you have in the spaces provided below each table or in comments attached to each item.”

Key informants were asked to rate the typical severity of negative effects on the victim of each of 48 types of AHTO with the following instruction: “On a scale of 0 to 10 – where zero is not affected at all; 1 is very minor; 5 is moderate negative impact; and 10 is the most severe negative impact – how much do you think each of these events or conditions negatively affected the victim?” The list of 48 items included in the questionnaire encompassed all AHTO items that are included in the national surveys of the multinational GENAHTO study (Greenfield et al., 2017; Wilsnack et al., 2018) and, thus, represent a comprehensive set of items used in AHTO survey research. Items were presented in four separate lists defined by the victim-perpetrator relationship:

1. Harms to a victim because of the drinking of a stranger or someone the victim does not know (12 items);

2. Harms to a victim because of the drinking of someone the victim knows (e.g., family member, household member, friend, acquaintance) (25 items);

3. Harms to a victim because of the drinking of someone the victim worked with (i.e., co-worker/colleague) or a boss (five items);

4. Harms to a child victim aged 17 or younger because of the drinking of someone (six items).

The specific wording for each item is shown in Table 1.

Table 1:

Mean severity of ratings across all 23 countries by whether item referred to harms to children (marked ^C and shown in black), harms from known people such as friends and family (marked ^K and shown in dark grey), harms from co-workers (marked ^W and shown in light grey), and harms from strangers (marked ^S and no shading)

Harm item	Mean (SD) rating
Mean rating ≥ 8
Physically hurt 2,C	8.90 (1.09)
Being forced or pressured into sex or something sexual K	8.59 (1.65)
Witness to serious violence in the home C	8.23 (1.66)
Being in a traffic accident caused by someone who had been drinking 1,S	8.18 (1.30)
A child protection agency or family services agency was called (because of the drinking of someone) C	8.14 (1.78)
Mean rating ≥ 7 < 8
Being harmed physically 2,K	7.86 (1.93)
Being in a traffic accident caused by someone who had been drinking 1,K	7.71 (1.79)
Being harmed physically 1,2,S	7.59 (1.74)
Having to leave home to stay somewhere else because of someone in the household's drinking K	7.41 (1.53)
Feeling threatened or afraid at home or in some other private setting 1,K	7.41 (1.59)
Being involved in an accident or a close call at work W	7.32 (1.64)
Family problems or marriage difficulties (because of the drinking of someone the victim knows) K	7.24 (1.26)
There was not enough money for the things needed by the child/children (because of the drinking of someone) C	7.23 (1.57)
Mean rating ≥ 6 < 7
Left in an unsupervised or unsafe situation C	6.77 (1.54)
Financial trouble (because of the drinking of someone the victim knows) K	6.76 (1.64)
Having house, car, or property damaged 1,S	6.64 (1.89)
Going without food because of someone in the household's drinking K	6.55 (2.18)
Yelled at, criticised, or otherwise verbally abused C	6.50 (2.60)
Having house, car, or property damaged 1,K	6.41 (1.65)
Caused there to be less money for household expenses (someone the victim knows did this because of drinking) K	6.41 (1.76)
Being emotionally hurt or neglected K	6.32 (1.76)
Took money or valuables that were yours because of their drinking (someone the victim knows did this because of drinking) K	6.14 (1.70)
Mean rating ≥ 5 < 6
Being made afraid by drinkers on the street 1,S	5.91 (1.88)
Being a passenger with a driver who had too much to drink K	5.86 (2.62)
Ability to do the job was negatively affected W	5.73 (1.52)
Not seeing friends or family as much because the victim is embarrassed about someone in the household's drinking K	5.73 (1.78)
Being harassed or bothered at a party or in some other private setting 1,K	5.73 (1.91)
Feeling unsafe in a public place because of a drinker or drinkers S	5.65 (1.90)
Having to work extra hours W	5.59 (1.68)
Being pushed or shoved 1,K	5.59 (1.79)
Being harassed or bothered on the street or in some other public place 1,S	5.48 (1.78)
Being involved in a serious argument 1,K	5.45 (1.99)
Problems with a friend or neighbour (because of the drinking of someone the victim knows) K	5.43 (1.66)
Stopping seeing someone because of their drinking K	5.36 (2.30)
Failed to do something they were being counted on to do (someone the victim knows did this because of drinking) K	5.27 (1.80)
Being annoyed by vomiting, urinating, or littering by a friend, acquaintance, or family member who had been drinking 1,K	5.14 (2.01)
Having clothes or other belongings ruined 1,K	5.09 (1.93)
Productivity at work was reduced W	5.00 (1.93)
Being involved in a serious argument with a drinker who is a stranger 1,S	5.00 (1.83)
Mean rating ≥ 4 < 5
Did not do their share of work around the house (someone the victim knows did this because of drinking) K	4.95 (1.84)
Being pushed or shoved 1,S	4.91 (1.85)
Having a social occasion the victim was at negatively affected K	4.91 (1.95)
Having to ‘cover’ for the drinker W	4.82 (1.65)
Having clothes or other belongings ruined 1,S	4.74 (1.76)
Being called names, or otherwise insulted 1,K	4.32 (2.32)
Being annoyed by vomiting, urinating, or littering by a drinker who is a stranger 1,S	4.26 (2.49)
Being kept awake at night by drunken noise S	4.13 (2.16)
Mean rating < 4
Being called names or otherwise insulted 1,S	3.48 (1.81)
CHarms to children from others’ drinking (Because of the drinking of someone, a child aged 17 or younger was …) Across all harms to children from others’ drinking: Single measures CA-ICC (95% CI) = 0.336 (0.138, 0.768) Single measures AA-ICC (95% CI) = 0.232 (0.085, 0.664) Average CA-ICC (95% CI) = 0.918 (0.779, 0.986) Average AA-ICC (95% CI) = 0.869 (0.672, 0.978)	7.63 *
K Harms from known people’s drinking (The following happened to the victim because of the drinking of someone the victim knows (e.g., family member, household member, friend, acquaintance) Across all harms from known people’s drinking: Single measures CA-ICC (95% CI) = 0.342 (0.227, 0.515) Single measures AA-ICC (95% CI) = 0.234 (0.139, 0.390) Average CA-ICC (95% CI) = 0.920 (0.866, 0.959) Average AA-ICC (95% CI) = 0.870 (0.780, 0.934)	6.15 *
W Harms from co-workers’ drinking (The following happened because of the drinking of someone the victim worked with (i.e., co-worker/colleague) or a boss) Across all harms from co-workers’ drinking: Single measures CA-ICC (95% CI) = 0.342 (0.128, 0.823) Single measures AA-ICC (95% CI) = 0.239 (0.080, 0.739) Average CA-ICC (95% CI) = 0.919 (0.763, 0.990) Average AA-ICC (95% CI) = 0.874 (0.658, 0.984)	5.69 *
S Harms from strangers’ drinking (The following happened to the victim because of the drinking of a stranger or someone that the victim does not know) Across all harms from strangers’ drinking: Single measures CA-ICC (95% CI) = 0.463 (0.285, 0.722) Single measures AA-ICC (95% CI) = 0.363 (0.203, 0.635) Average CA-ICC (95% CI) = 0.950 (0.897, 0.983) Average AA-ICC (95% CI) = 0.926 (0.849, 0.974)	5.50 *
Across all harms from others’ drinking: Single measures CA-ICC (95% CI) = 0.414 (0.320, 0.533) Single measures AA-ICC (95% CI) = 0.325 (0.234, 0.445) Average CA-ICC (95% CI) = 0.940 (0.912, 0.962) Average AA-ICC (95% CI) = 0.914 (0.870, 0.946)	6.12 *

Items in descending order of mean severity

¹Items present in list of harms from strangers’ drinking and list of harms from known people’s drinking

²Items present in list of harms from strangers’ drinking, list of harms from known people’s drinking, and list of harms to children from others’ drinking

^*Arithmetic mean harm item severity rating across countries; CA-ICC: Consistency of agreement intraclass correlation coefficient via two-way random effects model; AA-ICC: Absolute agreement intraclass correlation coefficient via two-way random effects model.

Participants were also asked to record their gender and age and provide feedback about the harm item severity-rating task, their experience of rating each of the four lists of harms, and their experience while participating in this study.

Analysis

Analyses were conducted at the country level; analyses weighted each participating country equally by averaging scores of the two raters when there was more than one informant from a country.

We calculated the mean and standard deviation of severity ratings for each harm across countries (key informants) and across the four sets of harm items (defined by the victim-perpetrator relationship; i.e., stranger, known person, co-worker, child). Paired-samples t-tests tested for differences in the mean severity rating of 10 harms from strangers that had been drinking compared to 10 harms from known drinkers (across key informants who answered all these items). Paired-samples t-tests tested for differences in the mean severity rating of physical harm to someone from strangers’ drinking, compared to physical harm to someone from known people’s drinking, and compared to physical harm to children from someone’s drinking (across key informants who answered all three of these items). Independent-samples t-tests tested for differences in mean severity ratings between male and female key informants (across items answered by all key informants).

Using two-way random effects models, absolute agreement intraclass correlation coefficients (AA-ICC) and consistency of agreement intraclass correlation coefficients (CA-ICC) were estimated to investigate the degree of absolute agreement (i.e., how similar were the raw ratings of the severity of harm items across key informants?) and consistency of agreement (i.e., how similar was the ‘ranking’ of severity of the harm items across key informants?; were certain items rated consistently higher than certain other items across key informants?) among key informants’ ratings of harm item severities (McGraw & Wong, 1996; StataCorp, 2017). The key informant from Uruguay was dropped from ICC analyses due to having missing ratings for 36/48 harm items. To ensure the full suite of harm items were considered in the ICC analyses, missing ratings were replaced with the mean severity rating across all remaining countries for the corresponding harm item rounded to the nearest integer.

All data analyses were completed using Stata Statistical Software (version 15; Stata Corp (2017)).

Important and common themes in the key informants’ open-ended feedback and reflections were identified by OS. Synthesis of key informants’ feedback about the harm item severity-rating task and their experience while rating the AHTO items and participating in this study were used to identify weaknesses in AHTO survey items and the severity rating task. Key informants’ qualitative reflections were also used to contextualize the findings and enhance the discussion.

Results

Part 1: Severity of AHTO items

Mean severity ratings of AHTO items

The mean severity ratings of each harm from others’ drinking across the 23 countries are presented in Table 1. The mean rating ranged from less than four out of 10 for being called names or otherwise insulted because of a strangers’ drinking (mean = 3.48, SD = 1.81) to almost nine for physical harm to children because of someone else’s drinking (mean = 8.90, SD = 1.09). Items were classified according to their mean severity rating into categories based on thresholds derived from viewing and discussing the distributions of severity ratings among the research team. Of the 48 harm items considered, five were classified as extremely severe (mean severity rating ≥8), 17 as high severity (≥6 <8), 17 as moderate-high severity (≥5 <6), eight as moderate-low severity (≥4 <5), and one as low severity (≤4) (See Table 1). Across all harms, the mean severity was rated approximately six out of 10 (mean = 6.12).

Harms with mean ratings at the highest level of severity (≥8) included three harms to children (being physically hurt, witnessing violence, and child protection agency being called), forced sex by a known person, and being in a traffic accident because of a stranger’s drinking.

The second highest level of severity (≥7 <8) included mostly harms from a known drinker (e.g., being physically harmed, in a traffic accident, having to leave home, and feeling frighted or afraid) as well as being physically harmed by a stranger and being in an accident or close call at work.

The next highest level (≥6 <7) also involved mainly concrete harms from known drinkers, including several related to finances (financial trouble, going without food, property damaged, less money for household expenses, having money or valuables stolen) and being emotionally hurt or neglected. Also in this category were two child harms (left unsupervised and verbally abused) and one stranger harm – having house, car, or property damaged. No harms to children were rated less severe than six, on average.

Mid-level severity harms (≥5 <6) included a mix of harms from known drinkers, strangers, and co-workers. For known drinkers, items at this level mainly reflected relationship problems (harassed/bothered, serious argument, pushed/shoved, problems with friends/neighbours, stopped seeing someone) except for being a passenger with a drunk driver. For strangers, items included being made to feel unsafe or afraid, being in a serious argument, and being harassed or bothered. Average ratings for three of the five co-worker items were in this category and reflected reduced productivity (ability to do job negatively affected, having to work extra hours, and reduced productivity).

Five out of 12 of the stranger harms were classified in the lowest levels of harm (being pushed/shoved, belongings ruined, annoyed by someone vomiting or urinating, being kept awake, and being called names or insulted), reflecting annoyances and possibly one-time events. Three harms from known drinkers were at this lowest level: failed house duties, a social occasion negatively affecting the victim, and being called names or insulted. Having to cover for a co-worker was also a low-level harm.

In terms of the four groupings of harms by victim-perpetrator relationship, items reflecting harms to children from others’ drinking were rated the most severe (arithmetic mean across six items = 7.63), followed by the harms to someone from known people’s drinking (arithmetic mean across 25 items = 6.15), harms from co-workers’ drinking (arithmetic mean across five items = 5.69), and harms from strangers’ drinking (arithmetic mean across 12 items = 5.50). The list of harms to children from others’ drinking contained the highest percentage of harms rated as high severity or extreme severity (≥6 = 6/6 items or 100%; ≥8 = 3/6 or 50%) followed by the harms from known people’s drinking (≥6 = 12/25 or 48%; ≥8 = 1/25 or 4%), the harms from strangers’ drinking (≥6 = 3/12 or 25%; ≥8 = 1/12 or 8%), and the harms from co-workers’ drinking (≥6 = 1/5 or 20%; ≥8 = 0/5 or 0%). However, these comparisons are made among AHTO lists that differ in numbers and types of harms included. Therefore, we compared the severity ratings of a subset of 10 items that differ between lists only according to the victim-perpetrator relationship described (i.e., a specific type of harm is present in more than one of the four lists).

We compared the 10 items that were asked for both strangers and known drinkers and found that these items were rated 9% more severe if they were experienced because of a known person’s drinking (mean = 6.02) than if they were experienced because of a stranger’s drinking (mean = 5.53); although the difference was not statistically significant (t(df) = −2.276(18), p = 0.353).

Physical harm was the only item that was present in the list of harms from strangers’ drinking, the list of harms from known people’s drinking, and the list of harms to children from others’ drinking. Physical harm was, on average, rated significantly more severe if it was experienced by children because of others’ drinking (mean = 8.90) than if it was experienced by someone because of known people’s drinking (mean = 7.86; t(df) = −2.781(19), p = 0.012) or strangers’ drinking (mean = 7.59; t(df) = −5.086(20), p = 0.001).

Consistencies/inconsistencies in ratings of severity of AHTO items

There was fair to excellent absolute agreement and consistency of agreement among the countries’ (key informants’) ratings of the severity of harms from others’ drinking (single measures CA-ICC [95% CI] = 0.414 [0.320, 0.533], single measures AA-ICC [95% CI] = 0.325 [0.234, 0.445]; average CA-ICC [95% CI] = 0.940 [0.912, 0.962)], average AA-ICC [95% CI] = 0.914 [0.870, 0.946)]; Cicchetti (1994); Koo and Li (2016); Table 1). The level of agreement among key informants’ ratings of each of the sets of harms by victim-perpetrator relationship was poor-fair based on single-measures ICCs and excellent based on average ICCs: harms from known people’s drinking, harms from strangers’ drinking, harms from co-workers’ drinking, and harms to children from others’ drinking (single measures ICCs ranged 0.232-0.463; average ICCs > 0.85; Cicchetti (1994); Koo and Li (2016)). The ICC analysis showed the consistency of agreement among key informants’ ratings was slightly higher than the absolute agreement. Thus, while some key informants may rate most harms consistently higher than the mean, whereas some others may rate most harms consistently lower than the mean, there was some agreement on which harms were more versus less severe.

The standard deviation of severity ratings ranged from as low as 1.09 for physical harm to children from others’ drinking to 2.62 for being a passenger of drunk driver. The standard deviations indicate slightly greater agreement on the severity of more-severe harms.

Among the 13 countries with both a male and a female key informant, no significant difference was found between male and female key informants’ mean severity rating across all harm items (males = 6.31; females = 5.99; t(df) = 0.578(24), p = 0.569), stranger harm items (males = 5.53; females = 5.05; t(df) = 0.922(24), p = 0.366), known people harm items (males = 6.29; females = 6.16; t(df) = 0.194(24), p = 0.848), co-workers harm items (males = 5.98; females = 5.51; t(df) = 0.865(24), p = 0.396), or child harm items (males = 7.90; females = 7.46; t(df) = 0.752(24), p = 0.459).

Part 2: Feedback from informants about the AHTO survey items and the severity rating task

Lack of specificity of harm items

Numerous key informants expressed that because harm item descriptions lack specificity, some harm items may vary greatly in possible severity. This made it difficult to condense all possible variations of an item into a single severity rating for each harm. This was illustrated in the following quotes:

“I feel like the severity of negative effects can range greatly for each harm, so I tried best to ‘average out’ or guess the ‘typical’ severity of each harm”

“The level of subjectivity in interpreting the harms seems very high, even after accounting for the instructions. For example, I can only make an educated guess [about what] a typical car accident looks like (i.e., whether anyone is hurt, how much damage [is there] to the car) or what typical damage to a house, car, or property might look like or cost.”

“Some items contain multiple actions that could be interpreted inconsistently. … For example, there are (1) being harassed and (2) being bothered in [items] 1.7 and 2.16. Personally, [I think] being bothered is much less severe than being harassed.”

Characteristics of the harmful drinker and victim and their relationship

Numerous key informants identified the absence of victim and perpetrator gender and the nature of the victim-perpetrator relationship as important reasons for the lack of specificity of some AHTO items. For example, key informants wrote:

“There is no mention about [the] gender of [the] victim and perpetrator. To me, this is a huge gap. The difference in severity of many of these experiences depends on whether the victim is female and the perpetrator is male or the other way around.”

“The vicinity to the known person may also impact the severity. If it’s a close relative, like your parent, the impact may be more severe in comparison to an aunt you see once a year at the Christmas party.”

“I think there is a huge difference between if ‘someone the victim knows’ is a spouse/other family member or just an acquaintance (the later would appear less harmful).”

“[The severity of negative effect on the victim] depends on the age and abilities of the child.”

Perceptions of harm severity are likely to be subjective

Informants noted that severity ratings are likely to be subjective based both on the characteristics of the rater (e.g., being wealthy enough to have the resources to repair damage done by drinkers) and by the personal knowledge and experience of the rater.

“There is also a bias in who is completing the questionnaire. My perceptions of severity are unavoidably biased by my ability to absorb the costs of repairing damage to my property, by my living in a relatively peaceful part of town, and so on. While I can try to imagine how these harms may operate for others in different circumstances, I am only speculating, and it might be better to think about my own experiences if these things happened to me and assume differences of circumstance will average out across a representative sample.”

“I [am] afraid, that my scoring might be affected by long-lasting work in the field of alcohol research, and also work with persons treated for alcohol use disorders.”

“My ratings are based on both the results of the surveys that we have conducted and my own field experience in doing the research on the subject.”

“In general, I think all you can really do is give your own assessment of the seriousness of the harm (and a few of these clearly may differ by gender of the rater).”

“We have [a] strong tradition of child protecting services in [key respondent’s country], and of considering children as vulnerable and in need of being protected from the wrongdoings of adults. Hence, high scores on severity.”

Discussion

This study found that harms that result in physical, financial, practical, or severe emotional harm to the victim were generally rated among the most severe. Conversely, harms involving verbal insults or arguments and general annoyances and that were vaguely-described were rated among the least severe types of harm from others’ drinking. This division of more- vs. less-severely rated harms is similar to Callinan and Room’s (Callinan, 2014; Callinan & Room, 2014) division of a smaller list of harms from others’ drinking via multiple correspondence analysis. The more-highly rated group of items tend to describe harms that have concrete or tangible impacts (i.e., tangible). The group of less-highly rated items describe harms that have less-objective impacts (i.e., harms which are a matter of perception and often fear) or relate to disturbances of social order or pleasantness.

Researchers and analysts of existing and future data may draw on these ratings to construct summary measures of harm experienced by respondents from others’ drinking. For example, they might generate a summary score from respondents’ answers to questions about specific types of harms from others’ drinking by assigning a severity-specific weight or score to each item. Summary scores may be derived for sub-sets of harm items – for example, harm from known people’s drinking or harm from strangers’ drinking. These severity ratings may also inform categorization of harms items – for example, lists of harms that are rated as extreme, high, moderate, and low severity. In addition to informing analyses, the study’s results may assist researchers when interpreting and discussing findings from population surveys of AHTO.

Analyses in the existing AHTO literature have been conducted separately for men and women (e.g., Graham, Bernards, Munné, and Wilsnack (2008)), for harms from strangers and known people (e.g., Room et al. (2019)), for harms to children and adults (e.g., Laslett et al. (2017)), and according to closeness of the victim-perpetrator relationship (e.g., Stanesby et al. (2018)). The key informant severity ratings and qualitative feedback in the present study indicate that such divisions in analyses are warranted.

Alcohol-related harms were rated as more severe if the victims were children rather than adults. Harms to children may be considered more severe than harms to adults due to children’s relative inexperience and immaturity, which make them generally less physically, socially, and emotionally able to protect themselves from risks and recover from harm than adults (Berk, 2004; Hunter, 2014). Also, due to the developmental stage of children, physical abuse, emotional maltreatment, neglect, sexual abuse, and witnessing family violence in childhood may cause learning and development delays, disorganized attachment, impaired self-image, and neurological changes in the developing brain, resulting in a higher likelihood and severity of depression, hyper vigilance, poor self-esteem, self-harm, and fear/anxiety (Gardner, 2008; Hunter, 2014; Queensland Family and Child Commission, 2017; Wethington et al., 2008). Thus, these results reinforce the importance of measuring and addressing AHTO relating to children. Within the broad definition of children (typically younger than 18-21 years), negative effects may tend to be more severe for younger than older children due to differences in vulnerability and resilience (e.g., infants versus adolescents), although many environmental factors are also related (Berk, 2004; Hunter, 2014).

With regard to harms from strangers versus known drinkers, the present study suggests that the severity of negative effects arising from a single instance of a specific harm may be greater when it is experienced because of a known person’s drinking than if that same harm is experienced because of a stranger’s drinking. These findings are consistent with two empirical studies that have found that fewer people reported “a lot” of negative effects from the harms they experienced from strangers’ drinking compared to known people’s drinking (Callinan, 2014; Ramstedt, Sundin, Landberg, & Raninen, 2014). Taken together, the findings of these three studies indicate that the nature or closeness of relationship between victims and perpetrators is an important consideration in the severity of negative effects from occurrences of AHTO.

Although no significant differences were found in ratings of male and female participants, feedback from participants indicted that gender of the perpetrator and victim of HTO would be an important consideration in rating severity. This feedback is consistent with previous research in which the gender of the victim and perpetrator was found to be related to the type and severity of alcohol-related harms experienced (Berends, Ferris, & Laslett, 2012, 2014; Callinan et al., 2017; Crane, Godleski, Przybyla, Schlauch, & Testa, 2016; Graham & Wells, 2001; Laslett et al., 2011; Room et al., 2019; Stanesby et al., 2018). For example, men are more likely than women to experience aggression from other men who had been drinking in bars or public places, such as strangers or friends and acquaintances from their extended social relationships, whereas women are more likely to experience aggression from a male who is a spouse, partner, or friend (Graham & Wells, 2001). Further, women are most likely to report a close male as the most harmful drinker in their life, while men are most likely to report a male in their extended social circle as their most harmful drinker (Stanesby et al., 2018).

Harm item severity ratings also varied according to the relationship between the perpetrator and the victim, with harms from known drinkers rated as more severe than the same harm from strangers. Therefore, future measurement and analyses should also consider that the severity of negative effects may also vary according to the social context within which the harm is experienced. For example, severity of verbal abuse experienced in public settings (i.e., usually from a stranger) may be experienced as less severe than verbal abuse experienced in private settings (i.e., usually from a family member, friend, or spouse) (Karriker-Jaffe, Greenfield, & Kaplan, 2017) because verbal abuse from a known drinker has more emotional impact or because potential escalation of harm is greater in private than in public settings. Many AHTO items specify the social context (e.g., “private setting”, “social occasion”, “at work”). However, for items that do not, this may contribute to the lack of specificity expressed by the key respondents in the current study.

Finally, severity of harms and the associations of factors with severities of harm may vary among societies and by different cultural contexts (Room et al., 2019; Stanesby et al., 2018). There was moderate agreement for the severity ratings of AHTO among our sample of alcohol research and policy experts from a variety of societies. However, because only one to two informants participated from most countries, it was impossible to determine the extent that variation in the severity of harms from others’ drinking was due to individual or cultural differences.

Given that the experience and severity of harms from others’ drinking differ according to the victim-perpetrator relationship, the gender of the victim and perpetrator, and the social context within which the harms are experienced. Thus, AHTO survey items that aim to be highly specific would ideally incorporate all of these elements. An example of how such an item might be designed is provided in Figure 1. Given the breadth of AHTO items used in population surveys, the complexity of the example item in Figure 1, and the limited number of items that can be included in surveys due to response burden (Rolstad, Adler, & Rydén, 2011), researchers need to find the right balance between the breadth and specificity of AHTO items to include when designing surveys. It is perhaps unlikely that a single survey could incorporate the elements of Figure 1 for all possible types of AHTO. Therefore, there may not exist a one-size-fits-all design for AHTO survey items, and so survey design should be tailored to study goals.

Figure 1: Example of a highly-specific harm from others’ drinking survey question.

^a Information about the victim (including gender) would be ascertainable via other survey questions (e.g., demographics).

The design of AHTO items might be altered in other ways to suit study aims. Figure 2 demonstrates how AHTO items may be designed to enable calculation of the attributable fraction of others’ drinking to a range of harms from others. The structure of items in Figure 2 reflect the recommendation of Dawson and Room (2000) to ask parallel questions about the experience of problems with and without attribution to drinking. The alcohol-attributable fraction denotes the proportion of a health outcome which is caused by alcohol (Walter, 1976; WHO, 2016). By asking all respondents to report the frequency that they experienced a specific harm from anyone and directly attributed to someone else’s drinking, this approach allow estimation of the proportion of incidences of harms from others that is caused by others’ drinking. Alcohol-attributable fractions are typically calculated based on the amount and patterns of alcohol consumption in populations and the relative risk of alcohol consumption for a disease or injury (Taylor, Shield, & Rehm, 2011; WHO, 2016). This survey item design may compliment calculations of alcohol-attributable fractions based on population consumption and relative risks by further quantifying the burden of disease that is specifically due to others’ drinking in various societies.

Figure 2: Example of a harm from others’ drinking survey question that is designed to allow for calculation of the alcohol-attributable fraction.

Combining the item design in Figure 2 with the high-specificity design elements illustrated in Figure 1 would add another layer of depth to the data collected but would increase the time needed to respond to each item. Therefore, as noted, the choice to include or exclude various elements would need to be tailored to the goals of each study.

In future research, it may be possible to include the harm items in their “standard” form – for comparison with earlier surveys such as those conducted as part of the multinational GENAHTO study (Wilsnack et al., 2018) and the World Health Organization (WHO) and ThaiHealth Promotion Foundation collaborative research project (Callinan et al., 2016) – as well as more specific “improved” items based on findings from the current study and other methodological analyses. Another potentially viable strategy to refine AHTO survey items while retaining comparability is adding follow-on items clarifying the primary question.

Limitations

The key informants expressed that a lack of specificity in the descriptions of many of the harm items made it difficult to select a single severity rating for each harm item. The most apparent reasons for the inherent vagueness of some harm items were: (i) the specific situation or circumstances of the harm were not described with sufficient detail, (ii) the gender of both the victim and perpetrator was not specified, and (iii) the nature of the victim-perpetrator relationship was not described with sufficient detail. However, as one goal of this study was to better enable comparative analyses using existing AHTO data, it was necessary to include the AHTO items in the same format as they are presented to respondents of population surveys.

Our study utilized a convenience sampling method for selecting key informants. As a result, the country-specific ratings of this study are not generalizable to the opinion of the corresponding countries and we cannot infer between-country differences in the ratings of severities of AHTO from these results. This limitation was also raised in the key informant reflective comments. In consideration of this, our discussion focuses on the findings that are drawn from the sample as whole – treating it as one sample of alcohol research and policy experts from a range of countries. We focus on interpreting, discussing and drawing implications about comparisons between the “types” of harms from others’ drinking (and avoid speculating about cross-country differences).

The relatively small sample of 36 key informants (from 23 countries) may have limited the power to detect significant differences in severity ratings between (sets of) harm items. In addition, the observation of no difference between male and female mean severity ratings may have been due to the small sample size.

Key informants were sampled from a wide variety of countries, encompassing a mix of low-, middle- and high-income countries from Africa, Asia, Europe, North America, Oceania, and South America. However, an important caveat is that the key informants were all middle- to upper-class and well educated. As noted below, future research should assess perceptions of harm severity among more diverse and representative samples from multiple countries.

This study provides essential foundations for conducting a more quantitatively viable assessment of how best to measure the type, frequency and severity of HTO. Even though key informants were asked to rate the severity of harm items according to the general opinion of people from their country, differences in the personal circumstances, characteristics, knowledge, experiences, and values may have influenced key informants’ ratings. Within-country variation is not necessarily a problem. However, to adequately explore between-country differences in severity ratings requires drawing samples that are broadly representative of each of the countries – for instance, via financially and practically demanding methods such as random digit dialing or multi-stage stratified random sampling of regions, sub-areas, households, and persons (Webb & Bain, 2011). A follow-up to this study using nationally representative sampling would enable more comprehensive cross-country comparisons of the severities of AHTO.

Conclusions

When designing new AHTO surveys and conducting analyses of existing data, researchers should pay close attention to harms with high perceived severity to identify effective ways to prevent severe AHTO, knowledge which could be used to reduce the negative health and social impacts of alcohol-related harms to others. For this purpose, in-depth analyses of specific sub-sets of harms (e.g., Dale and Livingston (2010)) and qualitative interviews with victims of severe AHTO may prove useful. Particular attention is recommended to alcohol’s harms to children, and harms causing physical, financial, practical, or severe emotional impacts, because of their high perceived severity.