Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2021 Jun 10;16(6):e0252796. doi: 10.1371/journal.pone.0252796

Suicide on YouTube:Factors engaging viewers to a selection of suicide-themed videos

Eun Ji Jung 1,2, Seongcheol Kim 2,*
Editor: Vincenzo De Luca3
PMCID: PMC8191908  PMID: 34111162

Abstract

Visual social media platforms can function as both facilitators and intervenors of concerning behaviors. This study focused on one of the health concerns worldwide, a leading cause of death related to mental health—suicide—in the context of a dominant visual social media platform, YouTube. This study employed content analysis method to identify the factors predicting viewer responses to suicide-themed content from the perspectives of who’s, what’s, and how’s of suicide-themed videos. The results of the hierarchical multiple regression showed that the characteristics of content provider and content expression were more significant predictors of viewer engagement than were the characteristics of the message. These findings have implications for not only platform service providers but also diverse groups of individuals who participate in online discussions on suicide. YouTube has the potential to function as a locus for open discussion, education, collective coping, and even the diagnosis of suicidal ideation.

1. Introduction

According to the latest data on suicide by the World Health Organization [1], nearly 800,000 people die every year due to suicide, meaning one person dies every 90 seconds. Suicide can occur at any time in life and is the second leading cause of death among individuals aged 15–29 years.

The role of the Internet, particularly social networking services (SNSs), on suicide-related thoughts and behaviors has been a topic of growing interest and debate. There have been longstanding concerns over how social networking services manage content that may negatively affect the psychological well-being of its audience, especially the young users. This became an urgent issue following the death of a British girl, Molly Russel, whose father, Ian Russel, stated in an interview with BBC that Instagram encouraged his daughter to commit suicide [2]. In 2017, Molly Russel, who was known to have been posting and searching for keywords related to suicide and self-harm, such as “cutting,” “biting,” and “burning,” ended her own life. The posts that she “liked” were identified to be images that glorified suicide. This prompted the discussion on the need for an advanced platform policy to prevent such incidents from happening again.

Furthermore, there have been several incidents of self-expressive YouTubers ending their own lives. Jamey Rodemeyer, a 14-year-old YouTuber who actively expressed himself through videos on matters of his sexuality; homophobia; and lesbian, gay, bisexual, and transgender rights, ended his life on 18 September 2011. Although the school counselors had advised him not to use social media to talk about his sexuality, he voiced his thoughts through his YouTube posts. He appeared to be strong as he shared videos about the “It Gets Better” project, which aimed to address prevention of teen suicide. His suicide was attributed to excessive hostile comments. This case shows that social media sites are becoming venues to share personal opinions and to express oneself—even painful thoughts [3]—but at the same time, are making it easier for cyber-bullies to target their victims [4].

The influence of social media on concerning behaviors is not limited to children and teenagers alone. The debate lies in how media function—whether as a facilitator or as an intervenor of such behaviors. Considering the debate, this study aims to examine how deliverers of suicide-themed contents discuss suicide and to examine what factors, among content provider characteristics, story characteristics, and content expression characteristics, predict viewer engagement. The current study focused on one of the mainstream online video platforms, YouTube, as a site of analysis. It is not only a visual media platform but also a social networking service, which makes the investigation into the ongoing suicide-themed discussions on the platform worthwhile.

2. Literature review and research questions

Suicide and the self

Suicide is defined as a “conscious act of self-induced annihilation” [5: p. 203] in the current Western society. A review of comparable concepts suggests that society has historically condemned the act of killing oneself. Synonyms of suicide, such as self-killing, self-disembodiment, and self-murder, have shared stigmatizing connotations. This is because individuals are the constituents of society, where the sanity of one represents the degree of social health and well-being of the society as a whole. Suicidal thoughts and behaviors have also been considered pathological in the context of religion or morality. For example, the Protestants attribute melancholic self-disintegration to the temptation of Satan or a diabolical entity, which is distinguished from the inner self [6]. The concept evolved in the eighteenth century, encompassing terms from voluntary death to involuntary self-killing. Since the term “suicide” indicates a change in attitude, relatively decriminalizing the act and the individual [6], it is used in the following discussions.

As a multidimensional malaise [5], one suicidal event involves “biological, psychological, intrapsychic, logical, conscious, and unconscious, interpersonal, sociological, cultural, and philosophical or existential” elements [7: p. 221]. In the field of suicidology, the utility of suicide note has been acknowledged [8]. Suicide notes provide information closest to the suicidal mind, which comprises multidimensional thoughts of an individual.

Suicide pertains to not only the self but also society, as society is regarded as an aggregate of many selves. There have been longstanding concerns over the diffusive nature of suicide. The diffusion process involves successful or unsuccessful suicide attempts that lead to serious suicidal ideations among others, and some of those contemplators make successful or unsuccessful attempts [9,10]. The diffusion of suicide in relation to the influence of media was studied following the widespread imitation of Werther’s suicide, as described in the novel The Sorrows of the Young Werther by Johann Wolfgang von Goethe [10]. The matter lies in determining whether and how the media augment or intervene in the diffusion of suicidal thoughts and behaviors.

Influence of media on suicidal individuals

Studies of the potential influence of media-publicized suicide stories of actual suicide have yielded inconsistent findings [11]. The existence of both media contagion effect and intervention effect on suicide has been observed [1]. Media contagion refers to an adverse effect of media, whereas media intervention refers to a positive function of media.

Media contagion effect

The relationship between social media and socially concerning behaviors is complex. Social media can be hazardous to the vulnerable, as some online communities advocate extreme beliefs and behaviors, such as anorexia, suicide, and deliberate amputation, which are otherwise considered socially unacceptable [12]. Online discussion forums and social media chatrooms may facilitate socially undesirable behaviors as a result of peer pressure [13].

Recent studies that aimed to replicate and extend Phillips’ imitation theorem suggest that widely publicized suicide stories trigger copycat suicides [10,11]. News or television coverage of suicide stories may provide role models for individuals at risk, which is related to a social learning theory of deviant behavior [14]. From this perspective, publicized suicide stories may encourage suicides in the real world, which makes it imperative to develop guidelines on how to deliver suicide stories in order to promote safe media content.

Media intervention effect

While a large body of research supports the propagative effect of media on suicide, another vein of research suggests that media has preventive functions. The protective function of media is referred to as the Papageno effect [15]. It was named after a character in Mozart’s opera, The Magic Flute. Papageno becomes suicidal upon the loss of his beloved Papagena; however, he refrains from committing suicide thanks to a hopeful song by three elves. Media intervention effect suggests that media has a preventive function through education or collective coping with adverse situations.

The effectiveness of media on health-promoting activities was highlighted when articles that cited stories of individuals who refrained from executing their suicidal plans and of those who instead positively coped with adverse circumstances were published [15,16]. SNSs can help create social connections among individuals with shared experiences, raise awareness about prevention programs and crisis hotlines, and provide access to other available resources [12].

The advancement of media has enabled speedy diffusion of information without boundaries. The current study aimed to analyze suicide-themed content in a dominant visual media service platform, considering its reach and potential influence on the users.

Characteristics of YouTube as a dominant media platform

Social Networking Services (SNS) and YouTube

A large number of SNSs exists, each with different technological affordances. They provide an array of features including profile-generating, making friends, commenting, and private messaging. Although designed to be available to a wide range of audiences, much of the populations for each site are segmented upon homogenous interests and purposes [17,18].

YouTube is an example of a community website that reflects the evolution of the Web environment [19], which can be characterized as follows. Web 1.0 environment was based on one-way information consumption, whereas in Web 2.0 environment, individual users and the networks among them are given the power; users have richer and more complex experiences; content distribution is not limited to content creators alone; and the boundaries of the devices are blurred [20]. With higher bandwidth, faster and more interactive experiences have been realized, providing users with rich visual media content such as audio and video streaming. According to YouTube Press, over 1.9 billion logged-in users visit YouTube monthly, which account for nearly one-third of the Internet users worldwide. YouTube provides multi-lingual experiences with a total of 80 different languages, covering about 95% of the entire Internet population. As it features a variety of video contents, YouTube, as a media-sharing website that has become an SNS, is drawing the attention of everyone, regardless of age, gender, race or ethnicity, occupation, etc [17].

Technological advancement that has enabled SNSs to disseminate high volumes and a diverse range of information at a rapid rate across online networks has not always been discussed from a positive perspective. Continuous efforts have been made by multiple SNSs to develop an auto-filtering system to screen for and remove unsafe content. Following the incident of the live streaming of the New Zealand terror attack by Branton Tarrant, Facebook officially announced that they would adopt artificial intelligence (AI) technology to automatically filter harmful information. Twitter announced its plan to filter hate speech and spam tweets, while Tumbler, an image-based microblogging SNS, censors pornographic or illegal adult content. All these measures have been enforced, considering the influence of such mainstream platforms with a large user base.

YouTube as a visual media platform

Video-sharing websites have been gaining popularity on the Internet since the launch of YouTube in 2005. Compared to other media platforms, the most-frequently and widely-visited visual social media platform is YouTube, where the posts includes some form of visual information. Videos related to suicide or self-harm are concerning, because it is possible that such behaviors might become normalized, reinforced, or disinhibited [21,22] when the message is presented with visual effects. Extant studies show that the inclusion of visual material in a message facilitates longer memory retention [23], more accurate comprehension of the message [24], greater likelihood of reacting to a call to action presented in the message [25], and an increase in online engagement [26].

YouTube’s current policy on suicide and self-injury states, “Content that promotes self-harm or is intended to shock or disgust users is not allowed on YouTube. We do allow users to post content discussing their experiences with depression, self-harm, or other mental health issues” [27]. When users come across content where the deliverer “expresses suicidal thoughts or is engaging in self-harm,” they are advised to contact local authorities and press the flag button, which brings the post to YouTube’s immediate attention, according to Andrea Faville, a spokesperson for YouTube [28]. The stance of the platform is that “the users should not be afraid to speak openly about the topics of mental health or self-harm,” and the platform provides community guidelines, according to which content “promoting or glorifying suicide, providing instructions on how to self-harm, graphic images of self-harm posted to shock or disgust viewers” is banned [27]. The platform applies the same policies across all products and features, such as video posts, content descriptions, comments, and live streams. In instances of violation, the content is removed, and the creator is sent an email if it is their first time violating the policy. If it is not the first time, the creator is given a strike; three strikes result in channel termination.

These measures taken by platform service providers can be understood from the perspectives of both the Werther effect and Papageno effect. Studies of the relationship between the media coverage of suicide and suicidal behaviors in the real world have yielded inconsistent findings [11].

Characteristics of the deliverer

The purposes of watching health-related YouTube videos include social utility, convenient information-seeking, leisure, and entertainment [29]. With increasing popularity of health-related social media usage, it is important to pay attention to the characteristics of the deliverers of the content. Given that shared content on YouTube is a source of health information and reflects one’s experiences and emotions, the credibility and the diversity of the source are pivotal concerns.

Founded on the user-generated content (UGC) model, the contents on YouTube are created by its own users [30]. Content creators on this platform are commonly referred to as “YouTubers.” Those who satisfy the community’s needs through their content gain popularity and become so-called “influencers,” who are considered micro-celebrities. New media scholar David Marshall [31] identified a transition from “representational” to “presentational” media and culture. In the age of social media and self-created content, the public self, public-private self, and transgressive intimate self are presented. This develops into the establishment of trust, credibility, and a sense of closeness between the creators and audience. The boundary between legacy media and new media as sources of information has become less meaningful, as the consumers of media have begun to identify these influencers as new information providers.

Furthermore, social media could be used to intervene in suicidal ideation or suicide attempt by encouraging help-seeking behavior that relies on the user’s anonymity. Expression of thoughts and intentions about a concerning behavior is stigmatized. The rate of help-seeking behavior for mental issues like suicidality is low due to social stigma. Evidence suggests that “55% of people who complete suicide have no contact with a primary care provider in the month before suicide and 68% have no contact with mental health services in the year before suicide” ([32] as cited in [33]: p.525). SNSs can solve this issue by creating an anonymous online sphere where how people communicate and behave is less influenced by social desirability or social influence.

Many studies suggest that self-disclosure and honesty tend to increase online when participants’ identities are hidden. Joinson’s study [34] shows that the visual anonymity in computer-mediated communication (CMC) settings heightens the level of self-disclosure. Bargh, Mckenna, and Fitzsimons’s experiment [35] also revealed that the likelihood of one’s true self being activated is higher in Internet setting than it is in face-to-face setting due to the relative anonymity. Thus, individuals can present themselves in ways that might not be possible in face-to-face settings, thereby promoting help-seeking and collective coping behaviors.

The results of the preliminary coding analysis in this study revealed specific characteristics of content deliverers. There are multiple groups of content uploaders, also known as channel operators, who are distinct from message deliverers. The group of content uploaders are listed as clinic and health organizations, news agencies, one-person creators, production organizations, educational facilities, religious groups, and others, while the group of message deliverers are listed as survivors of suicide attempt; family members of the deceased; friends; news personnel; rescuers; third-party narrators; artists, musicians, and film personnel; lecturers and educators; medical professionals; one-person creators; and others. Some of the message deliverers provide real names, whereas the others are anonymous. Considering the aforementioned characteristics of the content deliverer, the following hypotheses are proposed.

H1. The characteristics of the deliverer, that is, the one who delivers the suicide-themed message would determine the degree of viewer engagement.

  • H1-1: The degree of engagement with the suicide-themed content would differ according to the content uploader.

  • H1-2: The degree of engagement with the suicide-themed content would differ according to the message deliverer.

  • H1-3: The degree of engagement with the suicide-themed content would differ according to the anonymity of the message deliverer.

Characteristics of the content story

Media contagion effect has been examined from multiple perspectives. A meta-analysis of suicide induced by media identified factors and conditions that maximize or minimize the copycat effect. These factors include the characteristics of the suicide story (i.e. celebrity or politician vs. non-celebrity, real vs. fictional, and completion vs. attempt), the amount of coverage, period effects (i.e. pre-television era vs. post), characteristics of the suicide rate, and media type (i.e. newspapers vs. television) [11]. Studies that are based on newspapers compared to television (TV, 82% less likely), studies that include suicide stories of political/entertainment celebrity (14.3 times), studies based on real suicides (4.03 times) as opposed to fictional suicides in films and soap operas, and studies based on suicide attempts as an outcome measure as opposed to completed suicide rates or counts are more apt to investigate copycat effects [11]. In addition, media influence on suicide has been studied in multiple country-settings. Regardless of the small effect size compared to other psychosocial risk factors for suicide, media contagion shows that not only audience characteristics but also media content involve risk [36,37].

Since the current study did not consider the difference in viewer engagement among different media or the period effect, only the characteristics of suicide story were included among multiple factors. The results of the preliminary coding analysis in this study categorized content into three story characteristics: (1) celebrity stories, politician stories, and non-celebrity stories; (2) real stories and fictional stories; and (3) suicide attempts, complete suicides, and suicide ideation (see Table 2). Considering the aforementioned characteristics of the suicide-themed content, the following hypotheses are proposed.

Table 2. Descriptive statistics for story characteristics (what).

View count Number of likes Number of comments
Category N Mean SD Mean SD Mean SD
Celebrity 7 3,386 6,554 81 168 9 9
YouTuber 2 13,780 15,617 405 514 46 38
Non-celebrity 71 3,561 5,904 78 249 18 79
Other 20 56,259 48,574 366 300 37 38
Real 61 2,743 5,036 69 265 18 85
Fictional 13 16,710 24,510 185 207 18 23
Other 26 20,297 32,368 281 342 43 37
Attempt 26 4,319 7,432 142 405 32 127
Complete 41 3,016 4,734 47 92 9 12
Ideation 40 2,802 5,189 57 128 8 17

Unit: One thousand. Numbers below one thousand are marked as “-”.

H2. The characteristics of the story characteristics, that is, the type of messages delivered, would determine the degree of viewer engagement.

  • H2-1: The degree of engagement with suicide-themed content would be higher for celebrity suicide stories than for non-celebrity suicide stories.

  • H2-2: The degree of engagement with suicide-themed content would be higher for real suicide stories than for fictional suicide stories.

  • H2-3: The degree of engagement with suicide-themed content would be higher for suicide attempt stories than for complete suicide stories and suicide ideation stories.

Characteristics of content expression

To promote safe media environment, the WHO and national agencies developed guidelines on reporting suicide [38], which includes 11 recommendations as follows: “take the opportunity to educate the public about suicide,” “avoid language which sensationalizes or normalizes suicide, or presents it as a solution to problems,” “avoid prominent placement and undue repetition of stories about suicide,” “avoid explicit description of the method used in a completed or attempted suicide,” “avoid providing detailed information about the site of a completed or attempted suicide,” “word headlines carefully,” “exercise caution in using photographs or video footage,” “take particular care in reporting celebrity suicides,” “show due consideration for people bereaved by suicide,” “provide information about where to seek help,” and “recognize that media professionals themselves may be affected by stories about suicide” [38: p. 7]

The rationale for the guidelines is that some reporting characteristics could either prevent or trigger suicides. The guidelines are commonly used as educational material for journalists and editors of traditional media agencies and were developed for traditional news reports of suicide rather than online news or social media posts. Thus, reporters are advised to refrain from using visual material. However, majority of new media content is visual-based, indicating the need to customize existing guidelines based on the new media message or channel features.

Reflecting the reporting guidelines for suicide stories, the current analysis categorized content expression characteristics into five groups: existence of advertisement, expression of suicide method, existence and placement of warning signs, existence and placement of hotlines, and the genre category. In this current study, the results of the preliminary coding procedure listed graphic, verbal, and textual expressions of suicide method. Warning signs and hotlines were listed in the description, the first-half of the video clip, and in the second-half of the video clip. YouTube platform provides a special warning function. Only those who have clicked on the “I understand and wish to proceed” option after being shown the YouTube community warning for inappropriate or offensive content warranting viewer discretion are allowed to view the content. The genre categories were listed as Entertainment, People & Blogs, News & Politics, Music, Film & Animation, Nonprofits & Activism, and Education (see Table 3). Considering the aforementioned characteristics of expressive methods, the following hypotheses are proposed.

Table 3. Descriptive statistics for content expression characteristics (how).

View count Number of likes Number of comments
Category N Mean SD Mean SD Mean SD
Advertisement O 22 20,267 29,992 182 184 16 21
Advertisement X 78 4,509 11,199 86 262 18 76
Graphic Expression 20 17,360 30,586 192 235 22 25
Verbal Expression 29 3,355 6,475 113 380 30 120
Textual Expression 5 6,007 8,207 68 92 18 19
None 57 6,810 14,379 72 121 9 148
Warning sign O 10 4,681 6,204 99 108 15 19
Warning sign in title 1 1,130 - 58 - 6 -
Warning sign in the description 2 5,307 5,906 199 199 29 32
Warning sign in the first-half of the video 12 3,981 5,854 82 105 13 18
YouTube Warning 5 6,311 10,506 175 332 28 34
Warning Sign X 85 8,508 19,554 104 258 17 73
Hotline O 22 4,379 7,620 146 445 45 145
Hotline in description 13 6,539 9,187 237 581 68 181
Hotline in the first half of the video 4 1,457 2,138 20 25 16 28
Hotline in second-half of the video 15 3,874 8,142 179 546 62 183
Hotline X 78 8,988 20,116 97 97 11 17
Entertainment 20 15,058 31,373 156 227 18 23
People & Blogs 10 2,411 1,744 54 44 8 7
News & Politics 24 1,875 2,805 14 17 5 7
Music 10 24,119 25,094 225 204 18 26
Science & Technology 1 1,220 - 17 - 0 -
Film & Animation 8 14,778 12,885 204 150 19 15
Gaming 1 3,891 - 56 - 5 -
Nonprofits &Activism 14 3,041 8,021 167 549 47 171
Education 12 1,424 1,877 28 38 10 18

Unit: One thousand. Numbers below one thousand are marked as “-”.

H3. The characteristics of content expression, that is, how the message is delivered, would determine the degree of viewer engagement.

  • H3-1: The degree of engagement with suicide-themed content would be higher for advertised videos than for non-advertised videos.

  • H3-2: The degree of engagement with suicide-themed content would be higher for graphic illustration of suicide method than for verbal or textual illustration of suicide method.

  • H3-3: The degree of engagement with suicide-themed content would differ according to the existence of a warning sign.

  • H3-4: The degree of engagement with suicide-themed content would differ according to the position of the warning sign.

  • H3-5: The degree of engagement with suicide-themed content would differ according to the existence of the hotline.

  • H3-6: The degree of engagement with suicide-themed content would differ according to the position of the hotline.

  • H3-7: The degree of engagement with suicide-themed content would differ according to the genre of the content.

3. Methods

The current study employed a quantitative content analysis method to identify the factors that draw viewers to suicide-themed videos on YouTube. Content analysis is a research method that examines the characteristics of the content, and it involves a systematic, objective, quantitative analysis of the message characteristics [39]. A thorough exploration of the content, including what the users are exposed to or what kind of messages they are currently acquiring online, from whom, and in what ways the message is received, is the most suitable method for investigation.

The preliminary analysis employed a bottom-up grounded theory approach [40], which is a qualitative method, to examine the characteristics of suicide-themed videos. The sample included a selection of 100 videos from the top to bottom in the order of exposure in the keyword search results with the term “suicide” in English on YouTube. The keyword search was done in Seoul, South Korea at one point in time, September 2019, by the researchers using Incognito Window on Google Chrome browser. The search result was ranked by the default method that YouTube provides which is ‘relevance.’ Neither specific inclusion nor exclusion criteria was set in the sample selection process with the purpose of extracting all possible codes relevant to suicide-themed videos. The sample video content was coded to observe the specific instances of the content delivers, the messages, and the expressions. The characteristics of content deliverer were identified by three factors: uploader category, message deliverer category, and anonymity. The characteristics of the stories included four factors: whether the story involves a public or non-public figure, whether the story is real or fictional, whether the story is about a suicide attempt, completed suicide, or suicide ideation, and the number of suicide stories. The characteristics of story expressions were observed using 7 factors: the existence of advertisement, illustration of method, existence of a warning sign, placement of the warning sign, existence of hotlines, placement of hotlines, and genre category. Every newly observed item for each factor was recorded. The list of the items for each factor was built upon after several iterative processes until saturation had been reached. The iterative process continued until only redundant instances were observed and until no new codes occurred to the degree in which the researchers have agreed that further data collection or data coding is counter-productive [41].

Categories were extracted based on the list of characteristics and were used as a foundation for the coding protocols for the quantitative content analysis. In the search results for the keyword “suicide,” an additional 100 YouTube videos were retrieved and analyzed. Unlike the search results from search engines, YouTube has no clear distinction in terms of pages. After several videos, a swipe up motion leads to the loading of more videos. Considering that the number of videos presented before the first swipe up motion was 20, it can be regarded that a page on YouTube contains 20 videos. A total of 589 videos were available for the search term ‘suicide’ after pages loaded until ‘No more results’ were left to show. Among 589 videos, 100 videos were chosen in the order of exposure after excluding search results for superhero movie based on DC Comics ‘Suicide Squad.’ Those videos were discernible through the video title and thumbnail, as the major actors and characters were visible in the thumbnail area. The researchers have eliminated 7 Suicide Squad videos because of the low relevance to the health-related issue of suicide, and added 7 other videos to make 100. Videos categorized as ‘music’ or ‘film’ were not related to ‘Suicide Squad’ but they were videos created by individuals who express suicide-related information through the form of music or film.

The sample size (n = 100) was chosen because the purpose of the research was to reflect a basic query of what general people would likely to be exposed to with the keyword search. The first several pages of search result presented to the person searching the keyword engage most viewers whereas the following pages are less attended [42,43]. Multiple health information-related studies included the first several pages of search results in the sample, implying that people tend to select the information they are provided with first rather than the information they are provided later [4449]. The first two to three pages were the most commonly observed number for YouTube content analysis on health-related matters.

However, a power analysis was performed as Niederkrotenthaler, Schacherl, and Till [50] did to identify the minimum number of samples required. The desired sample size was computed with the software G*Power 3.1 [51]. In order to identify a medium-sized difference effect size (f2) = 0.15; with an Alpha-level of 0.05 and Power (1-β error prob) = 0.80, with the number of predictors n = 3 (who, what, how models), and the total number of predictors 44 (1 continuous variable and 43 dummy variables), a total of 82 videos were required as a minimum. Since each page holds 20 videos, this study required more than four pages to meet the minimum number of samples. Thus, five pages were included in the final sample, in other words, 100 videos.

A comprehensive content analysis was conducted which investigated the following: (1) who uploaded and delivered the suicide-themed videos, (2) what kind of messages were delivered and (3) how the messages were delivered. The data extracted for each video were as follows: (1) video identification information, which included the title, description, and upload date; (2) characteristics of content deliverer, which included the uploader category, message deliverer category, and anonymity; (3) characteristics of the stories, including whether the story involves a public or non-public figure, whether the story is real or fictional, whether the story is about a suicide attempt, completed suicide, or suicide ideation, and the number of suicide stories; and (4) characteristics of story expressions, which included the existence of advertisement, illustration of method, existence of a warning sign, placement of the warning sign, existence of hotlines, placement of hotlines, and genre category. A total of 14 variables were examined. The characteristics of content deliverer, stories, and expressions were dummy coded.

Furthermore, YouTube’s user engagement metrics, including the view count, the number of likes, and the number of comments of the selected 100 videos, were retrieved using YouTube Statistics, which is a free application that tracks the statistics for YouTube videos [52]. It is assumed that the degree of viewer engagement increases in the following order: view count, number of likes, and number of comments. Pressing the like button requires more engagement than merely watching the video, whereas active expression of an opinion through a comment requires additional time and effort. With dummy coding, statistical analysis using hierarchical multiple regression was conducted to observe the linear relationships between the categorical factors of suicide-themed video contents and the amount of attention or popularity.

Hierarchical multiple regression is a method that considers the relative effect of more than one explanatory variable on the dependent variable of interest. It enables the researchers to build several models to compare the proportion of explained variance in the dependent variable by sequentially adding models. The newly added models always include the previous models. The analysis can determine which model better explains and predicts the dependent variable in a statistically meaningful way. The current study takes three models, sometimes referred as blocks: who, what, and how variables of the suicide-themed content in explaining viewer engagement. The analysis was completed on IBM SPSS (Statistical Package for the Social Sciences) Statistics software [53].

4. Results

This study hypothesized that deliverer characteristics, story characteristics, and content expression characteristics would predict viewers’ attention or engagement with suicide-themed videos. Among 14 variables, 5 variables including message deliverer category, whether the story is about a suicide attempt, completed suicide, or suicide ideation, illustration of method, placement of the warning sign, and placement of hotlines were multi-coded. Thus, the number of instances coded in each category may not be equal to the total number of observed instances which is 100. The descriptive analyses of the main factors are presented in Tables 13. The results showed that the regression model with three levels (deliverer characteristics, story characteristics, and content expression characteristics) had different explanatory powers according to the degree of engagement, each measured by the number of views, likes, and comments.

Table 1. Descriptive statistics for content deliverer characteristics (who).

View count Number of likes Number of comments
Category N Mean SD Mean SD Mean SD
Content Uploader Clinic and Health Organization 5 542 559 5 7 1 1
News Agency 24 1,990 2,791 15 17 5 7
One Person Creator 6 6,252 11,848 369 834 111 260
Production Organization 24 22,464 32,103 187 209 17 24
Educational Facilities 13 988 1,798 25 46 2 5
Religious Group 2 1,185 1,519 6 6 1 -
Others 26 5,845 6,488 124,138 162 16 18
Message Deliverer Survivors 11 4,398 8,844 239 610 64 191
Family 16 845 1,177 20 28 3 6
Friends 7 2,571 3,969 46 59 9 11
News personnel 16 2,342 3,266 16 19 6 8
Rescuer 2 3,534 1,167 83 12 9 1
Narrator 10 2,241 1,720 40 37 15 19
Artist/Musician/Film personnel 24 24,034 31,266 218 206 19 23
Lecturer/Educator 8 330 346 5 5 - -
Medical personnel 8 386 449 4 4 1 -
One-person creator 7 10,034 12,457 445 766 108 237
Others 15 2,634 5,057 30 56 8 13
Anonymity Anonymous 41 12,791 20,216 151 196 17 22
Real name provided 59 4,574 15,939 75 276 18 87

Unit: One thousand. Numbers below one thousand are marked as “-”.

The results of the hierarchical multiple regression are illustrated in Table 4. Deliverer characteristics were the only predictor that was significant across all three viewer responses (p = .036, adjusted R2 = .132 for the view count; p = .011, adjusted R2 = .175 for the number of likes; and p = .048, adjusted R2 = .129 for the number of comments). Hypothesis 1 was partially supported, hypothesis 2 was not supported, and hypothesis 3 was partially supported. In particular, the regression analysis showed that survivors of suicide attempt (β = .338, t = 2.818, p = .006 for the number of likes and β = .443, t = 3.303, p = .001 for the number of comments), artists/musicians/film personnel (β = .575, t = 2.653, p = .010 for the number of likes and β = .538, t = 2.098, p = .039 for the number of comments), and one-person creators (β = .423, t = 3.218, p = .002 for the number of likes and β = .414, t = 2.813, p = .006 for the number of comments) were significant predictors.

Table 4. Results for the hierarchical multiple regression analysis for popularity.

Engagement (Standardized Coefficients beta)
View count Number of Likes Number of Comments
Factors (Characteristics) Model 1 Model 2 Model 3 Model 1 Model 2 Model 3 Model 1 Model 2 Model 3
Content Deliverer¶ (Who) Clinic and health organization -.104 -.111 -0.186 0.029 0.002 -0.096 0.073 0.062 -0.048
News agency -.218 -.128 0.066 0.068 0.126 0.401 0.122 0.192 0.465
One-person creator -.152 -.023 0.024 0.155 0.299 0.128 0.298 0.454 0.092
Educational facilities -.186 -.041 -0.238 0.034 0.121 -0.778 0.073 0.159 -1.987***
Religious groups -.060 -.011 -0.154 0.021 0.067 -0.321 0.015 0.074 -0.949***
Others -.166 -.095 -0.135 0.125 0.195 0.148 0.134 0.266 -0.049
Survivors -.046 -.045 0.1 0.338** 0.293 0.311 0.443*** 0.418 -0.042
Family -.113 -.102 -0.102 0.049 0.032 -0.056 0.127 0.111 -0.173
Friends .012 .021 0.069 0.021 0.042 0.055 -0.006 -0.009 0.034
News personnel -.025 -.014 0.004 0.119 0.123 0.045 0.237 0.286 -0.177
Rescuer -.027 -.019 0.077 0.09 0.059 0.31 0.122 0.087 0.389**
Narrator .006 -.021 -0.035 0.125 0.061 -0.21 0.291 0.359 -0.05
Artist/Musician/Film personnel .376 .286 0.183 0.575** 0.541 0.271 0.538* 0.742 0.121
Lecturer/Educator -.073 -.098 -0.017 0.1 0.053 0.08 0.178 0.171 -0.235
Medical personnel -.09 -.052 -0.006 0.012 0.078 0.102 0.035 0.167 0.028
One-person creator .107 .024 -0.147 0.423** 0.31 0.193 0.414** 0.395 0.136
Others -.011 .042 0.039 0.073 0.09 0.196 0.14 0.153 0.036
Anonymity -.190 -.171 -0.205 -0.009 -0.062 -0.075 -0.023 -0.023 0.038
Content Story¶ (What) Number of stories -.049 -0.077 -0.072 -0.007 -0.031 0.06
Celebrity -.014 -0.03 -0.099 -0.076 -0.236 -0.003
YouTuber .114 0.027 0.175 0.164 0.028 0.031
Other .215 0.244 0.031 0.05 0.045 0.144
Fictional -.029 -.059 0.033 0.137 -0.132 -0.029
Attempt -.032 0.095 0.069 -0.129 0.11 0.166
Complete -.099 0.092 -0.043 -0.026 0.039 0.136
Ideation -.103 -0.128 -0.078 -0.091 -0.05 0.108
Content Express-ion¶ (How) Advertise-ment -0.003 0.024 -0.052
Graphic expression of method 0.078 0.025 0.144
Verbal expression of method -0.038 -0.116 0.261
Textual expression of method -0.229 -0.122 -0.23*
Warning sign Existence 0.373 0.1 0.2
in Title 0.097 -0.03 -0.167
in description 0.192 0.117 0.232
in the first-half of the video 0.189 -0.044 -0.025
Hotline Existence -0.048 0.168
in the description 0.078 0.23 0.018
in the first-half of the video -0.038 -0.114 -0.017
in the second-half of the video -0.229 0.061 0.05
Entertainment 0.373 0.339 0.251
People & Blogs 0.097 0.183 0.425*
Music 0.192 0.297 0.167
Film & Animation 0.189 0.223 0.065
Gaming 0.069 0.001 -0.017
Nonprofits &Activism 0.386 1.298 2.51**
Education 0.234 0.43 0.168
R Square .292 .341 .455 .327 .364 .595 .300 .344 .782
Adjusted R Square .132 .090 -.048 .175 .122 .222 .129 .072 .554
F 1.827* 1.358 .905 2.155* 1.503 1.595 1.759* 1.264 3.427**
Durbin-Watson 2.016 2.006 1.849

* p < .05

** p < .01

*** p < .001.

Although the final model was not a significant predictor of view count and number of likes, it was a significant predictor of the number of comments, which indicates the highest level of viewer engagement (p < .001, adjusted R2 = .554). The analysis of the final model showed that educational facilities (β = - 1.987, t = -5.995, p < .001) and religious groups (β = -.949, t = -4.612, p < .001) as the video uploader and rescuer (β = .389, t = 2.911, p = .006) as the deliverer were significant predictors of the number of comments, whereas other deliverer-related variables were not. Hypotheses 1–1 and 1–2 were supported. However, anonymity was not a significant determinant of viewer response, thus rejecting hypothesis 1–3.

None of the story-related variables was significant, rejecting hypotheses 2–1, 2–2, and 2–3. On the other hand, three content expression-related variables were significant predictors of comments: textual expression of suicide method (β = -.230, t = -2.034, p = .048), People and Blogs as genre (β = .425, t = 2.183, p = .034), and Nonprofits & Activism as genre (β = 2.510, t = 7.645, p < .001), supporting hypotheses 3–2 and 3–7. The existence and placement of advertisements, warning signs, and hotlines did not have a significant influence on viewer response, rejecting hypotheses 3–1, 3–3, 3–4, 3–5, and 3–6. Thus, who delivers the suicide-themed-message and how the message is delivered are more significant predictors than what is discussed in terms of viewer engagement.

5. Discussion and conclusion

The findings showed that videos uploaded by educational facilities and religious groups, and videos textually expressing suicide methods had relatively fewer comments. On the contrary, videos categorized as People & Blogs and Non-profits & Activism, and content delivered by rescuers of suicide had relatively more comments. Content delivered by survivors of suicide attempt, artists, musicians, film personnel, and one-person creators also received more likes and comments.

These findings imply that viewers are more engaged with the content when the deliverers have close experience of suicide. Sharing suicide stories through art, music, and film such as in vlog format is involving, whereas lecture-based or preaching approaches are less involving. Traditionally, suicide has been discussed by suicide prevention organizations or medical professionals via news channels. Suicide stories have been publicized through news portals, and the influence of publicized suicide stories has been studied. Although it is difficult to deny the influence of informative and educational news content, this study shows that suicide conversations are carried out in online spheres, where rescuers and survivors of suicide attempt actively participate in the discussion. Suicide prevention organizations and educational facilities need to strategically engage viewers.

This study has implications for not only health and medical professionals but also platform service providers. As the deliverers of suicide-themed posts are survivors and rescuers rather than health professionals, the platform may play an important role as an arena for diagnosis. The symptoms and the reasons for suicide ideation may be explicitly stated on the platform, which may help health professionals to diagnose individuals who ideate suicide or those with suicide experiences. The extant studies suggest that individuals can positively cope with suicidal thoughts when they openly talk about their state of mind. Much of the video content analyzed in this study addressed the need for a platform to discuss suicide and to share personal feelings without judgement and stigma. As these dominant media platforms are becoming the locus for open discussion, education, and collective coping, platform service providers are recommended to continue facilitating the discussion by engaging more people.

The accountability of participants in suicide-themed online conversations should be equally emphasized as much as the accountability of platforms. The creators of suicide-themed videos and the viewers should take advantage of the platform, but with discretion. Uploaders should acknowledge the societal influence of their posts, and the viewers should actively alert the authorities of any harmful or triggering content.

Most importantly, this study also has implications for policymakers in terms of addressing the need for developing a proper guideline for suicide-themed new media content. The current guidelines include news reporting guidelines, which advise reporters to refrain from using visual material. However, a majority of new media content includes visual expressions. The current analysis showed that over 50% of suicide-themed content on YouTube involves graphic, verbal, or textual illustrations of suicide methods, while the majority did not provide any warning sign or crisis hotlines. Only 5% of the observed content required age registration by the platform, which has been highlighted as a problem [50]. As young adults tend to obtain information and resources through online channels, new media platforms might be the first or most-frequently visited sources of information. Therefore, there is a need to revise the existing guidelines to fit new media features.

This study is limited in that only content-related predictors were included in the analysis. Platform affordances or external factors were not taken into consideration. Moreover, alternative measurement of viewer engagement should be considered, such as the positive comment to negative comment ratio or the net comment calculated by the number of positive comments subtracted by the number of negative comments. A more appropriate measure of viewer engagement other than the number of views, likes, and comments will provide more fruitful implications as positive engagement on sensitive topics like suicide enables collective coping. In addition, the video samples analyzed in this study were search results, which had already been filtered by the platform. This indicates that extremely triggering or harmful content had already been removed from the website, which were, therefore, not included in the analysis. However, it can be concluded that the sample did consist of videos maintained available on the platform that an ordinary user would find using the same keyword. Lastly, the applicability of research results can be another limitation since selecting a certain number of videos at a specific time in a specific location with specific language may not incorporate all instances of suicide-themed videos on YouTube. Nonetheless, the selection of top several pages in the order of relevance was the best alternative as the formula of search result presentation on YouTube is unknown like a black box. Follow up studies pertaining to multiple location and language settings can be helpful.

In order to determine whether dominant visual media platforms facilitate the diffusion of suicidal thoughts, future studies are needed to identify the factors of dominant visual media platform that augment or spread suicidal thoughts. As society continues to undergo digital transformation, daily-visited new media sites, such as Twitter, Facebook, and YouTube, should not facilitate suicide, but help to mitigate suicidal thoughts. The findings of this germinal explorative study could help establish a cornerstone for a safe online community and a constructive communication ground, by examining societal issues and highlighting the responsibilities of dominant visual new media platforms.

Supporting information

S1 Codebook

(DOCX)

S1 File

(XLSX)

S2 File

(DOCX)

Data Availability

All relevant data are within the manuscript and we provide our codebook.

Funding Statement

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2019S1A3A2099973) and the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program(IITP-2020-0-01749) supervised by the IITP(Institute of Information & Communications Technology Planning & Evaluation).

References

  • 1.World Health Organization, International Association of Suicide Prevention. Preventing suicide: A resource for media professionals, 2017 update. World Health Organization. 2017. Available from: https://apps.who.int/iris/handle/10665/258814. [Google Scholar]
  • 2.Crawford A. Instagram ’helped kill my daughter.’ BBC News. 2019 Jan 22 [Cited 2019 May 24]. Available from: https://www.bbc.com/news/av/uk-46966009/instagram-helped-kill-my-daughter.
  • 3.Dagar A, Falcone T. High viewership of videos about teenage suicide on YouTube. Journal of the American Academy of Child & Adolescent Psychiatry. 2020; 59(1): 1–3. doi: 10.1016/j.jaac.2019.10.012 [DOI] [PubMed] [Google Scholar]
  • 4.James SD. Jamey Rodemeyer suicide: Police consider criminal bullying charges. ABC NEWS. 2011 Sep 22 [Cited 2019 June 1]. Available from: https://abcnews.go.com/Health/jamey-rodemeyer-suicide-ny-police-open-criminal-investigation/story?id=14580832.
  • 5.Shneidman E. Definition of suicide. Northvale, NJ: Jason Aronson; 1985. [Google Scholar]
  • 6.Bähr A. Between “self-murder” and “suicide”: The modern etymology of self-killing. Journal of Social History. 2013; 46(3): 620–632. doi: 10.1093/jsh/shs119 [DOI] [Google Scholar]
  • 7.Leenaars AA. Suicide: A multidimensional malaise. Suicide and Life-Threatening Behavior. 1996; 26(3): 221–235. doi: 10.1111/j.1943-278X.1996.tb00608.x [DOI] [PubMed] [Google Scholar]
  • 8.O’Connor RC, Leenaars AA. A thematic comparison of suicide notes drawn from Northern Ireland and the United States. Current Psychology. 2004; 22(4): 339–347. doi: 10.1007/s12144-004-1039-5 [DOI] [Google Scholar]
  • 9.Baller RD, Richardson KK. Social integration, imitation, and the geographic patterning of suicide. American Sociological Review. 2002; 67(6): 873–888. doi: 10.2307/3088974 [DOI] [Google Scholar]
  • 10.Phillips DP. The influence of suggestion on suicide: Substantive and theoretical implications of the Werther effect. American Sociological Review. 1974; 39(3): 340–354. doi: 10.2307/2094294 [DOI] [PubMed] [Google Scholar]
  • 11.Stack S. Media impacts on suicide: A quantitative review of 293 findings. Social Science Quarterly. 2000; 81(4): 957–971. [Google Scholar]
  • 12.Luxton DD, June JD, Fairall JM. Social media and suicide: A public health perspective. American Journal of Public Health. 2012; 102(S2): S195–S200. doi: 10.2105/AJPH.2011.300608 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Dunlop SM, More E, Romer D. Where do youth learn about suicides on the Internet, and what influence does this have on suicidal ideation? Journal of Child Psychology and Psychiatry. 2011; 52(10): 1073–1080. doi: 10.1111/j.1469-7610.2011.02416.x [DOI] [PubMed] [Google Scholar]
  • 14.Niederkrotenthaler T, Fu KW, Yip PS, Fong DY, Stack S, Cheng Q, et al. Changes in suicide rates following media reports on celebrity suicide: A meta-analysis. Journal of Epidemiology and Community Health. 2012; 66(11): 1037–1042. doi: 10.1136/jech-2011-200707 [DOI] [PubMed] [Google Scholar]
  • 15.Niederkrotenthaler T, Voracek M, Herberth A, Till B, Strauss M, Etzersdorfer E, et al. Role of media reports in completed and prevented suicide: Werther v. Papageno effects. The British Journal of Psychiatry. 2010; 197(3): 234–243. doi: 10.1192/bjp.bp.109.074633 [DOI] [PubMed] [Google Scholar]
  • 16.Hawton K, Williams K. Influences of the media on suicide: Researchers, policy makers, and media personnel need to collaborate on guidelines. The BMJ. 2002; 325(7377): 1374–1375. doi: 10.1136/bmj.325.7377.1374 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Boyd DM, Ellison NB. Social network sites: Definition, history, and scholarship. Journal of Computer‐Mediated Communication. 2007; 13(1): 210–230. doi: 10.1111/j.1083-6101.2007.00393.x [DOI] [Google Scholar]
  • 18.Kim YJ. New media, new communication. Seoul, South Korea: Ewha Press; 2014. [Google Scholar]
  • 19.Burgess J, Green J. YouTube: Online video and participatory culture. Cambridge, UK: John Wiley & Sons; 2018. [Google Scholar]
  • 20.Funk T. Web 2.0 and beyond: Understanding the new online business models, trends, and technologies. Santa Barbara, CA: ABC-CLIO; 2008. [Google Scholar]
  • 21.Carlyle KE, Guidry JP, Williams K, Tabaac A, Perrin PB. Suicide conversations on Instagram: Contagion or caring? Journal of Communication in Healthcare. 2018; 11(1): 12–18. doi: 10.1080/17538068.2018.1436500 [DOI] [Google Scholar]
  • 22.Lewis SP, Heath NL, St Denis JM, Noble R. The scope of non-suicidal self-injury on YouTube. Pediatrics-English Edition. 2011; 127(3): 552–557. doi: 10.1542/peds.2010-2317 [DOI] [PubMed] [Google Scholar]
  • 23.Whitehouse AJ, Maybery MT, Durkin K. The development of the picture‐Superiority effect. British Journal of Developmental Psychology. 2006; 24(4): 767–773. doi: 10.1348/026151005X74153 [DOI] [Google Scholar]
  • 24.Houts PS, Doak CC, Doak LG, Loscalzo MJ. The role of pictures in improving health communication: a review of research on attention, comprehension, recall, and adherence. Patient Education and Counseling. 2006; 61(2): 173–190. doi: 10.1016/j.pec.2005.05.004 [DOI] [PubMed] [Google Scholar]
  • 25.Lipkus IM, Hollands JG. The visual communication of risk. Journal of the National Cancer Institute Monographs. 1999; (25): 149–163. doi: 10.1093/oxfordjournals.jncimonographs.a024191 [DOI] [PubMed] [Google Scholar]
  • 26.Henneman A, Franzen-Castle L, Kinsey J. Picture this! Increasing online engagement by using photos and videos. Journal of Nutrition Education and Behavior. 2013; 45(4): S1–S2. doi: 10.1016/j.jneb.2013.04.007 [DOI] [Google Scholar]
  • 27.YouTube Help. Suicide and self injury. 2019 [Cited 2019 June 24]. Available from: https://support.google.com/youtube/answer/2802245?hl=en.
  • 28.Bever L. A pediatrician exposes suicide tips for children hidden in videos on YouTube and YouTube Kids. Washington Post. 2019 Feb 25 [Cited 2019 May 24]. Available from: https://www.washingtonpost.com/technology/2019/02/24/pediatrician-exposes-suicide-tips-children-hidden-videos-youtube-youtube-kids/.
  • 29.Park DY, Goering EM. The health-related uses and gratifications of YouTube: Motive, cognitive involvement, online activity, and sense of empowerment. Journal of Consumer Health on the Internet. 2016; 20(1–2): 52–70. doi: 10.1080/15398285.2016.1167580 [DOI] [Google Scholar]
  • 30.Welbourne DJ, Grant WJ. Science communication on YouTube: Factors that affect channel and video popularity. Public Understanding of Science. 2016; 25(6): 706–718. doi: 10.1177/0963662515572068 [DOI] [PubMed] [Google Scholar]
  • 31.Marshall PD. The promotion and presentation of the self: Celebrity as marker of presentational media. Celebrity Studies. 2010; 1(1): 35–48. doi: 10.1080/19392390903519057 [DOI] [Google Scholar]
  • 32.Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: a review of the evidence. American Journal of Psychiatry. 2002; 159(6): 909–916. doi: 10.1176/appi.ajp.159.6.909 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Calear AL, Batterham PJ, Christensen H. Predictors of help-seeking for suicidal ideation in the community: Risks and opportunities for public suicide prevention campaigns. Psychiatry Research. 2014; 219(3): 525–530. doi: 10.1016/j.psychres.2014.06.027 [DOI] [PubMed] [Google Scholar]
  • 34.Joinson AN. Self‐disclosure in computer‐mediated communication: The role of self‐awareness and visual anonymity. European Journal of Social Psychology. 2011; 31(2): 177–192. doi: 10.1002/ejsp.36 [DOI] [Google Scholar]
  • 35.Bargh JA, McKenna KY, Fitzsimons GM. Can you see the real me? Activation and expression of the “true self” on the Internet. Journal of Social Issues. 2002; 58(1): 33–48. doi: 10.1111/1540-4560.00247 [DOI] [Google Scholar]
  • 36.Pirkis J, Blood RW, Beautrais A, Burgess, Skehan J. Media guidelines on the reporting of suicide. Crisis. 2006; 27(2): 82–87. doi: 10.1027/0227-5910.27.2.82 [DOI] [PubMed] [Google Scholar]
  • 37.Stack S. Suicide in the media: A quantitative review of studies based on nonfictional stories. Suicide and Life-Threatening Behavior. 2005; 35(2): 121–133. doi: 10.1521/suli.35.2.121.62877 [DOI] [PubMed] [Google Scholar]
  • 38.World Health Organization. Preventing suicide: A resource for media professionals. World Health Organization. 2008. Available from: http://apps.who.int/iris/handle/10665/43954.
  • 39.Neuendorf KA. The content analysis guidebook. Thousand Oaks, CA: Sage Publications; 2002. [Google Scholar]
  • 40.Glaser BG, Strauss AL. Discovery of grounded theory: Strategies for qualitative research. New York: Routledge; 2017. [Google Scholar]
  • 41.Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Quality & Quantity. 2018; 52(4): 1893–1907. doi: 10.1007/s11135-017-0574-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.iProspect. iProspect Search Engine User Behavior Study. 2006. Available from: http://district4.extension.ifas.ufl.edu/Tech/TechPubs/WhitePaper_2006_SearchEngineUserBehavior.pdf.
  • 43.Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002; 324: 573–577. doi: 10.1136/bmj.324.7337.573 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Kelly-Hedrick M, Grunberg PH, Brochu F, Zelkowitz P. “It’s totally okay to be sad, but never lose hope”: content analysis of infertility-related videos on YouTube in relation to viewer preferences. Journal of medical Internet research. 2018; 20(5): e10199. doi: 10.2196/10199 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Stellefson M, Chaney B, Ochipa K, Chaney D, Haider Z, Hanik B, et al. YouTube as a source of chronic obstructive pulmonary disease patient education: A social media content analysis. Chronic respiratory disease. 2014; 11(2): 61–71. doi: 10.1177/1479972314525058 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Wasserman M, Baxter NN, Rosen B, Burnstein M, Halverson AL. Systematic review of internet patient information on colorectal cancer surgery. Dis Colon Rectum. 2014; 57(1): 64–69. doi: 10.1097/DCR.0000000000000011 [DOI] [PubMed] [Google Scholar]
  • 47.Sahin AN, Sahin AS, Schwenter F, Sebajang H. YouTube videos as a source of information on colorectal cancer: what do our patients learn?. Journal of Cancer Education. 2019; 34(6): 1160–1166. doi: 10.1007/s13187-018-1422-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Leong AY, Sanghera R, Jhajj J, Desai N, Jammu BS, Makowsky MJ. Is YouTube useful as a source of health information for adults with type 2 diabetes? A South Asian perspective. Canadian journal of diabetes. 2018; 42(4): 395–403. doi: 10.1016/j.jcjd.2017.10.056 [DOI] [PubMed] [Google Scholar]
  • 49.Bae SS, Baxter S. YouTube videos in the English language as a patient education resource for cataract surgery. International ophthalmology. 2018; 38(5): 1941–1945. doi: 10.1007/s10792-017-0681-5 [DOI] [PubMed] [Google Scholar]
  • 50.Niederkrotenthaler T, Schacherl R, Till B. Communication about suicide in YouTube videos: Content analysis of German-language videos retrieved with method-and help-related search terms. Psychiatry research. 2020; 290: 113170. doi: 10.1016/j.psychres.2020.113170 [DOI] [PubMed] [Google Scholar]
  • 51.Faul F, Erdfelder E, Buchner A, Lang AG. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analysis. Behavior Research Methods. 2009; 41(4): 1149–1160. doi: 10.3758/BRM.41.4.1149 [DOI] [PubMed] [Google Scholar]
  • 52.Zdravković N. YouTube statistics downloader (Version 2.5.0). 2013 [cited 17 July 2019]. In: YouTube Statistics [Internet]. Available from: http://yts.sourceforge.net/download.html.
  • 53.SAGE Publications. How-to Guide for IBM SPSS Statistics Software. 2019. Available from: https://methods.sagepub.com/base/download/DatasetHowToGuide/hierchical-linear-regression-prison-inmates.

Decision Letter 0

Vincenzo De Luca

23 Mar 2021

PONE-D-20-36601

Suicide on YouTube: Factors engaging viewers to suicide-themed videos

PLOS ONE

Dear Dr. Kim,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by May 07 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Vincenzo De Luca

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: No

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: 1. Summary of the research and your overall impression

Dear Authors,

Thank you for the privilege of reading your interesting work, Suicide on YouTube: Factors engaging viewers to suicide-themed videos.

In this work, you examine what factors are correlated with viewer engagement for a sample of 100 YouTube videos resulting from the keyword search “suicide”. You examine three groups of factors. “Characteristics of the deliver” (e.g. health professional vs survivor), “characteristics of the content story” (real vs fictional), and “characteristics of content expression” (e.g. genre, warning signs). Engagement factors include views, likes and comments. You then conduct hierarchical multiple regression to quantify the relation between these factors and engagement. You present your results, where you find some aspects of content delivery and expression are significantly correlated with greater or lesser engagement. Your conclusion discusses the results, and how they might apply to various groups such as platforms and health professionals.

I find general strengths of this work include addressing a gap in the literature in an important and applicable subject area; there does not seem to be a lot of work looking into suicidal content of youtube videos, despite their possible impact on an important health outcome. The paper also does a good job of reviewing some older literature, and has some methodological strengths such as you come up with the categories and their factors. Weaknesses include much of the methodology being poorly described, which makes it hard to assess the validity of some of the claims in the paper at this time.

I believe major revisions are warranted in order to address some of the above weaknesses. Please see my comments below for further details.

2. Discussion of specific areas for improvement

Major Issues:

1. The title of the paper, and various points within it, suggest that the study can draw conclusions about suicide videos on YouTube in general. However, only 100 videos are sampled, at one point in time. My understanding is that youtube video search results can very greatly based on the youtube account searched from, time, geography, and other factors. So, I am not sure how representative these 100 videos are, and whether general conclusions can be drawn.

a. Description of this search is required; e.g. where it was searched, when it was searched, and by whom. Additionally, you need to include how it was ranked; the default method I believe is “relevance” but it can also be by view count, like ratio, etc.

b. I believe having the study as an examination of 100 videos at a given point of time as searched by one person is probably still interesting and interpretable, but I believe your paper’s word choice should reflect this. E.g. title could be “…Factors engaging viewers on a sample of suicide-themed videos”, and generally the paper should acknowledge you are only examining one sample of 100 videos.

c. Some discussion about how persistent search results are would be helpful. If someone else searching for “suicide” a week later would get an entirely different set of 100 videos, are the results of your paper still useful? I tested briefly searching with two different Youtube/gmail accounts logged on, and found that there were some different videos resulting each time, though the majority were the same. A general audience unfamiliar with Youtube will want to know how applicable your results are.

d. You do not explain why 100 videos was the number chosen. It would be helpful to know how many videos are out, so we could know how representative this sample is. For example, you find religious organization videos are less engaged with – but what if religious videos in the 100-200th spots are the most engaged? Consider including how many videos might be out there in total, or how many are watched with a certain amount of views e.g. at least 1000 views. Please also see if there is literature discussing what rank of videos usually engage in; if the top 100 videos are usually what 99% of engagement is in, then your sample would be a lot more representative than if it’s only, say, 1%.

e. In summary, the sentence in the paper “However, it can be concluded that the sample consisted of videos that an ordinary user would find using the same keyword” needs to be further substantiated.

2. An important reference for your paper is the WHO 2008 “Preventing Suicide A Resource for Media Professionals” as you use this to determine the content expression characteristic . I believe you are missing this reference in your bibliography, so I assume it is this document you are referring to. However, this was updated in 2017 “Preventing suicide: a resource for media professionals - update 2017” by the WHO.

a. Please include a citation for the report you are using.

b. I believe you should be using the updated 2017 report for your study. On initial glance, your categories may still be applicable given the new update. However, given the importance of this reference, please consider incorporating any necessary changes into your paper.

3. I believe a general strength of the paper is being inclusive of all videos that result from a search. However, when I tried out such a search myself, the top 100 did seem to include at least five videos that were likely not related very much to suicide, such as trailers for the 2016 superhero movie “suicide squad”, and one about a type of car door called “suicide doors” named such because they led to accidental (not intentional) deaths in the past. This makes me wonder if your results are being affected by videos that have very little to do with suicide.

a. The paper does describe how many videos are “music” or “film”.

b. However, I think some discussion of the videos being included would be helpful, especially given that the sample size is not that big. If a basic filtering to remove results clearly not related to suicide is not performed e.g. “suicide doors”, then this should be acknowledged/discussed and perhaps quantified. If no filtering at all was done, please further substantiate and explain the impact of this choice.

Minor issues:

1. I don’t believe data availability is discussed in the paper; the form says it will be in the supplement but this was not available in my manuscript. It may be beneficial to add some details about the data you’ll provide to aid replication?

2. In the major issues section, I discuss how adding further details regarding methodology would be important. Additional areas of methodology should also be described more. Your statistical analysis is not something known by a general audience, and should be explained at least in summary. Additionally, you did not mention how the analysis was performed, including what software was used and any parameters. This is helpful for replication and extension. Discussing why you chose this method, vs other methods, may also be interesting and helpful to add.

3. The authors seem to generally do a good job of citing relevant prior work, and mention the lack of studies look at suicide-related videos on youtube. However, I was able to a few studies that do look quite related published recently in 2020, e.g. High viewership of videos about teenage suicide on YouTube by Dagar and Falcone, and Communication about suicide in YouTube videos: Content analysis of German-language videos retrieved with method-and help-related search terms by Niederkrotenthaler et al. It may be helpful to review and mention these works. This reviewer has no connection to these works or their authors.

4. Table 1 should likely contain median values, especially for the smaller groups where they may be some variation. Alternatively, the authors could consider incorporating graphics such as boxplots to describe the data. I find it a bit hard to read due to the large numbers. If continuing to use numbers, describing the numbers as the nearest thousand (e.g. 1990059 to 1990) might make the numbers easier to compare.

5. Some of the categories add up to more than 100, so I believe some categories can have multiple values. Please address in methodology if this is correct, or what happens if a category is unclear, or multi-valued e.g. a health professional who is also a survivor.

6. Thank you for addressing that you did not examine the like vs dislike ratio in your paper and it would be appropriate for further work. If you have the data readily available, I believe this could be a helpful addition to this paper as another engagement metric that may be quite different than others.

7. I would recommend a different word choice for the sentence “This study focused on one of the self-induced health concerns worldwide” in the abstract. In this context I believe it could be beneficial to describe it more directly as a result of mental health concerns, to emphasise that it is usually due to external factors rather than an individual “choice” e.g. the APA describes it as “Suicide is the act of killing yourself, most often as a result of depression or other mental illness”. Consider other choices such as “focused on a leading cause of death” or “a leading cause of death related to mental health”.

8. Please reconsider or further elaborate on the sentences “As the deliverers of suicide-themed posts are survivors and rescuers rather than health professionals, the platform may play an important role as an arena for diagnosis. The symptoms and the reasons for suicide ideation may be explicitly stated on the platform, which may help health professionals to diagnose individuals who ideate suicide or those with suicide experience”. Has any prior work investigated this? Is there a clinical group (teens?) that posts videos about suicide often enough that this could be clinically useful? Doesn’t youtube already have a “report” button that allows something like this to happen, without the health professionals needing to view the videos directly? As a health professional, this strikes me as too far a jump without a bit more substantiation.

Thank you again for being able to read your work, and I hope you find my feedback is helpful.

Reviewer #2: The paper targets a very interesting topic within social media. Nevertheless it requires major modifications

- The authors should provide sufficient information on the following:

1. The language and region information of the videos analysed. Do any of the videos require age registration?

2. Was the term "Suicide" searched in English? When was the search and video selection performed?

3. Were the browser cache and history cleared before each search and all filters switched off?

- The authors stated that no exclusion criteria were set. It will be useful to exclude unrelated contents (e.g. Music Videos, Playlists, etc.) and/or videos with a length of >10 minutes.

- The "Introduction" and "Literature review and research questions" sections are lengthy and contain redundant information.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: John-Jose Nunez

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 Jun 10;16(6):e0252796. doi: 10.1371/journal.pone.0252796.r002

Author response to Decision Letter 0


7 May 2021

Reviewer #1:

1. Summary of the research and your overall impression

Dear Authors,

Thank you for the privilege of reading your interesting work, Suicide on YouTube: Factors engaging viewers to suicide-themed videos.

In this work, you examine what factors are correlated with viewer engagement for a sample of 100 YouTube videos resulting from the keyword search “suicide”. You examine three groups of factors. “Characteristics of the deliverer” (e.g. health professional vs survivor), “characteristics of the content story” (real vs fictional), and “characteristics of content expression” (e.g. genre, warning signs). Engagement factors include views, likes and comments. You then conduct hierarchical multiple regression to quantify the relation between these factors and engagement. You present your results, where you find some aspects of content delivery and expression are significantly correlated with greater or lesser engagement. Your conclusion discusses the results, and how they might apply to various groups such as platforms and health professionals.

I find general strengths of this work include addressing a gap in the literature in an important and applicable subject area; there does not seem to be a lot of work looking into suicidal content of youtube videos, despite their possible impact on an important health outcome. The paper also does a good job of reviewing some older literature, and has some methodological strengths such as you come up with the categories and their factors. Weaknesses include much of the methodology being poorly described, which makes it hard to assess the validity of some of the claims in the paper at this time.

I believe major revisions are warranted in order to address some of the above weaknesses. Please see my comments below for further details.

(Response)

Thank you for giving us an opportunity to revise our paper. We are deeply grateful for your insightful and constructive comments. We have taken advantage of these comments in carefully preparing this revision. Added or revised parts were highlighted in the revised manuscript with track changes. Please see our detailed explanations (in blue color) in the individual responses to your comments (in black color).

2. Discussion of specific areas for improvement

Major Issues:

1. The title of the paper, and various points within it, suggest that the study can draw conclusions about suicide videos on YouTube in general. However, only 100 videos are sampled, at one point in time. My understanding is that youtube video search results can very greatly based on the youtube account searched from, time, geography, and other factors. So, I am not sure how representative these 100 videos are, and whether general conclusions can be drawn.

a. Description of this search is required; e.g. where it was searched, when it was searched, and by whom. Additionally, you need to include how it was ranked; the default method I believe is “relevance” but it can also be by view count, like ratio, etc.

(Response)

Thank you for your valuable comment. Search was done in Seoul, South Korea at one point in time, September 2019, by the researchers, using Chrome browser. The language was set as ‘English (US)’ and the location was set as ‘United States.’ The search result was ranked by the default method “relevance.” Please see our revision (in highlighted parts) in page 16.

b. I believe having the study as an examination of 100 videos at a given point of time as searched by one person is probably still interesting and interpretable, but I believe your paper’s word choice should reflect this. E.g. title could be “…Factors engaging viewers on a sample of suicide-themed videos”, and generally the paper should acknowledge you are only examining one sample of 100 videos.

(Response)

Thank you for your valuable comment. To respond to your comment, we have changed the title into “Suicide on YouTube: Factors engaging viewers to a selection of suicide-themed videos” as suggested. Please see the new title in page 1 of our revision.

c. Some discussion about how persistent search results are would be helpful. If someone else searching for “suicide” a week later would get an entirely different set of 100 videos, are the results of your paper still useful? I tested briefly searching with two different Youtube/gmail accounts logged on, and found that there were some different videos resulting each time, though the majority were the same. A general audience unfamiliar with Youtube will want to know how applicable your results are.

(Response)

Thank you for your insightful comment. It is true that YouTube search results can be different according to the owner of the account, geography, and so on. Thus, we opened the Google Chrome browser as Incognito Window, which enables private browsing without having to log in. Nevertheless, considering your comment, we have added the applicability of the sample as one of the limitations of the study. Please see our revision (in highlighted parts) in page 29.

Selecting a certain number of videos at a specific time in a specific location with specific language may not incorporate all instances of suicide-themed videos on YouTube. However, reviewing the first two to three pages can be representative enough because they are “most accessible and easiest to find, and therefore the most representative of what the average consumer would view” (Bae and Baxter, 2018: p. 1942).

� Bae SS, Baxter S. YouTube videos in the English language as a patient education resource for cataract surgery. International ophthalmology. 2018; 38(5): 1941-1945. doi: 10.1007/s10792-017-0681-5

In the end, YouTube’s search result suggestion is like a black box. Since the formula of search result presentation on YouTube is unknown, the selection of top several pages in the order of relevance was the best alternative, as many other studies have done the same.

d. You do not explain why 100 videos was the number chosen. It would be helpful to know how many videos are out, so we could know how representative this sample is. For example, you find religious organization videos are less engaged with – but what if religious videos in the 100-200th spots are the most engaged? Consider including how many videos might be out there in total, or how many are watched with a certain amount of views e.g. at least 1000 views. Please also see if there is literature discussing what rank of videos usually engage in; if the top 100 videos are usually what 99% of engagement is in, then your sample would be a lot more representative than if it’s only, say, 1%.

(Response)

Thank you for pointing out the importance of explaining the representativeness of the sample. We agree that we should provide more information on why only 100 videos were the number chosen for the final analysis. There were two different reasons behind the selection of the number of videos chosen for each of the analysis.

For the preliminary analysis, the number (n=100) was chosen because it was the point where saturation has been reached. The researchers have agreed that the iterative process can stop. No new codes that could be meaningful to the establishment of coding protocols were generated. The researchers have agreed that the continuation of analysis may lead to the waste of resources, thus, the iterative process has come to a cease. Please refer to line 365 to 370 in page 17.

“Every newly observed item for each factor was recorded. The list of the items for each factor was built upon after several iterative processes until saturation had been reached. The iterative process continued until only redundant instances were observed and until no new codes occurred to the degree in which the researchers have agreed that further data collection or data coding is counter-productive (Saunders et al., 2018).”

For the final analysis, 100 videos were chosen because the purpose of the research was to reflect a basic query of what general people would likely to be exposed to with the keyword “suicide.” This research suggests that the first several pages of search results presented to the person searching the keyword engage most viewers.

Previous surveys developed by iProspect, a performance-driven digital marketing agency, and Jupiter Research indicate that people tend to click what appears within the first page of results when using search engines. In the whitepaper, it says “62% of search engine users click on a search result within the first page of results, and a full 90% of search engine users click on a result within the first three pages of search results” (iProspect, 2006). A qualitative study by Eysenbach and Köhler (2002) also suggests that search results on the second or following pages were less attended by the searchers searching for health-related information. This implies that people tend to select the information they are provided with first rather than the information they are provided later. The former work (iProspect Survey) has been cited in multiple health information-related studies that included the first several pages in the sample:

� Kelly-Hedrick, Grunberg, Rochu, and Zelkowitz’s 2018 research with keyword search “infertility” where 80 top-viewed YouTube videos were included for the analysis, including the first 4 pages of results (20 results per page)

� Stellefson, Chaney B, Ochipa, Chaney D, Haider, Hanik, Chavarria, and Bernhardt’s 2014 work with keyword search “Chronic Obstructive Pulmonary Disease,” “COPD,” “COPD management,” and “COPD self-management” where 223 unique videos were saved for analysis, including the first 7 pages of results

� Wasserman, Baxter, Rosen, Burnstein and Halverson’s work in 2014, screened Web site links on the first 2 pages of search (18 to 20 links per page)

� Sahin AN, Sahin AS, Schwenter and Sebajang’s 2019 study where the first two pages of search results were reviewed for the keyword search “colorectal cancer,” “colon cancer,” and “bowel cancer.”

Unlike the search results from search engines, YouTube has no clear distinction in terms of pages. After several videos, a swipe up motion leads to the loading of more videos. Considering that the number of videos presented before the first swipe up motion was 20, it can be regarded that a page on YouTube contains 20 videos. A total of 589 videos were available for the search term “suicide” after pages loaded until ‘No more results’ were left to show.

The first two to three pages were the most commonly observed sample for YouTube content analysis on health-related matters. However, a power analysis was performed as Niederkrotenthaler, Schacherl, and Till did to identify the minimum number of samples required. The desired sample size was computed with the software G*Power 3.1 (Faul, Erdfelder, Buchner, and Lang, 2009). In order to identify a medium-sized difference effect size (f²)=0.15; with an Alpha-level of 0.05 and Power (1-β error prob)=0.80, with the number of predictors n=3 (who, what, how), and the total number of predictors 44 (1 continuous variable and 43 dummy variables), a total of 82 videos were required as a minimum. Since each page holds 20 videos, this study required more than four pages to meet the minimum number of samples. Thus, five pages were included in the final sample, in other words, 100 videos. Niederkrotenthaler, Schacherl and Till’s study in 2020 included less than 100 videos for each search term. The sample size (n=100) should be fine, considering the relevance of the videos on the first three pages. Please see our revision (in highlighted parts) in pages 17~18.

� iProspect. iProspect Search Engine User Behavior Study. 2006. Available from: http://district4.extension.ifas.ufl.edu/Tech/TechPubs/WhitePaper_2006_SearchEngineUserBehavior.pdf

� Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002; 324: 573–577. doi: 10.1136/bmj.324.7337.573

� Kelly-Hedrick M, Grunberg PH, Brochu F, Zelkowitz P. “It’s totally okay to be sad, but never lose hope”: content analysis of infertility-related videos on YouTube in relation to viewer preferences. Journal of medical Internet research. 2018; 20(5): e10199. doi: 10.2196/10199

� Stellefson M, Chaney B, Ochipa K, Chaney D, Haider Z, Hanik B, et al. YouTube as a source of chronic obstructive pulmonary disease patient education: A social media content analysis. Chronic respiratory disease. 2014; 11(2): 61-71. doi: 10.1177/1479972314525058

� Wasserman M, Baxter NN, Rosen B, Burnstein M, Halverson AL. Systematic review of internet patient information on colorectal cancer surgery. Dis Colon Rectum. 2014; 57(1): 64–69. doi: 10.1097/DCR.0000000000000011

� Sahin AN, Sahin AS, Schwenter F, Sebajang H. YouTube videos as a source of information on colorectal cancer: what do our patients learn?. Journal of Cancer Education. 2019; 34(6): 1160-1166. doi: 10.1007/s13187-018-1422-9

� Leong AY, Sanghera R, Jhajj J, Desai N, Jammu BS, Makowsky MJ. Is YouTube useful as a source of health information for adults with type 2 diabetes? A South Asian perspective. Canadian journal of diabetes. 2018; 42(4): 395-403. doi: 10.1016/j.jcjd.2017.10.056

� Bae SS, Baxter S. YouTube videos in the English language as a patient education resource for cataract surgery. International ophthalmology. 2018; 38(5): 1941-1945. doi: 10.1007/s10792-017-0681-5

e. In summary, the sentence in the paper “However, it can be concluded that the sample consisted of videos that an ordinary user would find using the same keyword” needs to be further substantiated.

(Response)

Thank you for your comment. To respond to your comment, we have changed the sentence to “However, it can be concluded that the sample consisted of videos maintained available on the platform that an ordinary user would find using the same keyword.” Please see our revision in line 549 of page 29.

Before this sentence, we described that extremely triggering or harmful content had already been removed by the platform because teams like YouTube’s ‘Trust and Safety team’ actively review ‘flagged’ content. This can be the limitation of the sample in that only the remaining videos can be observed, unless timely searched (searched ‘before’ the content has been put down or hidden by the platform). The flagged or removed content may or may not be more triggering than what is left on the platform. Since it is impossible to predict when those potentially ‘flagged’ content will be uploaded and to include those videos in the sample, observing what is remaining on the platform has been the best option. Thus, the observed videos are still meaningful in that they are similar to what an ordinary user would find because those videos are the ones maintained available, still accessible to ordinary users.

2.An important reference for your paper is the WHO 2008 “Preventing Suicide A Resource for Media Professionals” as you use this to determine the content expression characteristic . I believe you are missing this reference in your bibliography, so I assume it is this document you are referring to. However, this was updated in 2017 “Preventing suicide: a resource for media professionals - update 2017” by the WHO.

a. Please include a citation for the report you are using.

(Response)

As suggested, the WHO 2008 report was added to our reference. In addition, we have reviewed the updated 2017 version and have referred to the report.

b. I believe you should be using the updated 2017 report for your study. On initial glance, your categories may still be applicable given the new update. However, given the importance of this reference, please consider incorporating any necessary changes into your paper.

(Response)

Thank you for the update. The content of the two reports is very similar, suggesting guidelines for the media professionals. The main difference was that the latest version acknowledges the helpful impact of responsible reporting better than the former version. ‘Digital media as a double-edged sword in terms of suicide-related information’ has been the very essence of the manuscript. The 2017 updated report has been an additional support for our paper.

3.I believe a general strength of the paper is being inclusive of all videos that result from a search. However, when I tried out such a search myself, the top 100 did seem to include at least five videos that were likely not related very much to suicide, such as trailers for the 2016 superhero movie “suicide squad”, and one about a type of car door called “suicide doors” named such because they led to accidental (not intentional) deaths in the past. This makes me wonder if your results are being affected by videos that have very little to do with suicide.

a. The paper does describe how many videos are “music” or “film”.

b. However, I think some discussion of the videos being included would be helpful, especially given that the sample size is not that big. If a basic filtering to remove results clearly not related to suicide is not performed e.g. “suicide doors”, then this should be acknowledged/discussed and perhaps quantified. If no filtering at all was done, please further substantiate and explain the impact of this choice.

(Response)

Thank you for your valuable comment. The video selection process did not have exclusion criteria for the preliminary analysis. On the other hand, videos related to the superhero movie “Suicide Squad” were not included in the final analysis. This study explores suicide-themed videos, which are different from videos with the word ‘suicide.’ In order to examine suicide relevant videos, “Suicide Squad” videos were excluded. The authors have added information regarding the exclusion criteria for the final sample. “Suicide Squad” videos were discernible through the video title and thumbnail, as the major actors and characters were visible in the thumbnail area. The researchers have eliminated 7 “Suicide Squad” videos, and had to add 7 other videos to make 100. Videos categorized as ‘music’ or ‘film’ were not related to “Suicide Squad” but they were videos created by individuals who express suicide-related information through the form of music or film. Other videos like ‘suicide doors’ were included because those videos were difficult to exclude unless the content of the video was watched. Please see our revision in line 379 ~ 386 of pages 17~ 18.

Minor issues:

1.I don’t believe data availability is discussed in the paper; the form says it will be in the supplement but this was not available in my manuscript. It may be beneficial to add some details about the data you’ll provide to aid replication?

(Response)

Thank you for this comment. We have provided all relevant data underlying the findings within the manuscript. To aid replication, we would like to provide our codebook, so that any researcher who is interested in the further study may use it with his/her own sample.

2. In the major issues section, I discuss how adding further details regarding methodology would be important. Additional areas of methodology should also be described more. Your statistical analysis is not something known by a general audience, and should be explained at least in summary. Additionally, you did not mention how the analysis was performed, including what software was used and any parameters. This is helpful for replication and extension. Discussing why you chose this method, vs other methods, may also be interesting and helpful to add.

(Response)

Thank you for this valuable comment. The purpose of the study was to figure out which factor engages viewers to suicide-themed videos. In order to find the answer to the research question, it was necessary to incorporate multiple variables into consideration. Since hierarchical multiple regression enables the researchers to build several models to compare the proportion of explained variance in the dependent variable by sequentially adding models, this method was chosen.

We have elaborated the details regarding methodology in the manuscript, providing 1) a brief summary, 2) information on the software used, and 3) a reference material (step-by-step how-to guide for hierarchical linear regression) that can be helpful for the readers. Please refer to line 429 to 437, which reads:

Hierarchical multiple regression is a method that considers the relative effect of more than one explanatory variable on the dependent variable of interest. It enables the researchers to build several models to compare the proportion of explained variance in the dependent variable by sequentially adding models. The newly added models always include the previous models. The analysis can determine which model better explains and predicts the dependent variable in a statistically meaningful way. The current study takes three models, sometimes referred as blocks: who, what, and how variables of the suicide-themed content in explaining viewer engagement. The analysis was completed on IBM SPSS (Statistical Package for the Social Sciences) Statistics software (SAGE Publications, 2019).

3. The authors seem to generally do a good job of citing relevant prior work, and mention the lack of studies look at suicide-related videos on youtube. However, I was able to a few studies that do look quite related published recently in 2020, e.g. High viewership of videos about teenage suicide on YouTube by Dagar and Falcone, and Communication about suicide in YouTube videos: Content analysis of German-language videos retrieved with method-and help-related search terms by Niederkrotenthaler et al. It may be helpful to review and mention these works. This reviewer has no connection to these works or their authors.

(Response)

Thank you so much for the information. We have reviewed these two works and found high relevance to our manuscript. We have referred to these works and accordingly added them on the reference list.

4. Table 1 should likely contain median values, especially for the smaller groups where they may be some variation. Alternatively, the authors could consider incorporating graphics such as boxplots to describe the data. I find it a bit hard to read due to the large numbers. If continuing to use numbers, describing the numbers as the nearest thousand (e.g. 1990059 to 1990) might make the numbers easier to compare.

(Response)

Thank you for your suggestion. In order to make the manuscript concise, as it already contains many Tables with numbers, we would like to continue using numbers. However, as suggested, we have changed the numbers as the nearest thousand to make the comparison easier for the readers. The legend provides information that 1) the unit is one thousand and 2) numbers below one thousand are written as “-”.

5. Some of the categories add up to more than 100, so I believe some categories can have multiple values. Please address in methodology if this is correct, or what happens if a category is unclear, or multi-valued e.g. a health professional who is also a survivor.

(Response)

Thank you for this comment. 5 variables add up to more than 100 because those variables had multiple values.

Please refer to the manuscript line 442 to 446 which reads: “Among 14 variables, 5 variables including message deliverer category, whether the story is about a suicide attempt, completed suicide, or suicide ideation, illustration of method, placement of the warning sign, and placement of hotlines were multi-coded. Thus, the number of instances coded in each category may not be equal to the total number of observed instances which is 100.”

6. Thank you for addressing that you did not examine the like vs dislike ratio in your paper and it would be appropriate for further work. If you have the data readily available, I believe this could be a helpful addition to this paper as another engagement metric that may be quite different than others.

(Response)

Thank you for this comment. We would like to leave ‘like vs dislike ratio’ metric for future work. Although ‘the number of views,’ ‘the number of likes,’ ‘the number of comments,’ and ‘like vs dislike ratio’ are all engagement metrics, they have different meanings. It would be better to explore them in the future work.

7. I would recommend a different word choice for the sentence “This study focused on one of the self-induced health concerns worldwide” in the abstract. In this context I believe it could be beneficial to describe it more directly as a result of mental health concerns, to emphasise that it is usually due to external factors rather than an individual “choice” e.g. the APA describes it as “Suicide is the act of killing yourself, most often as a result of depression or other mental illness”. Consider other choices such as “focused on a leading cause of death” or “a leading cause of death related to mental health”.

(Response)

Thank you for your advice. As suggested, we have revised the sentence as follows: “This study focused on one of the health concerns worldwide, a leading cause of death related to mental health”.

8. Please reconsider or further elaborate on the sentences “As the deliverers of suicide-themed posts are survivors and rescuers rather than health professionals, the platform may play an important role as an arena for diagnosis. The symptoms and the reasons for suicide ideation may be explicitly stated on the platform, which may help health professionals to diagnose individuals who ideate suicide or those with suicide experience”. Has any prior work investigated this? Is there a clinical group (teens?) that posts videos about suicide often enough that this could be clinically useful? Doesn’t youtube already have a “report” button that allows something like this to happen, without the health professionals needing to view the videos directly? As a health professional, this strikes me as too far a jump without a bit more substantiation.

(Response)

Thank you for valuable comment. One of the findings of this study showed that content delivered by survivors of suicide attempt, artists, musicians, film personnel, and one-person creators received more likes and comments. Self-expressive individuals produce videos about their experiences and thoughts where the platform makes it possible. As the platform environment enables individuals to freely upload content, some individuals share a series of stories related to their own suicide attempt. If collective coping is really happening, it will be worthwhile for the health professionals to examine how an individual overcomes suicidal thoughts and how the interaction of content uploader-content-and-viewers helps the prevention of suicide. There can be interesting dynamics of collective coping.

The ‘report’ or ‘flag’ button raises the attention of YouTube regarding inappropriate content, so that the platform can take a legitimate action, either to remove the content that violates Community Guidelines, or to put age restriction. It has little to do with diagnosing individuals. The purpose of reviewing the content may be different between the platform ‘YouTube’ and health professionals.

Thank you again for being able to read your work, and I hope you find my feedback is helpful.

(Response) Thank you for your helpful comments again. We have really enjoyed your feedback and did our best to respond to your comments.

Reviewer #2:

The paper targets a very interesting topic within social media. Nevertheless it requires major modifications

- The authors should provide sufficient information on the following:

(Response)

Thank you for giving us an opportunity to revise our paper. We are deeply grateful for your insightful and constructive comments. We have taken advantage of these comments in carefully preparing this revision. Added or revised parts were highlighted in the revised manuscript with track changes. Please see our detailed explanations (in blue color) in the individual responses to your comments (in black color).

1. The language and region information of the videos analysed. Do any of the videos require age registration?

(Response)

Thank you for this comment. The videos were analyzed in English while the researchers were located in Seoul, South Korea. The number of videos that required age registration was 5. Please refer to Table 3 in the manuscript under the category, ‘YouTube Warning.’ We have added these details in the manuscript. Please refer to line 532~534.

2. Was the term "Suicide" searched in English? When was the search and video selection performed?

(Response)

Thank you for this comment. The term “Suicide” was searched in English. The search and video selection were performed at one point, September 2019.

3. Were the browser cache and history cleared before each search and all filters switched off?

(Response)

Thank you for this question. No, the browser cache and history were not cleared before the search. However, the browser was opened as Incognito Window, which enables private browsing without having to log in. All filters were switched off.

- The authors stated that no exclusion criteria were set. It will be useful to exclude unrelated contents (e.g. Music Videos, Playlists, etc.) and/or videos with a length of >10 minutes.

(Response)

Thank you for your comment. Neither specific inclusion nor exclusion criteria was set in the sample selection process for the preliminary analysis for the purpose of extracting all possible codes relevant to suicide-themed videos. In order to build a coding system, all possible instances were included.

Videos with a length of >10 minutes and music videos were included because the authors do find all videos relevant. As the purpose of this study is to examine a general search result and as the authors have not found evidence stating people choose videos depending on the running time (length) of the video, we could not exclude those videos. We have not excluded music videos. Playlists were excluded because the unit of the analysis was an ‘individual video.’ Since playlists are discernable in the thumbnail area that shows the number of videos in the playlist, it was easily spotted and was not included in the sample.

In the final analysis, videos related to “Suicide Squad,” a superhero movie based on DC comics, were excluded because “Suicide Squad” has low relevance to the health-related issue of suicide, although the movie title does include the word ‘suicide.’

- The "Introduction" and "Literature review and research questions" sections are lengthy and contain redundant information.

(Response)

Thank you for your advice. To respond to your advice, we have tried to eliminate some redundant information in our revised manuscript.

Attachment

Submitted filename: Response to Reviewers.docx

Decision Letter 1

Vincenzo De Luca

24 May 2021

Suicide on YouTube: Factors engaging viewers to a selection of suicide-themed videos

PONE-D-20-36601R1

Dear Dr. Kim,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Vincenzo De Luca

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you authors for your great work addressing my comments, I think it has added some additional rigor and reproducibility to your interesting and insightful paper. I have no additional comments.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: John-Jose Nunez, M.D.

Acceptance letter

Vincenzo De Luca

1 Jun 2021

PONE-D-20-36601R1

Suicide on YouTube:Factors engaging viewers to a selection of suicide-themed videos

Dear Dr. Kim:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Vincenzo De Luca

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Codebook

    (DOCX)

    S1 File

    (XLSX)

    S2 File

    (DOCX)

    Attachment

    Submitted filename: Response to Reviewers.docx

    Data Availability Statement

    All relevant data are within the manuscript and we provide our codebook.


    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES