The SmartSleep Experiment: Evaluation of changes in night-time smartphone behavior following a mass media citizen science campaign

Thea Otte Andersen; Agnete Skovlund Dissing; Tibor V Varga; Naja Hulvej Rod

doi:10.1371/journal.pone.0253783

. 2021 Jul 21;16(7):e0253783. doi: 10.1371/journal.pone.0253783

The SmartSleep Experiment: Evaluation of changes in night-time smartphone behavior following a mass media citizen science campaign

Thea Otte Andersen ^1,^*, Agnete Skovlund Dissing ¹, Tibor V Varga ¹, Naja Hulvej Rod ¹

Editor: Camelia Delcea²

PMCID: PMC8294485 PMID: 34288929

Abstract

The increasing 24-hour smartphone use is of public health concern. This study aims to evaluate whether a massive public focus on sleep and smartphone use generated through a large-scale citizen science project, the SmartSleep Experiment, influence participants’ night-time smartphone behavior. A total of 8,894 Danish adults aged 16 and above participated in the SmartSleep Experiment, a web-based survey on smartphones and sleep behavior. The survey was carried out for one week in 2018, combined with an extensive national mass media campaign focusing on smartphone behaviors and sleep. A follow-up survey aimed at evaluating whether survey-participants had changed their night-time smartphone behavior was carried out two weeks after the campaign. A total of 15% of the participants who used their smartphone during sleep hours at baseline had changed their night-time smartphone behavior, and 83% of those indicated that they used their smartphone less at follow-up. The participants who had changed their smartphone behavior had primarily taken active precautions to avoid night-time smartphone use, e.g., activating silent mode (36%) or reduced their smartphone use before (50%) and during sleep hours (52%). The reduction in sleep problems (54%), recognition of poor smartphone behavior (48%), and the increased focus on night-time smartphone use (42%) were motivational factors for these behavior changes. Using citizen science and mass media appeared to be associated with changes in night-time smartphone behavior. Public health projects may benefit from combining citizen science with other interventional approaches.

Introduction

Smartphones have become an integrated part of everyday life, and the increasing and widespread use of smartphones is an inevitable trend in today’s digitized society. Smartphones are frequently used around the clock [1–3]. Previous studies have shown that night-time smartphone use is related to poor sleep quality and shorter sleep duration [1, 3–6]. Furthermore, a recent randomized trial with 38 college students found that restricting mobile phone use before bedtime reduced sleep latency and increased sleep duration [7]. Because poor and short sleep is a well-known risk factor for obesity, cardio-vascular diseases, diabetes, and mortality [8, 9], the increasing 24-hour smartphone use highlights a pressing public health issue. Thus, there is a need for public health prevention strategies to change night-time smartphone behavior to eventually improve sleep behavior and health.

Citizen science, defined as public participation in scientific research in which the public are engaged directly in one or more of the research processes [10, 11] has proven to be an effective strategy to maximize the social impacts of research [12]. It has also been highlighted as a strategy to improve the linkage between research, education, and action [13, 14]. The use of citizen science has expanded in recent years, especially in biomedical and public health research [15, 16]. Citizen science in public health research helps generate scientific knowledge and new insights into complex problems [12, 17]. Also, it makes it possible to investigate the nuanced understandings of public health trends and mechanisms [13, 14]. Furthermore, it has been argued that citizen science and mass media campaigns may serve as a method of health promotion, as participants might increase their health literacy, change their attitudes and norms, and increase their awareness of their own health behaviors [16–19]. All these changes may lead to subsequent behavior changes [17, 20–22].

Only a few public health citizen science projects to date have evaluated whether the citizen science approach per se has an interventional effect on the participants’ health behavior [18, 23]. This may be due to the fact that there are no commonly established measures for evaluating citizen science projects or mass media campaigns and collecting evidence of their impact [20]. Therefore, there is only sporadic evidence in the literature on potential behavioral interventional impacts of mass media campaigns and participating in citizen science public health projects.

In 2018, the SmartSleep Experiment, a web-based survey about smartphone use and sleep, was carried out in collaboration with the Danish Broadcasting Corporation (DR) using a citizen science approach. During one week of data collection, DR created a national mass media campaign focusing on night-time smartphone behaviors and sleep, which was used as a vehicle to create attention and public interaction around the research project. The public attention helped form a framework for recruiting more than 25,000 participants for an online survey. This paper aims to evaluate whether the massive public focus on night-time smartphone use and sleep using mass media during the SmartSleep Experiment was associated with reductions in night-time smartphone use among participants. We will specifically 1) explore whether participants had changed their night-time smartphone behavior and in which direction, and 2) examine behavioral and motivational factors that may have influenced their smartphone behavior changes among a subsample of 8,894 people who participated in the SmartSleep Experiment and were followed-up immediately after the massive public focus on smartphone use and sleep.

Material and methods

The SmartSleep Experiment

The SmartSleep Experiment was carried out for one week in November 2018. The core scientific element of the SmartSleep Experiment was an online survey aimed at documenting smartphone use during sleep hours in the adult population. The participants were actively involved in the data collection by filling out a survey, which was an integrated part of the media campaign. This was combined with direct individual feedback to the participants and real-time presentation of preliminary results from the data collection to all radio listeners during the week of the campaign. The participants accessed the survey either via the SmartSleep website (www.smartsleep.ku.dk) or a link on DR’s website (www.dr.dk/soverdugodt) to which they were directed from the radio programs, teasers, online articles, and social media. The participants had to be 16 years or older and understand written Danish. The online survey contained information on demographics, smartphone use, sleep behavior, and health-related questions.

The present study was approved by the Danish Data Protection Agency through the joint notification of The Faculty of Health and Medical Sciences at The University of Copenhagen (record no 514-0237/18-3000 and 514-0288/19-3000). All data was handled according to GDPR guidelines. Survey-based studies do not require ethical approval by the Danish National Committee on Health Research Ethics according to Danish Law. Written informed consent was obtained from all participants. The participants were informated about the purposes of the research and their rights to withdraw.

Direct personal feedback

To motivate people to participate in the survey, they received immediate personal feedback on their night-time smartphone use after completing the survey. The personal feedback included a description of the participant’s night-time smartphone use and a comparison with results from the Danish Regional Health Survey 2017 [24]. The comparison included the proportion of participants who did not get enough sleep to feel rested and the proportion of participants who stated that the reason for insufficient sleep was the use of or disturbance of their smartphone according to various age groups (16–24, 25–34, 35–44, 45–54, 55–64, 65–74, 75+). Please see an example of a personal feedback in S1 Fig.

Massive public media coverage on night-time smartphone use and sleep

During the SmartSleep Experiment week, DR created a public campaign, "Do you sleep well?" on several of their platforms including radio programs, their website, and social media pages. The three-hour national morning radio show "Good morning P3" on the radio channel DR P3 was the main driving force of the SmartSleep Experiment. Each morning during the week, the radio program focused on smartphone use or sleep from different perspectives and encouraged listeners to engage in discussions and participate in the survey. To further create a feedback process between the researchers and participants, preliminary results–"The results of the day" were generated by the researchers at the University of Copenhagen each day and reported live on the radio. Furthermore, the researchers were live in the studio every morning during that week. The radio channel DR P3 has primarily young to middle-aged listeners and was the second most listened to radio channel in Denmark in 2018. Media analyses from DR show that during the SmartSleep Experiment week, 1.5 million people listened on P3. Furthermore, each morning during the week, approximately 489,000 listened to the morning radio program "Good Morning P3" and there were 807,000 unique listeners (15% of the Danish population aged 12 and above) for the entire week to "Good Morning P3"

Local radio programs at the regional radio channel DR P4 were also used as a platform to engage participants to improve geographical representation and generate further attention to the SmartSleep Experiment. The researchers were also interviewed about the project on the local radio programs on DR P4. The radio programs were also focused on various themes related to smartphone use and sleep. The regional radio channel P4 has primarily middle-aged listeners, and it is the most listened to radio channel covering ten regions of Denmark. During the week of the SmartSleep Experiment, approximately 2.7 million individuals (53% of the Danish population aged 12 and above) listened to the P4 regional radio channels.

More than 20 news and feature articles focusing on the importance of sleep for health and well-being and on how night-time night-time smartphone use may result in disturbed sleep were published during the week of the experiment on DR’s website and their social media pages, including Twitter, Facebook, and Instagram. Furthermore, an animated video was published on DR’s online platforms demonstrating the potential health problems associated with night-time smartphone use. The video also encouraged people to participate in the survey.

A countdown timer was set up on DR’s website on the last day of the SmartSleep Experiment showing the time left for respondents to answer the survey. Furthermore, various radio programs at DR encouraged their listeners to participate in the survey before the end of the experiment at midnight.

Follow-up survey

Two weeks after the SmartSleep Experiment, participants who previously indicated they wanted to participate in future studies received an online follow-up survey via e-mail. The follow-up survey aimed to evaluate whether the participants had changed their smartphone behavior after the massive public focus on smartphone use and sleep. Specifically, the follow-up survey focused on night-time smartphone use, changes in night-time smartphone use, and motivational factors for smartphone behavior changes. Up to three reminders were sent to those not responding to the first e-mail.

Study population and data collection

A total of 25,135 Danish adults participated in the core survey on the SmartSleep Experiment during the citizen science week. Two weeks after the SmartSleep Experiment, 12,348 (49%) participants who had indicated that they wanted to participate in future studies received the online follow-up survey. In total, 8,911 responded to the follow-up survey (72% response rate). Seventeen participants were excluded from the study population because they did not have a smartphone. Thus, 8,894 participants were eligible for the analyses. S2 Fig shows the flowchart of the study population. S1 Table shows characteristics of individuals who at baseline did not agree to participate in future studies (n = 12,887), individuals who agreed at baseline but did not participate in the follow-up study (n = 3,337), and individuals who participated in the follow-up study (n = 8,911). Those who participated in the follow-up study were generally older, had higher educational level, and were slightly less likely to use their smartphone during sleep hours at baseline than those who did not participate.

Measurements

Perceived changes in night-time smartphone use were assessed by the question: "Have you changed your smartphone behavior at night after you have answered the last SmartSleep survey?"

Yes, I have changed them
I have tried changing them, but it was unsuccessful
No, I have not changed behavior.

The direction of change was assessed by asking the participants whether they check their smartphone during sleep hours when they would typically sleep more or less since completion of the SmartSleep survey

More often
Less often
at the same level as before

Behavioral factors

Those participants who reported to have changed their night-time smartphone behavior were asked how they had changed their behavior; response options: a) I set my smartphone on silent mode, flight mode or do not disturb, b) I turn off my smartphone, c) I place my smartphone out of reach, d) I reduce my smartphone use before falling asleep, e) I reduce my smartphone use during sleep hours, f) I use an analog alarm clock instead of the alarm clock on my smartphone, and g) other. The participants could select multiple response options.

Motivational factors

Those participants who changed their night-time smartphone behavior were asked about motivational factors, response options: a) Increased focus on night-time smartphone use in the media, e.g., the theme "Do you sleep well?" on DR, b) I have received new knowledge about the health consequences of poor sleep, c) I have discussed with people close to me about my smartphone behavior, d) I felt that my smartphone behavior was bad for me, e) I wanted to reduce my sleep problems, and f) other. The participants could select multiple response options.

Age (10-year bands), gender, educational level (low (primary school); medium (upper secondary school; technical vocational education); high education (short, medium, and long cycle higher education); and other based on the International Standard Classification of Education 2011 [25]), occupational status (employed; student; unemployed; outside labor market; long-term sick leave; other), cohabitation (living alone versus not living alone), baseline night-time smartphone user (yes versus no) and sleep quality was obtained in the baseline survey. Baseline night-time smartphone user was assessed by asking how often the smartphone was used after falling asleep and during sleep hours within the past three months with the following response options: every night or almost every night; several nights a week; several nights a month or less; and never. Smartphone use was defined in the baseline survey and referred to both short and long activation of the smartphone. Sleep quality was assessed using a Danish translation of a validated short version of the Karolinska Sleep Questionnaire (KSQ) [26]. The KSQ includes four items covering the frequency of poor sleep quality rated from never to every night or almost every night. KSQ ranges from one to five and a higher score indicates poorer sleep quality.

Analytical strategy

First, we report the distribution of participation during the week of the SmartSleep Experiment among 25,135 participants. We plotted the distribution of participants according to the date and time each participant finished the online survey. Of these, 8,894 participants also took part in the follow-up survey. We described characteristics of the participants according to their changes in night-time smartphone behavior to explore differences between the groups. Differences in categorical variables were tested with a chi-squared test and for differences in continuous variables, we used ANOVA.

To investigate whether participants who used their smartphone during sleep hours at baseline had changed their night-time smartphone behavior after the SmartSleep Experiment, we restricted the population to those who reported using their smartphone during sleep hours at baseline (n = 4,926). We calculated the proportions who reported to have changed their smartphone behavior at follow-up. Finally, we assessed the proportions of behavioral and motivational factors for changes in night-time smartphone behavior among those who reported to had changed their night-time smartphone behavior to explore the mechanisms behind behavior changes. All analyses were performed using R version 3.6.3.

Results

Distribution of participation

A total of 25,135 people participated in the SmartSleep Experiment. Fig 1 shows the distribution of participation including key events during the week of the SmartSleep Experiment. The daily distribution of participants clearly shows an impact of the partnership with DR. While the survey was online on DR’s website and on the website of the SmartSleep Experiment from day 1 (9 Nov 2018), only a few articles were published on DR between day 1 and 3 (9–11 Nov 2018). Thus, only 2,448 responded to the survey before 12 Nov 2018. The fourth day (12 Nov 2018) marked the launch day of the SmartSleep Experiment in various radio programs including "Good Morning P3!" There is a corresponding major increase in the survey participants on this day (total 9,478 participants on 12 Nov 2018). Between 12–16 Nov 2018, participation was primarily observed in the morning and the afternoon/evening. Notably, 10% of all participants filled in the survey at night between 11 PM and 7 AM. There was a peak in participation at the end of the experiment on the final day of the survey (16 Nov 2018), probably because of the countdown presented on all media platforms on this day.

Characteristics of participants with changed smartphone behavior

Table 1 shows the characteristics of the study population according to whether they have changed their smartphone behavior at follow-up. Almost one in ten of the participants (9%) indicated that they had changed their night-time smartphone behavior after participating in the SmartSleep Experiment. Younger participants were more likely to change or try to change their smartphone behavior than older participants. Slightly more men than women did not change smartphone behavior. More participants with high education did not change their smartphone behavior compared to participants with low or medium education. Additionally, more participants with low or medium education tried to change their smartphone behavior, but were unsuccessful compared to participants with high education. Participants who were employed or outside labor market were more likely not to change their smartphone behavior, while unemployed and students were more likely to change or try to change their smartphone behavior. As expected, participants who used their smartphone during sleep hours at baseline were more likely to change or try to change their smartphone behavior than those who did not use their smartphone during sleep hours. S2 Table shows the socio-demographic differences in baseline night-time smartphone use. It appears that participants with high education, employed people, and participants outside labor market were less likely to use their smartphones during sleep hours at baseline, while students were more likely to use their smartphones during sleep hours at baseline. Participants who changed or tried to change their smartphone behavior had poorer sleep quality compared to participants who did not change their smartphone behavior.

Table 1. Characteristics of the study population according to changed night-time smartphone behavior among 8,894 participants in the SmartSleep Experiment.

	Total n = 8,894	Changed behavior n = 838 (9%)	Tried to change behavior, but was unsuccessful n = 549 (6%)	Have not changed behavior n = 7,507 (85%)	P-value
Age, n (%)
16–25	923(10)	102 (11)	119 (13)	702 (76)
26–35	1,620 (18)	177 (11)	130 (8)	1,313 (81)
36–45	1,710 (19)	152 (9)	112 (7)	1,446 (85)
46–55	2,109 (24)	202 (10)	116 (6)	1,791 (85)
56–65	1,702 (19)	139 (8)	63 (4)	1,500 (88)
+65	830 (9)	66 (8)	9 (1)	755 (91)	<0.001
Gender^a, n (%)
Female	5,387 (61)	547 (10)	376 (7)	4,464 (83)
Male	3,496 (39)	289 (8)	173 (5)	3,024 (87)	<0.001
Educational level^b, n (%)
Low	380 (4)	38 (10)	40 (11)	302 (80)
Medium	2,042 (23)	205 (10)	157 (8)	1,680 (82)
High	6,283 (71)	573 (9)	340 (5)	5,370 (86)
Other	189 (2)	22 (12)	12 (6)	155 (82)
Occupational status, n (%)
Employed	5,853 (66)	530 (9)	319 (6)	5,004 (86)
Student	1,218 (14)	135 (11)	142 (12)	941 (77)
Unemployed	233 (3)	30 (13)	24 (10)	179 (77)
Outside labor market	1,144 (13)	96 (8)	25 (2)	1,023 (89)
Long-term sick leave	119 (1)	10 (8)	16 (13)	93 (78)
Other	327 (4)	37 (11)	23 (7)	267 (82)	<0.001
Baseline night-time smartphone user, n (%)
Yes^c	4,926 (55)	720 (15)	509 (10)	3,697 (75)
No^d	3,968 (45)	118 (3)	40 (1)	3,810 (96)	<0.001
Cohabitation, n (%)
Living alone	1,935 (22)	195 (10)	141 (7)	1,599 (83)
Not living alone	6,959 (78)	643 (9)	408 (6)	5,908 (85)	0.029
Sleep quality, mean (SD)	2.8 (1)	3.1 (0.9)	3.3 (0.9)	2.7 (1)	<0.001

Open in a new tab

^a NA, n = 11

^c Low education: Primary school; Medium education: Upper secondary school or technical vocational education; High education: Short, medium, or high cycle higher education

^c Reporting ‘every night or almost every night’, ‘several nights a week’ or ‘several nights a month or less’

^d Reporting ‘never’.

Change in night-time smartphone use and the direction of change at follow-up among baseline night-time smartphone users

In total, 720 participants (15%) out of 4,926 participants who used their smartphone during sleep hours at baseline reported having changed their night-time smartphone behavior after the SmartSleep Experiment (Table 1). As demonstrated in Fig 2, 83% of these participants stated that they used their smartphone less during sleep hours. Also, 91% of those who indicated did not change behavior at follow-up stated that they used their smartphone at the same level as before the SmartSleep Experiment.

Behavioral and motivational factors for changes in night-time smartphone behavior

Approximately half of the participants who changed their night-time smartphone behavior reduced their smartphone use before falling asleep at night (50%) and/or during sleep hours (52%), as shown in Fig 3. Furthermore, around one-third of the participants had taken specific preventative precautions to avoid night-time smartphone use. E.g. 36% indicated that they had set their smartphone on silent mode, flight mode, or ’do not disturb’ during sleep hours, and 29% reported that they had placed their smartphone out of reach during sleep hours.

Of the 720 participants who had changed their night-time smartphone behavior, more than half (54%) changed night-time smartphone behavior because they wanted to reduce their sleep problems (Fig 4). In total, 48% changed behavior because they felt that their smartphone behavior was unhealthy, and 42% changed behavior due to the increased public focus on smartphone and sleep behavior. Furthermore, 28% changed their night-time smartphone behavior because they had received more knowledge about the health consequences of poor sleep.

Discussion

In a large-scale citizen science project, we show that partnering with a public media platform greatly influenced the participation in the SmartSleep Experiment. More than 25,000 Danish adults participated within one week, and the pattern of participation corresponds with the timing of media exposure. Furthermore, we find that this mass media campaign and the citizen science approach with direct interaction between scientists and radio listeners during the SmartSleep Experiment appeared to be associated with changes in the participants’ night-time smartphone behavior. In the current study, 15% of the participants who used their smartphone during sleep hour at baseline indicated that they had changed their night-time smartphone behavior, and 83% of these indicated that they used their smartphone less during sleep hours, but the underlying mechanisms are still unsettled.

A Dutch study from 2017 evaluated the impacts of participation in two public health citizen science projects using qualitative and quantitative methods [18]. The authors suggest that the citizen science projects functioned as health promotion interventions as the citizen scientists reported changed lifestyle behavior based on the evaluations. An environmental study from 2013 explored the linkage between citizen science and the potential impacts on conservation attitudes and behavior [13]. Based on an evaluation of two citizen science projects concerning environmental conservation, the study found that participation in citizen science projects had influenced the participants’ conservation behavior. To explain the perceived impacts of conservation attitudes and behavior, the authors proposed a theoretical model. They argue that participants in citizen science projects may raise awareness of their own behavior leading to changes in attitudes and behavior [13]. Based on the proposed model, behavioral intention and willingness to change behavior are key factors in successfully changing night-time smartphone behavior.

In the current study, more than half of the participants who changed their night-time smartphone behavior were motivated by wanting to reduce their sleep problems. Sleep problems are a prevalent and increasing public health issue in adult populations [27, 28] and evidence has shown that sleep problems are related to adverse short- and long-term health effects including poor mental health, risk-taking behavior, cardiovascular diseases, diabetes, and mortality [8, 9]. Furthermore, several studies have shown that night-time smartphone use is related to poor sleep [1, 5, 6]. The massive public focus on sleep and night-time smartphone use in the mass media during the SmartSleep Experiment may have increased awareness of sleep and smartphone behavior, which may be an effective strategy to improve sleep behavior [29]. Participants who changed their night-time smartphone behavior reported that they wanted to reduce unhealthy smartphone behavior. This result indicates that increased awareness and knowledge are key factors contributing to changes in the attitudes and behavior of participants.

Night-time smartphone behavior changes may also be influenced by the individual perception of behavior control and the perceived ease or difficulty of changing the behavior. Around half of the participants who changed night-time smartphone behavior stated that they have reduced their smartphone use before falling asleep at night and during sleep hours when they would typically sleep. Moreover, around one-third of the participants with changed smartphone behavior have taken specific preventative precautions to reduce night-time smartphone use, e.g., activating silent mode or placing the smartphone out of reach. This result indicates that relatively simple precautionary behaviors may influence night-time smartphone use and contribute to smartphone behavior changes. Thus, using citizen science and mass media campaign in terms of recruiting participants and creating attention around sleep and smartphone behavior may positively impact the participants’ night-time smartphone behavior.

While the citizen science approach is promising, still 75% of those who used their smartphone during sleep hours at baseline indicated that they did not change their night-time smartphone behavior following the campaign. The massive public focus on sleep and smartphone use in the mass media was carried out for only one week, and this timeframe may be short to affect participants’ night-time smartphone behavior. Moreover, it has previously been shown that citizen science projects that are ’scientist-led’ and where the participants primarily serve as data collectors have a lower potential for behavior changes compared to projects where participants are involved in more or all aspects of the scientific processes [10, 11]. Furthermore, while short-term behavior changes may be achieved using mass media campaigns and citizen science, the long-term effects may be more difficult to maintain. It has previously been proposed that longer and more intense mass media campaign are likely to be more effective to maintain long-term behavior changes [19].

Strength and limitations

This study provides novel insights into the behavioral effects on night-time smartphone behavior after participating in a citizen science project and being exposed to a mass media campaign. Furthermore, the present study elucidates the behavioral and motivational factors that may lead to such a change in night-time smartphone behavior. The results of the present study are based on perceived changes in night-time smartphone behavior and thus, it would be interesting to explore the actual changes in night-time smartphone use including the size of the interventional effect on night-time smartphone use. However, this was not possible in the present study as the measures on night-time smartphone use were not directly comparable in the two surveys and the measure on night-time smartphone use does not allow for assessment of the size of the changes. Even though we cannot determine the direct interventional effects from participating in the SmartSleep Experiment, the present findings give important insights into the connection between citizen science, mass media, and interventional behavior impacts, which may be of value in future prevention strategies to improve sleep and smartphone behavior.

Using mass media in the recruitment of participants have great potential, as more than 25,000 adults participated in the study within one week. All participants in the SmartSleep Experiment were exposed to at least parts of the media campaign, as this was the platform from which they were recruited. Unfortunately, we do not have specific information on how many elements of the campaign each person were exposed to, or how much of the information they recalled after the two week follow-up period. Furthermore, this recruitment strategy may also create selection problems as the selection mechanisms into the study are not transparent, and the participants are likely not representative of the Danish adult population. Moreover, participants who agreed to participate in the follow-up study may be more likely to change their smartphone behavior compared to those who did not want to participate, which may have resulted in a slight overestimation of the effect of the intervention. Furthermore, the direct personal feedback after completing the survey varied across age groups, and this may have influenced age-specific findings in likelihood of changing night-time smartphone use. According to the S1 Table, the participants in the follow-up study were older and had a higher education than those who did not participate in the follow-up study. For future studies, it would be interesting to explore whether using mass media campaigns to promote behavior changes would be more/less effective in younger populations. Moreover, all study participants have been exposed to the extensive mass media campaign about smartphone and sleep behavior on DR. Thus, these limitations make it difficult to investigate the direct effects of using citizen science and mass media.

Conclusion

This study shows that massive media attention and the citizen science approach with direct interaction between scientists and radio listeners is associated with subsequent changes in night-time smartphone behaviors. Increased knowledge and awareness seemed to be the key motivational drivers. Public health projects may benefit from exploring the citizen science approach in combination with other interventional approaches.

Supporting information

S1 Fig. An example of the direct personal feedback after answering the baseline questionnaire.

(TIFF)

Click here for additional data file.^{(103KB, tiff)}

S2 Fig. Flowchart of the study population.

(TIFF)

Click here for additional data file.^{(27.3KB, tiff)}

S1 Table. Characteristics of individuals who participated in the follow-up study and individuals who did not participate in the follow-up study.

(PDF)

Click here for additional data file.^{(286KB, pdf)}

S2 Table. Baseline night-time smartphone use and socio-demograhic factors.

(PDF)

Click here for additional data file.^{(283.5KB, pdf)}

Data Availability

The data set contains personally identifiable and sensitive survey data information. We are therefore not allowed to make them publicly available according to the Danish Protection Agency (Danish data protection legislation (datatilsynet.dk)) and Danish law. Inquiries for secure data access under conditions stipulated by the Danish Data Protection Agency should be directed at the data manager of the SmartSleep project (liks@sund.ku.dk) or principle investigator of the SmartSleep project Professor Naja Hulvej Rod (nahuro@sund.ku.dk).

Funding Statement

The project was funded by the Independent Research Fund Denmark (grant number 7025-00005B). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

1.Rod NH., Dissing AS., Clark A., Gerds TA., & Lund R. Overnight smartphone use: A new public health challenge? A novel study design based on high-resolution smartphone data. PLoS One, 2018. 13(10): p. e0204811. doi: 10.1371/journal.pone.0204811 [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Shoval D., Tal N., and Tzischinsky O., Relationship of smartphone use at night with sleep quality and psychological well-being among healthy students: A pilot study. Sleep health, 2020. doi: 10.1016/j.sleh.2020.01.011 [DOI] [PubMed] [Google Scholar]
3.Exelmans L. and Van den Bulck J., Bedtime mobile phone use and sleep in adults. Social Science & Medicine, 2016. 148: p. 93–101. doi: 10.1016/j.socscimed.2015.11.037 [DOI] [PubMed] [Google Scholar]
4.Gradisar M, Wolfson AR, Harvey AG, Hale L, Rosenberg R & Czeisler CS, The sleep and technology use of Americans: findings from the National Sleep Foundation’s 2011 Sleep in America poll. J Clin Sleep Med, 2013. 9(12): p. 1291–9. doi: 10.5664/jcsm.3272 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Christensen MA., Bettencourt L., Kaye L., Moturu ST., Nguyen KT., Olgin JE, et al., Direct Measurements of Smartphone Screen-Time: Relationships with Demographics and Sleep. PLoS One, 2016. 11(11). doi: 10.1371/journal.pone.0165331 [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Thomee S., Mobile Phone Use and Mental Health. A Review of the Research That Takes a Psychological Perspective on Exposure. Int J Environ Res Public Health, 2018. 15(12): p. 2692. doi: 10.3390/ijerph15122692 [DOI] [PMC free article] [PubMed] [Google Scholar]
7.He J.-W., Tu Z-H., Xiao L., Su T., Tang Y-X., Effect of restricting bedtime mobile phone use on sleep, arousal, mood, and working memory: A randomized pilot trial. PloS one, 2020. 15(2): p. e0228756–e0228756. doi: 10.1371/journal.pone.0228756 [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Medic G., Wille M., and Hemels M.E., Short- and long-term health consequences of sleep disruption. Nat Sci Sleep, 2017. 9: p. 151–161. doi: 10.2147/NSS.S134864 [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Itani O., Jike M., Watanabe N. & Kaneita Y., Short sleep duration and health outcomes: a systematic review, meta-analysis, and meta-regression. Sleep Med, 2017. 32: p. 246–256. doi: 10.1016/j.sleep.2016.08.006 [DOI] [PubMed] [Google Scholar]
10.Shirk J., Ballard H., Wilderman CC., Phillips T., Wiggins A., Jordan R. et al., Public Participation in Scientific Research: a Framework for Deliberate Design. Ecology and Society, 2012. 17(2). 10.5751/ES-04705-170229 [DOI] [Google Scholar]
11.Bonney R., Cooper C., Dickinson J., & Kelling S., Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. Bioscience, 2009. 59(11): p. 977–984. 10.1525/bio.2009.59.11.9 [DOI] [Google Scholar]
12.Brouwer S. and Hessels L.K., Increasing research impact with citizen science: The influence of recruitment strategies on sample diversity. Public Underst Sci, 2019. 28(5): p. 606–621. doi: 10.1177/0963662519840934 [DOI] [PubMed] [Google Scholar]
13.Toomey A.H. and Domroese M.C., Can citizen science lead to positive conservation attitudes and behaviors? Human Ecology Review, 2013. 20(1): p. 50–62. [Google Scholar]
14.Buyx A., Del Savio L., Prainsack B., & Völzke H., Every participant is a PI. Citizen science and participatory governance in population studies. International Journal of Epidemiology, 2017. 46(2): p. 377–384. doi: 10.1093/ije/dyw204 [DOI] [PubMed] [Google Scholar]
15.Fiske A., Del Savio L., Prainsack B., & Buyx A., Conceptual and Ethical Considerations for Citizen Science in Biomedicine, in Personal Health Science. 2019. p. 195–217. doi: 10.1007/978-3-658-16428-7_10 [DOI] [Google Scholar]
16.Wiggins A. and Wilbanks J., The Rise of Citizen Science in Health and Biomedical Research. Am J Bioeth, 2019. 19(8): p. 3–14. doi: 10.1080/15265161.2019.1619859 [DOI] [PubMed] [Google Scholar]
17.Den Broeder L., Devilee J., Van Oers H., Schuit A. J., & Wagemakers A, Citizen Science for public health. Health Promot Int, 2018. 33(3): p. 505–514. doi: 10.1093/heapro/daw086 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Den Broeder L., Lemmens L., Uysal S., Kauw K., Weekenborg J., Schönenberger M., et al., Public Health Citizen Science; Perceived Impacts on Citizen Scientists: A Case Study in a Low-Income Neighbourhood in the Netherlands. Citizen Science: Theory and Practice, 2017. 2(1). doi: 10.5334/cstp.89 [DOI] [Google Scholar]
19.Wakefield M.A., Loken B., and Hornik R.C., Use of mass media campaigns to change health behaviour. Lancet,2010. 376(9748): p. 1261–1271. doi: 10.1016/S0140-6736(10)60809-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Kieslinger B., Schäfer T., Heigl F., Dörler D., Richter A. & Bonn A. Evaluating citizen science–Towards an open framework, in Citizen Science: Innovation in Open Science, Society and Policy, Hecker S., Haklay M., Bowser A., Makuch Z., Vogel J., Bonn A. 2018, UCL Press, London. p. 81–96. [Google Scholar]
21.Haywood B.K., A “Sense of Place” in Public Participation in Scientific Research. Science Education, 2014. 98(1): p. 64–83. doi: 10.1002/sce.2108722 [DOI] [Google Scholar]
22.Stead M., Angus K., Langley T., Katikireddi S.V., Hinds K., et al., Mass media to communicate public health messages in six health topic areas: a systematic review and other reviews of the evidence. 2019, NIHR Journals Library, Southampton (UK). doi: 10.3310/phr07080 [DOI] [PubMed] [Google Scholar]
23.Gommerman L. and Monroe M., Lessons Learned from Evaluations of Citizen Science Programs 1. Gainesville, FL, School of Forest Resources and Conservation, Florida Cooperative Extension Service, Institute of Food and Agricultural Sciences, University of Florida, 2012. [Google Scholar]
24.Jensen H, Davidsen M, Ekholm O, Christensen AI, Kristensen PL, Gårn A, et al., Hvordan har du det?—Trivsel, sundhed og sygdom blandt voksne i Region Syddanmark 2017 [How are you? Well-being, Health, and Illness among Adults in the Region of Southern Denmark 2017]. 2018: Vejle: Region Syddanmark. [Google Scholar]
25.Schneider S., The International Standard Classification of Education 2011. Comparative Social Research, 2013. 30: p. 365–379. doi: 10.1108/S0195-6310(2013)0000030017 [DOI] [Google Scholar]
26.Kecklund G. and Åkerstedt T., The psychometric properties of the Karolinska Sleep Questionnaire. J Sleep Res, 1992. 1(Suppl 1), 113. [Google Scholar]
27.Ferrie JE., Kumari M., Salo P., Singh-Manoux A., & Kivimáki M., Sleep epidemiology—a rapidly growing field. Int J Epidemiol, 2011. 40(6): p. 1431–7. doi: 10.1093/ije/dyr203 [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Chattu V.K., Manzar M.D., Kumary S., Burman D., Spence D.W., et al., The Global Problem of Insufficient Sleep and Its Serious Public Health Implications. Healthcare (Basel, Switzerland), 2018. 7(1). doi: 10.3390/healthcare7010001 [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Filip I., Tidman M., Saheba N., Bennett H., Wick B., et al., Public health burden of sleep disorders: underreported problem. Journal of Public Health, 2017. 25(3): p. 243–248. 10.1007/s10389-016-0781-0 [DOI] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0253783.r001

Decision Letter 0

Camelia Delcea

26 Feb 2021

PONE-D-21-02819

The SmartSleep Experiment: The interventional effects of using a citizen science approach and mass media to change smartphone and sleep behaviors

PLOS ONE

Dear Dr. Andersen,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

In the revised version of the paper, besides addressing the reviewers' comments listed at the bottom of this email, please provide more details related to the results you obtained, while keeping the scientific rigor related to data presentation. Please discuss the limitations of the study and the impact of the research on the society.

Please submit your revised manuscript by Apr 12 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Camelia Delcea

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

3.We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4.We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This manuscript examines the impact of a massive public media campaign on smartphone use at night in adults. This study suggests that public health efforts can be made through the mass media that can influence behavior, at least in the short term. I provide several questions regarding the study methods and conclusions along with recommendations for improving the paper below.

Introduction

• In the introduction, there is good discussion of the use of citizen science to conduct research, however this section could benefit from a discussion of the impact of smartphone use on sleep behavior and past interventions addressing this public health problem.

• The study purports to assess intervention effects on both smartphone use at night and sleep behavior but none of the study measures assess sleep behavior (e.g., sleep duration, seep onset latency, etc.), making the second aim impossible.

Methods

• The method used for recruiting participants presents a significant limitation to drawing conclusions from the study findings. Because only those who indicated interest in participating in research following the citizen science intervention, a sample bias likely exits because people who agree to participate in the may have been more likely to have changed their behavior. It is difficult to conclude that the intervention produced the change given this sampling bias.

• Another significant limitation is the inability to determine the size of the effect of the intervention on smartphone use at night. The four response choices in the measure do not allow for assessment of the size of the intervention effect.

• The utility of the measure of smartphone use is unclear because there are no reported studies assessing the psychometrics of this measure. Relatedly, the paper could be improved by reporting reliability and validity for the current study sample.

• Regarding the question assessing “behavioral interventions”, were participants allowed to select more than one of the options (e.g., used an analog clock as alarm AND placed my phone out of reach) or were they asked to select only one option? It could be helpful to clarify this with the “motivational factors” as well.

• Is there any additional information the authors could give about the rationale for including age, gender, and education level in the survey while they do not report other possible variables that could impact sleep (e.g., sleep disorders; socioeconomic status)?

Discussion

• It is interesting how many people in the study sample endorsed less than weekly smartphone use at night time. The authors mention that this could be because there were more older individuals in the follow-up sample. I wonder how effective these kinds of campaigns could be for younger individuals. This could be an interesting point to make for future directions.

• It could also be beneficial to include a note in the discussion about the potential long-term effects of public health campaigns such as this. Have past interventions given a sense for longevity of impact?

• Can the authors provide any estimates for the uptake of this intervention? It would be helpful to know how many people heard the public health messages, how many recalled information about smartphones and sleep conveyed in the public health campaign, etc.

Reviewer #2: This study reports interesting results from a mass media education campaign aiming to promote healthy sleep behaviors; the intervention included multiple modes through which information on nighttime smart phone use and sleep behavior was delivered to a large audience through radio programming and social media, with personalized feedback for survey respondents, and encouragement to participate in an online survey. Overall, the intervention is a nice example of an innovative mass media intervention to promote sleep hygiene. However, the manuscript currently has a few weaknesses that limit the contributions of this research to the larger evidence base, as described below.

Introduction

1. A large share of the framing of this study concerns the value of citizen science, yet it remains unclear whether this intervention is an example of citizen science. This terms is often used when a large number of the public is involved in the research process, generally data collection, categorizing, or other procedures. An adequate working definition is not provided for this term, as public participation in scientific research is too vague, and, more importantly, how this is an example of citizen science is needed. As I understand this intervention, I would categorize it as a mass media or public education campaign that is evaluated using a large survey.

2. The introduction references the increasing degree to which smartphones are integrated into society, but it need to provide additional evidence/citations for statements, such as “Smartphones are frequently used around the clock.”

Methods

3. Additional justification for the intervention is needed. For example, how was the personal feedback messaging chosen, and how might the role of age influence the effectiveness? The personal feedback appears to be using a social norms approach, but these are highly dependent on the age group of participants (which would arguably be expected to influence the effectiveness of this part of the intervention).

4. What was the specific messaging in the news articles, video, and interviews, and, in particular, how did the messaging/recommendations relate to the changes in nighttime smartphone use behavior that were assessed two weeks later? It is important to understand what behavioral changes would be expected if the intervention was a success.

a. An interesting future direction, if the messaging varied in important ways from day-to-day, would be to consider whether the specific behavioral changes varied based on the day participants completed their initial survey. This would assume that participants were not exposed to the education in other days, which may be too strong of an assumption. To be clear, I am not requesting these analyses be considered in this paper, rather I am just bringing up this possibility.

5. How might “The results of the day” intervention influence responses for participant who completed the survey (i.e., is there potential for priming or norms to bias reports)? However, this is likely a minor issue for this study due to the primary reliance on follow-up data, other than weekly night-time smartphone use.

6. Can you define news articles on page 126? I am wondering if these are just social media posts.

7. Typo on line 162, which should probably read as “More often”

8. Typo on line 178 for ‘classification.’ Also, can you define the ISCED categories?

9. Additional information is needed about baseline night-time smartphone use. For example, how was night-time defined for participants, and what is meant by weekly vs. less than weekly? So, a single time during the week counts as weekly?

10. Figure 1 is unhelpful. If you are to retain, then you need to restructure and provide additional information about the intervention components and potentially when they were delivered during the intervention week.

Results

11. The description of the education finding is that “More participants with high education did not change their smartphone habits compared to participants with low or medium education.” It looks like the finding is that more low or medium education participants tried to change habits but were unsuccessful relative to high education; this should be clarified.

12. It appears that baseline night-time use is by far the strongest predictor of who changes their habits, and so it would be useful to report tests of socio-demographic differences where you simultaneously adjust for these variables (including baseline night-time use). Maybe this explains the reason for the lower success by education categories.

13. The term ‘the sleep period’ is used in reference to the baseline data collection and the two-week follow-up, and so it is unclear what is meant.

14. The analysis plan is very simple, with simple calculations of percentages. This does not lend itself to drawing conclusions about differences between the various groups or the most common behavioral and motivational factors for changes.

Discussion

15. Limitations are mentioned, such as the use of self-report and non-representativeness of the sample. Given the use of a non-randomized trial and clear messaging around the desired/recommended smartphone behavior (increasing social desirability bias), and the lack of statistical tests, the authors need to be more cautious about drawing causal claims (i.e., had direct interventional effects).

16. On line 294, it is unclear what is meant by before and/or during the sleep period.

17. There is no reference to any other sleep-related literature in the Discussion. The authors need to do a more comprehensive literature search so that they can integrate their findings into the broader literature.

Reviewer #3: The current study investigated the SmartSleep Experiment targeting nighttime smart phone behaviors and use, a public health concern, in a sample of nearly 9000 Danish adolescents aged 16 years and above. The authors report that 9% of the study participants changed their nighttime smart phone habits as a result of the intervention; 78% of these indicated that they continue to do so at follow-up. Behavioral changes included activating silent mode, or reducing use before and during sleep.

The introduction is straightforward and focuses on implementing what the authors term public health citizen science projects to test an intervention called Smart sleep experiment among Danish adults. The authors highlight how smart phone use has become a public health concern as children as well as adults frequently use smart phones around the clock thus impacting the quality and duration of sleep. Thus the advocate for a citizen science project which addresses a public health concern on a large scale in a population of interest.

The methods are straightforward as well then describe data collection procedures, as well as human subjects protections considerations. The intervention consisted of participants receiving immediate feedback about nighttime smart phone use thus raising awareness among study participants with the goal or intention of reducing the same to promote higher quality and longer duration sleep. The study was accompanied by a public media campaign that included principally radio programs, the websites of the same as well as social media processes associated with radio stations based on the description provided. Following the initial assessment and feedback, a two-week follow-up survey was completed to ascertain the extent to which study participants made a change to their sleep hygiene.

The sample originally included over 25,000 Danish adults, half of which indicated that they would complete a follow up with an actual response rate of about 8900 participants. Data analyses focused principally on chi-square tests to examine changes in the distribution of variables as well as ANOVA was for continuous variables.

Study findings provided evidence of important differences between individuals who changed habits, try to change habits, or have not changed any sleep habits by sex, by educational level, as well as the extent to which smart phones were used at baseline, and by residential status, living alone versus not living alone. As noted, a bit over 800 participants are 9% of the sample change their nighttime smart phone habits following the intervention. The majority of participants made these changes because they wanted to improve on their sleep hygiene, reducing sleep problems and increasing positive sleep. Participants used different strategies to achieve changes that are detailed in the manuscript. The authors also tapped into understanding the underlying motivational factors as they labeled them, which among others included wanting to reduce sleep problems, wanting to reduce unhealthy smart phone habits, increase knowledge on health consequences of poor sleep, and discussion of smart phone habits with friends.

In conclusion, the current large-scale citizen science project, which partnered with a radio-based media campaign, that implemented raising awareness about how smart phone use interfered with proper sleep hygiene provided some promising preliminary evidence. The mechanism of change underlying the observed behavioral changes among study participants were largely related to raising awareness as well as novel insights as to how smart phone use might impact poor sleep as well as reduced sleep duration. Some inherent threats to the current study design includes the fact that over half of study participants were not willing to provide follow-up data and of those approximately 70% who agreed to do so provided data. There is no way of knowing the extent to which the current study findings are idiosyncratic and not representative simply because of this issue. This does not profoundly change the promise of the current effort but needs to be addressed adequately in the manuscript including perhaps some additional follow-up analyses of individuals who agreed to provide follow-up data initially versus ones who did not as well as between the group of individuals who agreed at baseline and then actually provided data versus the ones who did not. Doing so simply will instill greater confidence and study findings.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 Jul 21;16(7):e0253783. doi: 10.1371/journal.pone.0253783.r002

Author response to Decision Letter 0

15 Apr 2021

Editor’s comments

Please provide more details related to the results you obtained, while keeping the scientific rigor related to data presentation. Please discuss the limitations of the study and the impact of the research on the society.

Response: We have followed al of the recommendations from the editor and reviewers to the best of our abilities. Please see details below.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming.

Response: We have adjusted the format including the file naming accordingly.

Response: We have included additional details regarding participant consent in the method section, lines 101-102:

Written informed consent was obtained from all participants. The participants were informed about the purpose of the research and their rights to withdraw.

3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

The data set contains personally identifiable and sensitive survey data information. We are therefore not allowed to make them publicity available according to the Danish Protection Agency (Danish data protection legislation (datatilsynet.dk)) and Danish law. Inquiries for secure data access under conditions stipulated by the Danish Data Protection Agency should be directed at the data manager of the SmartSleep project (liks@sund.ku.dk) or principle investigator of the SmartSleep project Professor Naja Hulvej Rod (nahuro@sund.ku.dk)

4.We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

Response: We have updated the Funding Information and Financial Disclosure so the sections match.

Reviewer 1

This manuscript examines the impact of a massive public media campaign on smartphone use at night in adults. This study suggests that public health efforts can be made through the mass media that can influence behavior, at least in the short term. I provide several questions regarding the study methods and conclusions along with recommendations for improving the paper below.

Response: Thank you for your constructive feedback. Please see our detailed response below.

Reviewer 1, Question 1

In the introduction, there is good discussion of the use of citizen science to conduct research, however this section could benefit from a discussion of the impact of smartphone use on sleep behavior and past interventions addressing this public health problem.

Response: Thank you for highlighting this. Only few previous interventions have directly addressed the impact of night-time smartphone use on sleep. However, updating our literature search we identified a recent randomized trial, which we have added to the discussion about the effects of night-time smartphone use on sleep, Introduction section lines 47-49:

Furthermore, a recent randomized trial with 38 college students found that restricting mobile phone use before bedtime reduced sleep latency and increased sleep duration [7].

Reviewer 1, Question 2

The study purports to assess intervention effects on both smartphone use at night and sleep behavior but none of the study measures assess sleep behavior (e.g., sleep duration, seep onset latency, etc.), making the second aim impossible.

Response: We agree, but we unfortunately have no information on sleep behavior in the follow-up questionnaire, and we can therefore not address whether the massive media campaign directly affected the participants’ sleep behavior. Thus, the aim of the paper is to assess whether the massive public campaign have an impact on the participants’ night-time smartphone use. We have tried to make this more clear in the introduction section:

Lines 51-53:

Thus, there is a need for potential public health prevention strategies to change night-time smartphone behavior to eventually improve sleep patterns and health.

Lines 73-74:

During one week of data collection, DR created a national mass media campaign focusing on night-time smartphone behaviors and sleep…

Lines 80-81:

2) examine behavioral and motivational factors that may have influenced their smartphone behavior changes…

Reviewer 1, Question 3

The method used for recruiting participants presents a significant limitation to drawing conclusions from the study findings. Because only those who indicated interest in participating in research following the citizen science intervention, a sample bias likely exits because people who agree to participate in the may have been more likely to have changed their behavior. It is difficult to conclude that the intervention produced the change given this sampling bias.

Response: We acknowledge that there is a risk of sampling bias in the follow-up study due to the recruitment strategy. We have discussed this limitation in the discussion section lines 372-375:

Moreover, participants who agreed to participate in the follow-up study may be more likely to change their smartphone behavior compared to those who did not want to participate in future studies, which may have resulted in a slight over estimation of the effect of the intervention.

Reviewer 1, Question 4

Another significant limitation is the inability to determine the size of the effect of the intervention on smartphone use at night. The four response choices in the measure do not allow for assessment of the size of the intervention effect.

Response: We agree that it is a limitation and a potential for future research. We have added a sentence to the discussion section lines 357-358:

… it would be interesting to explore the actual changes in night-time smartphone use including the size of the interventional effect on night-time smartphone use.

Reviewer 1, Question 5

The utility of the measure of smartphone use is unclear because there are no reported studies assessing the psychometrics of this measure. Relatedly, the paper could be improved by reporting reliability and validity for the current study sample.

Response: Measuring smartphone behavior is a major challenge as digital technology and smartphone use is rapidly changing, and the measures in previous studies quickly becomes outdated. Thus, we found it necessary to develop our own measure. We generally agree that it is beneficial to validate new measures, but given it is a single-item count measure and not a scale with multiple items, we find this less pressing/doable. In future work, it would be interesting to validate the self-reported night-time smartphone use with objective measures using sensor-driven smartphone tracking data.

Reviewer 1, Question 6

Regarding the question assessing “behavioral interventions”, were participants allowed to select more than one of the options (e.g., used an analog clock as alarm AND placed my phone out of reach) or were they asked to select only one option? It could be helpful to clarify this with the “motivational factors” as well.

Response: Thank you for this comment. The participants were allowed to select more than one of the response options in both behavioral factors and motivational factors. We have clarified this in the material section line 183 and line 189.

The participants could select multiple response options.

Reviewer 1, Question 7

Is there any additional information the authors could give about the rationale for including age, gender, and education level in the survey while they do not report other possible variables that could impact sleep (e.g., sleep disorders; socioeconomic status)?

Response: Thank you for these relevant suggestions. We have added education and occupational status (as measures of socio-economic status) and sleep quality to Table 1, as suggested. We unfortunately did not have information on diagnosed sleep disorders.

Please see the updated Table 1 at the end of the rebuttal.

Reviewer 1, Question 8

It is interesting how many people in the study sample endorsed less than weekly smartphone use at night time. The authors mention that this could be because there were more older individuals in the follow-up sample. I wonder how effective these kinds of campaigns could be for younger individuals. This could be an interesting point to make for future directions.

Response: We agree that it would be interesting to investigate whether a mass media campaign like the SmartSleep Experiment will be more effective in terms of changing health behavior among younger individuals. Please see the Discussion section line 377-379:

For future studies, it would be interesting to explore whether using mass media campaigns to improve behavior changes would be more/less effective in younger populations.

Reviewer 1, Question 9

It could also be beneficial to include a note in the discussion about the potential long-term effects of public health campaigns such as this. Have past interventions given a sense for longevity of impact?

Response: It is unfortunately difficult to estimate the long-term effects of the campaign, but a review investigating the health behavior effects from mass media campaign proposes that longer and more intensive campaigns may be more effective in changing long-term health behavior. We have included a note about this in the Discussion lines 348-351:

Furthermore, while short-term behavior changes may be achieved using mass media campaigns and citizen science, the long-term effects may be more difficult to maintain. It has previously been proposed that longer and more intense mass media campaign are likely to be more effective to maintain long-term behavior changes [19].

Reviewer 1, Question 10

Can the authors provide any estimates for the uptake of this intervention? It would be helpful to know how many people heard the public health messages, how many recalled information about smartphones and sleep conveyed in the public health campaign, etc.

Response: We have information on how many unique listeners listened to the radio channels (Good morning P3 and P4) during the week of the SmartSleep Experiment, which is in the material section. As there were a lot of interviews, teasers, discussions etc. about smartphone use and sleep during the mass media campaign on these radio channels, this may be a good estimate of how many people have heard about the SmartSleep Experiment. However, we do not have information on how many listeners actually recalled the information from the public health campaign. We added this limitation in our Discussion line 366-370:

All participants in the SmartSleep Experiment were exposed to at least parts of the media campaign, as this was the platform from which they were recruited. Unfortunately, we do not have specific information on how many elements of the campaign each person were exposed to, or how much of the information they recalled after the two week follow-up period.

Reviewer #2:

This study reports interesting results from a mass media education campaign aiming to promote healthy sleep behaviors; the intervention included multiple modes through which information on nighttime smart phone use and sleep behavior was delivered to a large audience through radio programming and social media, with personalized feedback for survey respondents, and encouragement to participate in an online survey. Overall, the intervention is a nice example of an innovative mass media intervention to promote sleep hygiene. However, the manuscript currently has a few weaknesses that limit the contributions of this research to the larger evidence base, as described below.

Response: Thank you for your constructive feedback and criticism. We have tried to address each of your specific comments below.

Reviewer 2, Question 1

A large share of the framing of this study concerns the value of citizen science, yet it remains unclear whether this intervention is an example of citizen science. This terms is often used when a large number of the public is involved in the research process, generally data collection, categorizing, or other procedures. An adequate working definition is not provided for this term, as public participation in scientific research is too vague, and, more importantly, how this is an example of citizen science is needed. As I understand this intervention, I would categorize it as a mass media or public education campaign that is evaluated using a large survey.

Response: Thank you for raising this discussion. The participants in this study were directly involved in the data collection, wherefore this study is an example of a citizen science study and not only a mass media campaign. There are several ways to involve the public in the research process and the extent of involvement may differ in citizen science project [10]. We acknowledge that participants in this study may not be highly involved in the project; however, we will still categorize it as citizen science as participants were an active part of the data collection.

We agree that public participation in scientific research may be too vague for a definition of citizen science and we have added a clearer definition of citizen science in our Introduction lines 54-55:

Citizen science, defined as public participation in scientific research in which the public are engaged directly in one or more of the research processes [10,11]…

Furthermore, we have added a sentence on how the participants were actively involved in the data collection in the method section lines 88-91:

The participants were actively involved in the data collection by filling out a survey, which was an integrated part of the media campaign. This was combined with direct individual feedback to the participants and real-time presentation of preliminary results from the data collection to all radio listeners during the week of the campaign.

Reviewer 2, Question 2

The introduction references the increasing degree to which smartphones are integrated into society, but it need to provide additional evidence/citations for statements, such as “Smartphones are frequently used around the clock.”

Response: We have added additional references to support the statement.

Reviewer 2, Question 3

Additional justification for the intervention is needed. For example, how was the personal feedback messaging chosen, and how might the role of age influence the effectiveness? The personal feedback appears to be using a social norms approach, but these are highly dependent on the age group of participants (which would arguably be expected to influence the effectiveness of this part of the intervention).

Response: We acknowledge that there may be several ways of giving personal feedback, which may influence the effectiveness of the intervention. For this specific project, the personal feedback was merely considered a feature to motivate people to participate in the survey, and it only played a smaller role in the overall media campaign. While we acknowledge that there is a whole literature on different approaches to personal feedback interventions, focusing on this element was not the purpose of the current study. We have clarified the minor motivational role of the personal feedback in the current study to prevent confusion, in the method section, lines 105-106:

To motivate people to participate in the survey, they received immediate personal feedback on their night-time smartphone use after completing the survey.

Reviewer 2, Question 4

What was the specific messaging in the news articles, video, and interviews, and, in particular, how did the messaging/recommendations relate to the changes in nighttime smartphone use behavior that were assessed two weeks later? It is important to understand what behavioral changes would be expected if the intervention was a success.

Response: The messaging in the news and feature articles, video, and interviews were generally focused on the importance of sleep for health and well-being and on how night-time smartphone use could lead to disturbed sleep. We were unfortunately not able to identify the individual effects of each of these elements, and the presented results should be seen as evaluation of the full media campaign. We have highlighted this in the method section lines 136-139:

More than 20 news and feature articles focusing on the importance of sleep for health and well-being and on how night-time smartphone use may result in disturbed sleep were published during the week of the experiment on DR's website and their social media pages, including Twitter, Facebook, and Instagram.

Reviewer 2, Question 5

An interesting future direction, if the messaging varied in important ways from day-to-day, would be to consider whether the specific behavioral changes varied based on the day participants completed their initial survey. This would assume that participants were not exposed to the education in other days, which may be too strong of an assumption. To be clear, I am not requesting these analyses be considered in this paper, rather I am just bringing up this possibility.

Response: This is an interesting suggestion, but the theme and the interviews did not vary from day to day. Also, the majority of the listeners would probably tune in on multiple days as this a very popular morning show, and it would be difficult to tease out the individual effects of the elements presented one day as opposed to another.

Reviewer 2, Question 6

How might “The results of the day” intervention influence responses for participant who completed the survey (i.e., is there potential for priming or norms to bias reports)? However, this is likely a minor issue for this study due to the primary reliance on follow-up data, other than weekly night-time smartphone use.

Response: ‘The results of the day’ were actually used to improve representation of the study population e.g. gender and age distributions and geographical representation. This means that we encourage people who were less represented in the survey to participate e.g. male, specific age groups, specific educational levels etc.

Reviewer 2, Question 7

Can you define news articles on page 126? I am wondering if these are just social media posts.

The articles referring to were news articles and feature articles at DR’s webpage (www.dr.dk) and not just social media posts. The articles included e.g. results from previous research on sleep disorders or how the blue light from smartphones may affect sleep. We have clarified this in the method section lines 136-139:

More than 20 news and feature articles focusing on how night-time disturbances, including night-time smartphone use, could lead to disturbed sleep and the importance of sleep for your health were published during the week of the experiment on DR’s website and their social media pages, including Twitter, Facebook, and Instagram.

Reviewer 2, Question 8

Typo on line 162, which should probably read as “More often”

Response: Thank you for noticing. It has been corrected.

Reviewer 2, Question 9

Typo on line 178 for ‘classification.’ Also, can you define the ISCED categories?

Response: Thank you for noticing the typo. We have added more information on the categorization of educational level including a reference to the ISCED 2011 in the method section lines 190-192:

educational level (low (primary school); medium (upper secondary school; technical vocational education); high education (short, medium, and long cycle higher education); and other based on the International Standard Classification of Education 2011 [25]

Reviewer 2, Question 10

Additional information is needed about baseline night-time smartphone use. For example, how was night-time defined for participants, and what is meant by weekly vs. less than weekly? So, a single time during the week counts as weekly?

Response: We define night-time smartphone use as any short-term and/or long-term activation of the smartphone after falling asleep within the past three month with four response options ranging from every night or almost every night to never. We have added additional information about the definition of night-time smartphone use in the manuscript. For descriptive purposes, we have categorized baseline night-time smartphone use into two categories: weekly (every night or almost every night and a few nights a week) and less than weekly (a few nights a month or less and never).

Method section lines 195-199:

Baseline night-time smartphone use was assessed by asking how often the smartphone was used after falling asleep within the past three months with the following response options: every night or almost every night; a few nights a week;, a few nights a month or less; and never. Smartphone use was defined in the baseline survey and referred to both short and long activation of the smartphone.

Reviewer 2, Question 11

Figure 1 is unhelpful. If you are to retain, then you need to restructure and provide additional information about the intervention components and potentially when they were delivered during the intervention week.

Response: We have merged the information from Figure 1 and 2 into the new figure

Please see the updated figure at the end of the rebuttal.

Reviewer 2, Question 12

The description of the education finding is that “More participants with high education did not change their smartphone habits compared to participants with low or medium education.” It looks like the finding is that more low or medium education participants tried to change habits but were unsuccessful relative to high education; this should be clarified.

Response: We agree and we have added a sentence for clarification in the result section lines 245-246:

Additionally, more participants with low or medium education tried to change their smartphone habits, but were unsuccessful compared to participants with high education.

Reviewer 2, Question 13

It appears that baseline night-time use is by far the strongest predictor of who changes their habits, and so it would be useful to report tests of socio-demographic differences where you simultaneously adjust for these variables (including baseline night-time use). Maybe this explains the reason for the lower success by education categories.

Response: We have compared night-time smartphone use across groups with different educational level and occupational status in Supporting Material table 2. We have briefly commented on these differences in the revised manuscript lines 252-255:

S2 Table shows the socio-demographic differences in baseline night-time smartphone use. It appears that participants with high education and employed people are less likely to use their smartphones during sleep hours several times a week or more, while students are more likely to use their smartphones during sleep hours.

Reviewer 2, Question 14

The term ‘the sleep period’ is used in reference to the baseline data collection and the two-week follow-up, and so it is unclear what is meant.

Response: The sleep period refers to sleep hours when the participants would typically sleep. We have modified the description of the term throughout the manuscript.

Reviewer 2, Question 15

The analysis plan is very simple, with simple calculations of percentages. This does not lend itself to drawing conclusions about differences between the various groups or the most common behavioral and motivational factors for changes

Response: While we appreciate the value of descriptive analysis in a relatively unexplored field, we agree and have modified our conclusion to be less conclusive.

Abstract lines 37-38:

Using citizen science and mass media appeared to have some interventional impacts on night-time smartphone behavior.

Discussion section lines 296-298:

Furthermore, we find that this mass media campaign and the citizen science approach with direct interaction between scientists and radio listeners during the SmartSleep Experiment appeared to have some interventional effects on the participants' night-time smartphone behavior.

Line 383-386: This study shows that massive media attention and the citizen science approach with direct interaction between scientists and radio listeners may have had some interventional effects on night-time smartphone behaviors. Increased knowledge and awareness seemed to be the key motivational drivers.

Reviewer 2, Question 16

Limitations are mentioned, such as the use of self-report and non-representativeness of the sample. Given the use of a non-randomized trial and clear messaging around the desired/recommended smartphone behavior (increasing social desirability bias), and the lack of statistical tests, the authors need to be more cautious about drawing causal claims (i.e., had direct interventional effects).

Response: We agree, and we have modified our conclusion to be less conclusive.

Abstract lines 37-38:

Using citizen science and mass media may appear to have some interventional impacts on night-time smartphone behavior.

Discussion lines 296-298:

Furthermore, we find that this mass media campaign and the citizen science approach with direct interaction between scientists and radio listeners during the SmartSleep Experiment may appear to have some interventional effects on the participants' night-time smartphone behavior.

Reviewer 2, Question 17

On line 294, it is unclear what is meant by before and/or during the sleep period.

Response: We have added more information to clarify.

Lines 327-330: Around half of the participants who changed night-time smartphone behavior stated that they have reduced their smartphone use before falling asleep at night and during sleep hours when they would typically sleep.

Reviewer 2, Question 18

There is no reference to any other sleep-related literature in the Discussion. The authors need to do a more comprehensive literature search so that they can integrate their findings into the broader literature.

Response: We have included additional sleep-related literature in the discussion section lines 315-322:

Sleep problems are a prevalent and increasing public health issue in adult populations [27, 28] and evidence has shown that sleep problems are related to adverse short- and long-term health effects including poor mental health, risk-taking behavior, cardiovascular diseases, diabetes, and mortality [8, 9]. Furthermore, several studies have shown that night-time smartphone use is related to poor sleep [1, 5, 6]. The massive public focus on sleep and night-time smartphone use in the mass media during the SmartSleep Experiment may have increased awareness of sleep and smartphone habits, which may be an effective strategy to improve sleep behavior [29].

Reviewer #3:

The current study investigated the SmartSleep Experiment targeting nighttime smart phone behaviors and use, a public health concern, in a sample of nearly 9000 Danish adolescents aged 16 years and above. The authors report that 9% of the study participants changed their nighttime smart phone habits as a result of the intervention; 78% of these indicated that they continue to do so at follow-up. Behavioral changes included activating silent mode, or reducing use before and during sleep.

In conclusion, the current large-scale citizen science project, which partnered with a radio-based media campaign, that implemented raising awareness about how smart phone use interfered with proper sleep hygiene provided some promising preliminary evidence. The mechanism of change underlying the observed behavioral changes among study participants were largely related to raising awareness as well as novel insights as to how smart phone use might impact poor sleep as well as reduced sleep duration. Some inherent threats to the current study design includes the fact that over half of study participants were not willing to provide follow-up data and of those approximately 70% who agreed to do so provided data. There is no way of knowing the extent to which the current study findings are idiosyncratic and not representative simply because of this issue. This does not profoundly change the promise of the current effort, but needs to be addressed adequately in the manuscript including perhaps some additional follow-up analyses of individuals who agreed to provide follow-up data initially versus ones who did not as well as between the group of individuals who agreed at baseline and then actually provided data versus the ones who did not. Doing so simply will instill greater confidence and study findings

Response: Thank you for your helpful comments and feedback. We have added some additional analyses to investigate the representativeness of the study population in Supplemental information Table 1 in the method section lines 160-165:

S1 Table shows characteristics of individuals who at baseline did not agree to participate in future studies (n=12,787), individuals who agreed at baseline but did not participate in the follow-up study (n=3,437), and individuals who participated in the follow-up study (n=8,911). Those who participated in the follow-up study were generally older, had higher educational level, and were slightly less likely to use their smartphone during sleep hours than those who did not participate.

References:

[1] Rod, N.H., et al., Overnight smartphone use: A new public health challenge? A novel study design based on high-resolution smartphone data. PLoS One, 2018. 13(10): p. e0204811

[5] Christensen, M.A., et al., Direct Measurements of Smartphone Screen-Time: Relationships with Demographics and Sleep. PLoS One, 2016. 11(11).

[6] Thomee, S., Mobile Phone Use and Mental Health. A Review of the Research That Takes a Psychological Perspective on Exposure. Int J Environ Res Public Health, 2018. 15(12): p. 2692.

[7] He, J.-W., et al., Effect of restricting bedtime mobile phone use on sleep, arousal, mood, and working memory: A randomized pilot trial. PloS one, 2020. 15(2): p. e0228756-e0228756.

[8] Medic, G., M. Wille, and M.E. Hemels, Short- and long-term health consequences of sleep disruption. Nat Sci Sleep, 2017. 9: p. 151-161.

[9] Itani, O., et al., Short sleep duration and health outcomes: a systematic review, meta-analysis, and meta-regression. Sleep Med, 2017. 32: p. 246-256.

[10] Shirk, J., et al., Public Participation in Scientific Research: a Framework for Deliberate Design. Ecology and Society, 2012. 17(2).

[11] Bonney, R., et al., Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. Bioscience, 2009. 59(11): p. 977-984.

[19] Wakefield, M.A., B. Loken, and R.C. Hornik, Use of mass media campaigns to change health behaviour. Lancet (London, England), 2010. 376(9748): p. 1261-1271.

[25] Schneider, S., The International Standard Classification of Education 2011. Comparative Social Research, 2013. 30: p. 365-379.

[27] Ferrie, J.E., et al., Sleep epidemiology--a rapidly growing field. Int J Epidemiol, 2011. 40(6): p. 1431-7.

[28] Chattu, V.K., et al., The Global Problem of Insufficient Sleep and Its Serious Public Health Implications. Healthcare (Basel, Switzerland), 2018. 7(1).

[29] Filip, I., et al., Public health burden of sleep disorders: underreported problem. Journal of Public Health, 2017. 25(3): p. 243-248.

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(196.8KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0253783.r003

Decision Letter 1

Camelia Delcea

24 May 2021

PONE-D-21-02819R1

The SmartSleep Experiment: The interventional effects of using a citizen science approach and mass media to change smartphone and sleep behaviors

PLOS ONE

Dear Dr. Andersen,

In the revised version, please consider the comments made by Reviewer #2 listed below.

Please submit your revised manuscript by Jul 08 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Camelia Delcea

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: (No Response)

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: No

Reviewer #2: (No Response)

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: Yes

Reviewer #2: (No Response)

Reviewer #3: Yes

**********

6. Review Comments to the Author

Reviewer #1: (No Response)

Reviewer #2: I appreciate the revisions the authors made to improve the clarity of the methods and address study limitations.

For the most part, the authors adequately responded to my comments. The few exceptions are as follows:

1. in my opinion, the most substantial limitation is that there was no risk-adjustment made based on baseline nightly smartphone use when reporting results of the intervention and demographic differences. For instance, 40% of participants who regularly use their smartphone at night changed or attempted to change habits relative to 11% of respondents who used their smartphone at night several nights a month or less. These data suggest baseline nightly smartphone use is a large predictor of whether or not people attempt change. The second paragraph of the results now mentions differences in the outcome as well as sociodemographic characteristics by baseline night-time smartphone use; this is very helpful. However, it remains unknown the extent to which differences in who is changing or attempting to change nightly smartphone habits are merely an artifact of baseline use. This is a problem particularly if the interpretation suggests differences in the effectiveness of the intervention by demographic differences, but these likely partly if not completely stem from baseline risks (e.g., do younger folks attempt/successfully change habits only because they are much more likely to initially have problems with nightly use?). Another example where this comes up is how the data are generally interpreted (e.g., "still 85% of the study population indicated that they had not changed night-time smartphone behavior), which suggests that lack of change is a failure of the intervention. Some might not have anywhere to change to in that they reported no use before. It is thus essential that some risk-adjustment is made and that the data are interpreted based on who has potential to change due to baseline problems.

2. causal language is still used throughout including in the title ("interventional effects") despite this study being descriptive and susceptible to bias from sampling and self-reported measures. The descriptive nature of this study should be explicit.

3. there is no recognition of treatment heterogeneity based on age. Whether or not the personal feedback was included strictly as a motivational tool, the feedback was highly age-dependent and specific to use of smartphone at night and therefore may have influenced age-specific findings in likelihood of changing nighttime smartphone use. This should at the very least be briefly mentioned as a limitation or future direction when interpreting age-specific findings.

Also, the language is generally clear. But, there are some grammatical errors throughout. The results could also be edited for clarity, particularly the first two paragraphs.

Reviewer #3: The authors have been fairly responsive to the reviewer feedback, no additional comments at this time

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Alexander T. Vazsonyi

PLoS One. 2021 Jul 21;16(7):e0253783. doi: 10.1371/journal.pone.0253783.r004

Author response to Decision Letter 1

7 Jun 2021

Editor’s comment:

Response: We have made some corrections to the reference list, which should now be complete and correct.

Reviewer #2

I appreciate the revisions the authors made to improve the clarity of the methods and address study limitations. For the most part, the authors adequately responded to my comments. The few exceptions are as follows:

1. In my opinion, the most substantial limitation is that there was no risk-adjustment made based on baseline nightly smartphone use when reporting results of the intervention and demographic differences. For instance, 40% of participants who regularly use their smartphone at night changed or attempted to change habits relative to 11% of respondents who used their smartphone at night several nights a month or less. These data suggest baseline nightly smartphone use is a large predictor of whether or not people attempt change. The second paragraph of the results now mentions differences in the outcome as well as sociodemographic characteristics by baseline night-time smartphone use; this is very helpful. However, it remains unknown the extent to which differences in who is changing or attempting to change nightly smartphone habits are merely an artifact of baseline use. This is a problem particularly if the interpretation suggests differences in the effectiveness of the intervention by demographic differences, but these likely partly if not completely stem from baseline risks (e.g., do younger folks attempt/successfully change habits only because they are much more likely to initially have problems with nightly use?). Another example where this comes up is how the data are generally interpreted (e.g., "still 85% of the study population indicated that they had not changed night-time smartphone behavior), which suggests that lack of change is a failure of the intervention. Some might not have anywhere to change to in that they reported no use before. It is thus essential that some risk-adjustment is made and that the data are interpreted based on who has potential to change due to baseline problems.

Response: This is a valid point. In order to take the baseline night-time smartphone use into account in our interpretation of the results, we have restricted all analyses to the participants who report using their smartphone during sleep at baseline (n=4,926 participants). We have updated Figure 2-4 and corrected the results throughout the paper. The overall findings are unchanged.

2. Causal language is still used throughout including in the title ("interventional effects") despite this study being descriptive and susceptible to bias from sampling and self-reported measures. The descriptive nature of this study should be explicit.

Response: We changed the title accordingly: The SmartSleep Experiment: Evaluation of changes in night-time smartphone behavior following a mass media citizen science intervention.

Furthermore, we have included a sentence about the lack of causal inference in the discussion section lines 358-362:

Even though we cannot determine the direct interventional effects from participating in the SmartSleep Experiment, the present findings give important insights into the connection between citizen science, mass media and interventional behavior impacts, which may be of value in future prevention strategies to improve sleep and smartphone habits.

3. There is no recognition of treatment heterogeneity based on age. Whether or not the personal feedback was included strictly as a motivational tool, the feedback was highly age-dependent and specific to use of smartphone at night and therefore may have influenced age-specific findings in likelihood of changing nighttime smartphone use. This should at the very least be briefly mentioned as a limitation or future direction when interpreting age-specific findings.

Response: We agree, and we have mentioned this limitation in the discussion section lines 375-377:

Furthermore, the direct personal feedback after completing the survey varied across age groups, and this may have influenced age-specific findings in likelihood of changing night-time smartphone use.

4. Also, the language is generally clear. But, there are some grammatical errors throughout. The results could also be edited for clarity, particularly the first two paragraphs.

Response: Thank you for highlighting this. We have edited the result section and corrected the grammatical errors throughout the manuscript. The manuscript has also been proof read by a professional proof reader.

Attachment

Submitted filename: Response to reviewers.docx

Click here for additional data file.^{(16.4KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0253783.r005

Decision Letter 2

Camelia Delcea

14 Jun 2021

The SmartSleep Experiment: Evaluation of changes in night-time smartphone behavior following a mass media citizen science campaign

PONE-D-21-02819R2

Dear Dr. Andersen,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Camelia Delcea

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

PLoS One. doi: 10.1371/journal.pone.0253783.r006

Acceptance letter

Camelia Delcea

24 Jun 2021

PONE-D-21-02819R2

The SmartSleep Experiment: Evaluation of changes in night-time smartphone behavior following a mass media citizen science campaign

Dear Dr. Andersen:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Camelia Delcea

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 Fig. An example of the direct personal feedback after answering the baseline questionnaire.

(TIFF)

Click here for additional data file.^{(103KB, tiff)}

S2 Fig. Flowchart of the study population.

(TIFF)

Click here for additional data file.^{(27.3KB, tiff)}

S1 Table. Characteristics of individuals who participated in the follow-up study and individuals who did not participate in the follow-up study.

(PDF)

Click here for additional data file.^{(286KB, pdf)}

S2 Table. Baseline night-time smartphone use and socio-demograhic factors.

(PDF)

Click here for additional data file.^{(283.5KB, pdf)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(196.8KB, docx)}

Attachment

Submitted filename: Response to reviewers.docx

Click here for additional data file.^{(16.4KB, docx)}

Data Availability Statement

[pone.0253783.ref001] 1.Rod NH., Dissing AS., Clark A., Gerds TA., & Lund R. Overnight smartphone use: A new public health challenge? A novel study design based on high-resolution smartphone data. PLoS One, 2018. 13(10): p. e0204811. doi: 10.1371/journal.pone.0204811 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0253783.ref002] 2.Shoval D., Tal N., and Tzischinsky O., Relationship of smartphone use at night with sleep quality and psychological well-being among healthy students: A pilot study. Sleep health, 2020. doi: 10.1016/j.sleh.2020.01.011 [DOI] [PubMed] [Google Scholar]

[pone.0253783.ref003] 3.Exelmans L. and Van den Bulck J., Bedtime mobile phone use and sleep in adults. Social Science & Medicine, 2016. 148: p. 93–101. doi: 10.1016/j.socscimed.2015.11.037 [DOI] [PubMed] [Google Scholar]

[pone.0253783.ref004] 4.Gradisar M, Wolfson AR, Harvey AG, Hale L, Rosenberg R & Czeisler CS, The sleep and technology use of Americans: findings from the National Sleep Foundation’s 2011 Sleep in America poll. J Clin Sleep Med, 2013. 9(12): p. 1291–9. doi: 10.5664/jcsm.3272 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0253783.ref005] 5.Christensen MA., Bettencourt L., Kaye L., Moturu ST., Nguyen KT., Olgin JE, et al., Direct Measurements of Smartphone Screen-Time: Relationships with Demographics and Sleep. PLoS One, 2016. 11(11). doi: 10.1371/journal.pone.0165331 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0253783.ref006] 6.Thomee S., Mobile Phone Use and Mental Health. A Review of the Research That Takes a Psychological Perspective on Exposure. Int J Environ Res Public Health, 2018. 15(12): p. 2692. doi: 10.3390/ijerph15122692 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0253783.ref007] 7.He J.-W., Tu Z-H., Xiao L., Su T., Tang Y-X., Effect of restricting bedtime mobile phone use on sleep, arousal, mood, and working memory: A randomized pilot trial. PloS one, 2020. 15(2): p. e0228756–e0228756. doi: 10.1371/journal.pone.0228756 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0253783.ref008] 8.Medic G., Wille M., and Hemels M.E., Short- and long-term health consequences of sleep disruption. Nat Sci Sleep, 2017. 9: p. 151–161. doi: 10.2147/NSS.S134864 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0253783.ref009] 9.Itani O., Jike M., Watanabe N. & Kaneita Y., Short sleep duration and health outcomes: a systematic review, meta-analysis, and meta-regression. Sleep Med, 2017. 32: p. 246–256. doi: 10.1016/j.sleep.2016.08.006 [DOI] [PubMed] [Google Scholar]

[pone.0253783.ref010] 10.Shirk J., Ballard H., Wilderman CC., Phillips T., Wiggins A., Jordan R. et al., Public Participation in Scientific Research: a Framework for Deliberate Design. Ecology and Society, 2012. 17(2). 10.5751/ES-04705-170229 [DOI] [Google Scholar]

[pone.0253783.ref011] 11.Bonney R., Cooper C., Dickinson J., & Kelling S., Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. Bioscience, 2009. 59(11): p. 977–984. 10.1525/bio.2009.59.11.9 [DOI] [Google Scholar]

[pone.0253783.ref012] 12.Brouwer S. and Hessels L.K., Increasing research impact with citizen science: The influence of recruitment strategies on sample diversity. Public Underst Sci, 2019. 28(5): p. 606–621. doi: 10.1177/0963662519840934 [DOI] [PubMed] [Google Scholar]

[pone.0253783.ref013] 13.Toomey A.H. and Domroese M.C., Can citizen science lead to positive conservation attitudes and behaviors? Human Ecology Review, 2013. 20(1): p. 50–62. [Google Scholar]

[pone.0253783.ref014] 14.Buyx A., Del Savio L., Prainsack B., & Völzke H., Every participant is a PI. Citizen science and participatory governance in population studies. International Journal of Epidemiology, 2017. 46(2): p. 377–384. doi: 10.1093/ije/dyw204 [DOI] [PubMed] [Google Scholar]

[pone.0253783.ref015] 15.Fiske A., Del Savio L., Prainsack B., & Buyx A., Conceptual and Ethical Considerations for Citizen Science in Biomedicine, in Personal Health Science. 2019. p. 195–217. doi: 10.1007/978-3-658-16428-7_10 [DOI] [Google Scholar]

[pone.0253783.ref016] 16.Wiggins A. and Wilbanks J., The Rise of Citizen Science in Health and Biomedical Research. Am J Bioeth, 2019. 19(8): p. 3–14. doi: 10.1080/15265161.2019.1619859 [DOI] [PubMed] [Google Scholar]

[pone.0253783.ref017] 17.Den Broeder L., Devilee J., Van Oers H., Schuit A. J., & Wagemakers A, Citizen Science for public health. Health Promot Int, 2018. 33(3): p. 505–514. doi: 10.1093/heapro/daw086 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0253783.ref018] 18.Den Broeder L., Lemmens L., Uysal S., Kauw K., Weekenborg J., Schönenberger M., et al., Public Health Citizen Science; Perceived Impacts on Citizen Scientists: A Case Study in a Low-Income Neighbourhood in the Netherlands. Citizen Science: Theory and Practice, 2017. 2(1). doi: 10.5334/cstp.89 [DOI] [Google Scholar]

[pone.0253783.ref019] 19.Wakefield M.A., Loken B., and Hornik R.C., Use of mass media campaigns to change health behaviour. Lancet,2010. 376(9748): p. 1261–1271. doi: 10.1016/S0140-6736(10)60809-4 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0253783.ref020] 20.Kieslinger B., Schäfer T., Heigl F., Dörler D., Richter A. & Bonn A. Evaluating citizen science–Towards an open framework, in Citizen Science: Innovation in Open Science, Society and Policy, Hecker S., Haklay M., Bowser A., Makuch Z., Vogel J., Bonn A. 2018, UCL Press, London. p. 81–96. [Google Scholar]

[pone.0253783.ref021] 21.Haywood B.K., A “Sense of Place” in Public Participation in Scientific Research. Science Education, 2014. 98(1): p. 64–83. doi: 10.1002/sce.2108722 [DOI] [Google Scholar]

[pone.0253783.ref022] 22.Stead M., Angus K., Langley T., Katikireddi S.V., Hinds K., et al., Mass media to communicate public health messages in six health topic areas: a systematic review and other reviews of the evidence. 2019, NIHR Journals Library, Southampton (UK). doi: 10.3310/phr07080 [DOI] [PubMed] [Google Scholar]

[pone.0253783.ref023] 23.Gommerman L. and Monroe M., Lessons Learned from Evaluations of Citizen Science Programs 1. Gainesville, FL, School of Forest Resources and Conservation, Florida Cooperative Extension Service, Institute of Food and Agricultural Sciences, University of Florida, 2012. [Google Scholar]

[pone.0253783.ref024] 24.Jensen H, Davidsen M, Ekholm O, Christensen AI, Kristensen PL, Gårn A, et al., Hvordan har du det?—Trivsel, sundhed og sygdom blandt voksne i Region Syddanmark 2017 [How are you? Well-being, Health, and Illness among Adults in the Region of Southern Denmark 2017]. 2018: Vejle: Region Syddanmark. [Google Scholar]

[pone.0253783.ref025] 25.Schneider S., The International Standard Classification of Education 2011. Comparative Social Research, 2013. 30: p. 365–379. doi: 10.1108/S0195-6310(2013)0000030017 [DOI] [Google Scholar]

[pone.0253783.ref026] 26.Kecklund G. and Åkerstedt T., The psychometric properties of the Karolinska Sleep Questionnaire. J Sleep Res, 1992. 1(Suppl 1), 113. [Google Scholar]

[pone.0253783.ref027] 27.Ferrie JE., Kumari M., Salo P., Singh-Manoux A., & Kivimáki M., Sleep epidemiology—a rapidly growing field. Int J Epidemiol, 2011. 40(6): p. 1431–7. doi: 10.1093/ije/dyr203 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0253783.ref028] 28.Chattu V.K., Manzar M.D., Kumary S., Burman D., Spence D.W., et al., The Global Problem of Insufficient Sleep and Its Serious Public Health Implications. Healthcare (Basel, Switzerland), 2018. 7(1). doi: 10.3390/healthcare7010001 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0253783.ref029] 29.Filip I., Tidman M., Saheba N., Bennett H., Wick B., et al., Public health burden of sleep disorders: underreported problem. Journal of Public Health, 2017. 25(3): p. 243–248. 10.1007/s10389-016-0781-0 [DOI] [Google Scholar]

PERMALINK

The SmartSleep Experiment: Evaluation of changes in night-time smartphone behavior following a mass media citizen science campaign

Thea Otte Andersen

Agnete Skovlund Dissing

Tibor V Varga

Naja Hulvej Rod

Roles

Abstract

Introduction

Material and methods

The SmartSleep Experiment

Direct personal feedback

Massive public media coverage on night-time smartphone use and sleep

Follow-up survey

Study population and data collection

Measurements

Behavioral factors

Motivational factors

Analytical strategy

Results

Distribution of participation

Fig 1. Distribution of participation during the week of the SmartSleep Experiment among 25,135 participants.

Characteristics of participants with changed smartphone behavior

Table 1. Characteristics of the study population according to changed night-time smartphone behavior among 8,894 participants in the SmartSleep Experiment.

Change in night-time smartphone use and the direction of change at follow-up among baseline night-time smartphone users

Fig 2. Change in night-time smartphone behavior according to the direction of change among 4,926 participants who used their smartphone during sleep hours at baseline.

Behavioral and motivational factors for changes in night-time smartphone behavior

Fig 3. The behavioral factors for changes in night-time smartphone behavior among 720 participants who changed behavior in the SmartSleep Experiment.

Fig 4. The motivational factors for changes in night-time smartphone behavior among 720 participants who changed behavior in the SmartSleep Experiment.

Discussion

Strength and limitations

Conclusion

Supporting information

Data Availability

Funding Statement

References

Decision Letter 0

Camelia Delcea

Roles

Author response to Decision Letter 0

Decision Letter 1

Camelia Delcea

Roles

Author response to Decision Letter 1

Decision Letter 2

Camelia Delcea

Roles

Acceptance letter

Camelia Delcea

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases