Required components of a sleep-wake online daily diary for accurate and efficient human research

Abstract

Accurate documentation of sleep timing (enabling calculation of duration and variability) is imperative for research, health, and safety recommendations. Methods in real-world settings include one-time self-report, diaries, and wearables that vary by ease, expense, and accuracy. Here we highlight how programmable online daily diaries compare to gold-standard polysomnography for measuring sleep timing, should be preferred over one-time self-report, and complement wearables for longitudinal clinical and research studies and real-world applications.

Sleep is fundamental for optimal health, wellbeing, and safety (i.e., reducing accidents and errors)¹. However, the methods used for documenting sleep timing and/or duration may affect the results obtained. For example, one-time subjective self-report, while easy and inexpensive to implement, may not accurately reflect actual sleeping behaviors due to night-to-night variability compounded by recall or other biases^2–4. A study examining cardiovascular risk found a significant association between shorter sleep duration and higher incidence of coronary artery calcification when using actigraphically-measured (i.e., wearable) multi-day sleep, but no association when sleep duration was determined from a one-time self-reported assessment⁵. Objective sleep data collected using wearable devices or polysomnography (PSG) require more expensive equipment, additional personnel time for preparation of measurement and primary data analyses, and possible modification of sleep behaviors related to the recording itself⁶. For some work, subjective information is preferred (e.g., related to insomnia symptoms or behaviors such as consuming caffeine when feeling tired), while objective measures (e.g., sleep duration) are preferred for associations with health and safety. Documenting the accuracy and relationships among different measurement techniques for collecting sleep timing and/or duration is necessary for interpreting results and determining accurate associations between sleep and other outcomes, including tracking disease progression or monitoring response to an intervention. This information is useful for others as they plan research or clinical work collecting sleep timing data. Within this Comment we will describe potential strengths and weaknesses of common ways to capture sleep timing and duration in real-world settings and provide data advocating for the use of online sleep-wake diaries.

Discrepancy in methods

Recording sleep timing for multiple days would be expected to both provide more accurate information than a one-time self-reported retrospective question and allow calculation of other important aspects of sleep (e.g., variability of sleep duration and timing). The collection of these data on paper in large populations has multiple drawbacks: diaries must be distributed and then collected; data must be entered into electronic formats for analyses (increasing staff burden and risk for transcription errors); handwritten entries may be difficult to discern; there may be errors/inconsistencies (e.g., AM/PM or date mistakes) or anomalous entries (e.g., 30-h sleep duration); information may be omitted if inadequate space is provided for multiple events (e.g., several naps in one day); paper diaries may be lost, damaged, or forgotten; the time of entries is not known and therefore an entry may be completed many days after the specific sleep episode (thereby increasing risk of recall bias); and data are not available in real-time. Some of these issues can be attenuated via usage of email or text messaging for reminders⁷, but this can still require high participant and staff burden, and data are rarely checked for errors at the time of entry on paper (when the participant is the only one with access). There is increasing use of some online diaries (e.g., REDCap); the recommended features of such a collection method are detailed below.

An alternative option is wearable-based methods (e.g., actigraphy). These devices can be expensive and must be given/sent to and recovered from participants, data can be missed due to technical challenges (e.g., battery problems, removal for charging, participants not wearing the device correctly, software updates), and they typically still require participant input from diary or an event marker for research or clinical-grade accuracy, which increases participant and research staff burden significantly⁸. There are also multiple potential problems with the algorithms used for deriving sleep-wake from wearables⁹: they may be inaccurate in determining the timing of an ‘in-bed’ episode or actual sleep¹⁰, particularly for individuals with movement or sleep disorders (e.g., insomnia)^11,12, leading to inaccurate estimates of sleep-statistics (e.g., sleep latency, sleep efficiency)¹⁰. Different devices have different hardware and algorithms, including across versions within a single manufacturer, potentially making comparisons across individuals or studies problematic. Software algorithms are not always published, can be poor at recognizing daytime naps or differentiating naps from quiet wakefulness, and may not be validated against the gold-standard of polysomnography (PSG) for determining sleep/wake state (and therefore timing). Algorithm validations based on in-laboratory PSG in healthy individuals with regular sleep patterns may overestimate the algorithm accuracy in a field (“real-world”) study of non-healthy participants (e.g., those with sleep apnea or other common sleep disorders) or those with irregular sleep patterns¹³. Devices that can upload data daily or more frequently usually must be removed, charged, sometimes reinitialized, and then put on again – all of which add burden, potential error (including not putting it on again), and there are potential privacy issues if the wearable company uses an individual’s data for other purposes. Moreover, due to the requirement of subjective input of time-in-bed from the user for research-grade accuracy, the presumed objectivity of the measure is reduced. Therefore, most currently available methods do not have the desired attributes of low levels of participant and research staff burden, low cost, high accuracy, and close-to-real time results needed for identification of sleep metrics that can be targeted or monitored to improve health, safety, and well-being. Wearables do have other advantages, including other metrics (e.g., heart rate variability, light exposure) and the ability to calculate sleep-adjacent variables (e.g., Intradaily Stability, Interdaily Variability) that have been reported to be correlated or predictive of some health conditions.

To account for these issues and increase the accuracy and feasibility in reporting sleep outcomes, we advocate for programmable daily online sleep-wake diary (eDiary) methods that reduce participant and research staff burden and accurately collect sleep timing and duration data for both small and larger population studies. Specifically, herein we (i) describe how an eDiary method can provide accurate results as compared to gold-standard PSG, (ii) estimate the staffing time required to obtain accurate outcomes, and (iii) determine how an eDiary method compares to one-time questionnaires and actigraphy (i.e., a wearable) in a large group of individuals living on their habitual schedules.

Sleep diary questions compared to gold-standard polysomnography (PSG) and actigraphy methods

We first validated sleep diaries against gold-standard PSG for sleep timing and duration. As part of a larger study¹⁴, medical interns (n = 20) during two 3-week hospital Intensive Care Unit (ICU) rotations completed a daily paper sleep diary (Fig. S1) after their major sleep episodes while being assessed simultaneously using continuous wrist-worn actigraphy (Minimitter Actiwatch L, Philips Respironics; scored with Actiware v6.0.7) and ambulatory PSG (Vitaport-2/3, TEMEC Instruments B.V., The Netherlands) during both at-home and at-work schedules for 3–4 days of each week. Data were collected as part of a study designed to document relationships between work hours and sleep in medical interns¹⁴. All study procedures were approved by the Human Research Committee of Partners Healthcare and the Brigham and Women’s Hospital [#2001P000814] and participants provided written informed consent and all procedures were done in accordance with the Declaration of Helsinki. Full methods in Supplement.

We compared the sleep onset, offset, and duration recorded using PSG versus the sleep diary and actigraphy for sleep episodes >20-minute duration in these participants. A previous preliminary comparison of the total sleep time derived from PSG correlated highly with a stated duration of sleep duration recorded in the sleep diary¹⁴; we now add replication of those findings using the differing method of calculating sleep duration using sleep onset/offset times.

Based upon retrospectively calculated effort, it took a highly motivated project manager and research staff approximately 3.5 h for contemporaneous checking, post-study corrections, and data entry for a 30-day diary for each participant. Data were collapsed into averages per participant across the study and analyzed using Pearson correlations. Potential “outliers” or large discrepancies between methods were not excluded from the analysis to demonstrate varying outcomes when measuring the same sleep episode using different methods. Between diary and PSG, we found that sleep onset (r = 0.99, p < 0.0001), offset (r = 0.99, p < 0.0001), and duration (r = 0.98, p < 0.0001) were tightly correlated between PSG and diary (all sleep episodes displayed in Fig. 1). Bland-Altman plots showed high agreement between the diary and PSG measures at the daily level, with 81% of sleep onsets within 30 min, 92% within 45 min, and 95% within a 1 h difference between diary and PSG onsets (Fig. 1). Sleep offsets and duration had similarly high agreements, with 81% of sleep offsets (72% for duration) within 30 min, 90% (86% for duration) within 45 min, and 94% (91% for duration) within a 1 h difference between diary and PSG (Fig. 1).

Fig. 1. Comparison of paper sleep diaries with the polysomnography (PSG, gold standard for sleep recordings).

Left column reports sleep onset data (n = 336 sleep episodes from n = 20 individuals); middle column reports sleep offset data (n = 334 sleep episodes from n = 20 individuals); and right column reports sleep duration data (n = 334 sleep episodes from n = 20 individuals). First row is comparison of sleep times and duration as calculated by PSG and diary for or all recorded sleep episodes(a-c), middle row data are Bland-Altman plots of the differences between PSG and diary outcomes for all recorded sleep episodes (d-f), and the bottom row is the distribution of the degree of accordance (minute difference of each measurement) between PSG and sleep diaries (g-i). The dashed line in the top row panels depict the line of unity, the median and 95% limits of agreement in the middle panels, and the cumulative distribution in the bottom panels.

The correlation of actigraphy with PSG was weaker when data were collapsed into averages per participant across the study, with sleep onset (r = 0.95, p < 0.0001), offset (r = 0.63, p = 0.01), and duration (r = 0.70, p = 0.004) still significantly correlated (Fig. 2). Bland-Altman plots showed similarly high agreement between the actigraphy and PSG measures at the daily level (e.g., 78% of sleep onsets, 74% offsets, and 66% of sleep duration within 30 min difference between actigraphy and PSG onsets) (Fig. 2).

Fig. 2. Comparison of wrist actigraphy with the polysomnography (PSG, gold standard for sleep recordings).

Left column reports sleep onset data (n = 193 sleep episodes from n = 20 individuals); middle column reports sleep offset data (n = 191 sleep episodes from n = 20 individuals); and right column reports sleep duration data (n = 189 sleep episodes from n = 20 individuals). First row is comparison of times and duration as calculated by PSG and actigraphy outcomes for all recorded sleep episodes, middle row data are Bland-Altman plots of the differences between PSG and actigraphy outcomes for all recorded sleep episodes, and the bottom row is the distribution of the degree of accordance (minute difference of each measurement) between PSG and actigraphy. The dashed line in the top row panels depict the line of unity, the median and 95% limits of agreement in the middle panels, and the cumulative distribution in the bottom panels.

Sleep diary questions in an online format

Sleep timing questions from the paper diary were then transitioned to a programmable online format (eDiary) that included both (i) real-time error checking and (ii) feedback and used in a population of individuals that were expected to have highly variable sleep/wake timing (i.e., college undergraduates) (Figs. S2–S5). Specifically, we utilized an eDiary with a visualization of the individual’s sleep schedule (Fig. 3a) to track sleep and other daily behaviors for ~30 days in a study of n = 266 college undergraduates^15,16. The protocol was approved by the Partners Healthcare (#2012P001631) and Massachusetts Institute of Technology (#1209005240) Institutional Review Boards and performed in accordance with the Declaration of Helsinki. Students were instructed to (i) complete a single survey with questions about sleep prior to the ~30-day study; and (ii) complete the eDiary prior to initiating sleep and immediately upon awakening; and (iii) wear an actigraph (Motionlogger, Ambulatory Monitoring, Ardsley, NY; scored using the Motionlogger University of California San Diego (UCSD) sleep scoring algorithm¹⁷) continuously during the ~30-day study. Links to the eDiary were emailed to students twice daily, once in the morning and once in the evening. Data input by the student were illustrated and shown to the student (in raster plot format) so that errors were immediately observable (e.g., AM/PM mistakes, activity overlaps). In this way, the user completed the first pass of data cleaning.

Fig. 3. Outputs of an online sleep-wake diary that reduces research staff and participant burden.

The online diary (eDiary) provides participants (a) a raster plot of their daily entries. In a raster plot, time of day is along the x-axis and successive days are along the y-axis. The plotting allows for quick confirmation of accurate timing and non-overlap of events. Examples of events that can be identified by the user are shown in different colors on the raster plot, which in this case were designed for a college student (i.e., overnight sleep, naps, academic events, exercise, and extracurricular activities), but could be other events for other populations. b Sleep timing and duration calculated from eDiary data differed from data collected using traditional methods of sleep measurement; (c) differences relative to eDiary sleep duration; (d) sleep timing and duration differences on workday and free day sleep episodes depending on method. Violin plots represent the distribution of the individual data, with bolded dashed lines representing the median and light dashed lines representing the 25^th and 75^th quartile of the data. The circular symbols next to the violin plots represent mean and standard deviations. Brackets denote significant differences from paired t-tests at the end of each line. ***p < 0.0001, **p < 0.001, *p < 0.05. Asterisks above circular symbols represent significant differences between sleep measurements and either eDiary or actigraphy.

If an eDiary was not completed for that day, a reminder email (“feedback”) was automatically delivered to the participant; participants were limited to 36 h to record an entry and 72 h to edit an entry. A calendar view of the daily entries allowed study staff to efficiently check the data daily to determine if any entries needed to be manually checked due to participant comments and/or inconsistent raster plots or illogical entries. Study staff used email scripts to request clarification or correction of an entry. Data entry averaged 95% (2.5%)¹⁵ and errors decreased over time; we hypothesize these positive results were partially due to the frequent feedback from staff to the participants. For comparison, the proportion of days of data collected for actigraphy with diary over that same timespan was 87% and without diary 93%.

Using retrospectively calculated effort, it took research staff approximately 45 min for real-time checking and post-study corrections for a ~ 30-day diary for each participant. We next analyzed sleep outcomes using a mixed-effects model with method as fixed effect and participant as a random effect to account for interparticipant differences. To compare individual methods, paired t-tests were utilized. Sleep onset (linear mixed model method effect; F_2,441 = 98.3, p < 0.0001), offset (F_2,444 = 159.5, p < 0.0001), and duration (F_2,443 = 76.3, p < 0.0001) differed significantly depending on method Fig. 3b); in particular, both the single-time subjective questionnaire and multi-day actigraphy recorded longer sleep durations compared to the eDiary: 0.7 h (SEM, range, paired t-test; 0.08 h, −6.7 to 3.8 h, p < 0.0001) and 0.5 h (0.04 h, −1.6 h to 3.1 h; p < 0.0001), respectively (Fig. 3c). These findings were consistent on both work/school days and free/weekend days and were driven by differences in both sleep onset and offset times (Fig. 3d).

An accurate and efficient way forward

We find that using sleep diaries with automated error checking (including data display for the participant to use while checking) can greatly reduce staff burden, increase the amount of valid data available for analyses, provide significantly lower estimates of sleep duration compared with those from other commonly used methods of sleep measurement, and demonstrate a high agreement rate with gold-standard PSG measurement. Though many sleep diaries, such as the Consensus Sleep Diary¹⁸, have been well-validated^19–21, we advocate that online diaries should include additional features, including real-time checking, prompt feedback to participants both about completing entries and about common potential entry errors (e.g., AM/PM errors, activity overlap), and easy access by staff (for daily review of multiple participant use and errors) to ensure high accuracy and reduce staff time.

eDiaries represent a tool that can be implemented to measure multiple days of sleep timing and duration (and other derived measures such as variability) easily and accurately with reduced staff burden in a large number of participants spanning large geographical areas²². Importantly, an online eDiary can reduce the research staff time needed to manually check diaries for errors by up to 78% compared to paper diaries. There are, however, nuances to consider. For example, when comparing the diaries and actigraphy for the absolute difference in sleep onset timing with PSG, actigraphy appears to have an earlier peak (i.e., lower absolute difference) than diary, potentially suggesting a more accurate measurement of sleep onset. This potential improvement in accuracy would need to be weighed by the ability to capture the sleep onset, as the diary had over 34% more usable onsets than actigraphy (Figs. 1f and 2), as well as other factors noted previously.

There are some limitations to consider. We note that eDiaries can be limited to individuals with internet access and/or devices that can connect to the internet, and our observed results may not apply those with disorders that may limit computer use. Moreover, despite the ease of use of the eDiary and other forms of diaries, the information gathered remains limited to what the user chooses to provide and is subject to bias or misreporting. This could be particularly true if the recall period is long²³; long recall periods are not a problem with daily entries, especially those enforced by a time-limited entry window for online diaries. eDiaries can be modified (i.e., questions added/subtracted/text modified/time for reminder to be sent) by staff to tailor the diary language to be more efficient (e.g., not asking about sleep at night for shift workers who sleep during the day) or add questions relevant to specific populations such as patients with sleep disorders, pain, or anxiety²⁴. While our data suggest high compliance and agreement of diary with PSG, it is worth considering that our study population consisted of medical residents that were completing their diaries while actively being recorded with PSG, which may have increased recall accuracy in this potentially highly compliant population. We chose to utilize this population for these comparisons as they (1) were simultaneously completing diaries with PSG and actigraphy, (2) were working and sleeping at all hours of the day, thereby allowing the examination of sleep at all times with differing durations, and (3) were compliant in not only completing diaries, but also in wearing actigraphy and PSG equipment. It is also important to note that though participants were completing diaries while wearing PSG, they were also completing diaries while wearing actigraphy and thus any recall bias is likely similar between measures. Regardless, we still recommend that research staff examine eDiary entries within 1-2 days, both for data accuracy and to provide positive feedback to participants about diary completion (likely increasing completion rate). The marked differences between sleep duration estimation methods and the questions used²⁵ may explain why different sleep measurement tools yield differing associations with disease⁵. Further work is needed to identify when the eDiary provides accurate associations with disease as compared to gold-standard markers (e.g., PSG).

In conclusion, we propose that data collected with an eDiary holds potential to uncover new associations and mechanisms vital for optimal health, safety, and well-being and should be considered in sleep research moving forward.

Author contributions

A.W.M. and M.S.H: conceptualization, investigation, writing—original draft; A.S., J.P., S.Q., S.T., L.K.B, A.J.K.P., E.F., ānd C.O.: investigation, writing—review & editing; S.W.L., C.A.C. E.B.K.: -conceptualization, investigation, project administration, supervision, writing—review & editing.

Data availability

The datasets generated and/or analyzed during the current study are not publicly available due to confidentiality constraints but are available from the corresponding author on reasonable request.

Declarations

Competing interests

A.W.M. reports consulting for Pure Somni Corporation and Portland Public Schools. M.S.H. reports current research grants from NIH and Merrimack College. She received an equipment award from Labfront and Garmin Health. She serves as a subject matter expert for easyJet plc and as a consultant for TimeShifter, Inc. AS provided consulting services for Suntory Global Innovation Center and received honoraria from Oak Ridge Associated Universities, the Taiwanese Society for Nutritional Psychiatry Research, the Korea Advanced Institute of Science and Technology, Amrita Vishwa Vidyapeetham, Aalto University, Dartmouth Center for Technology and Behavioral Health, University of Nebraska-Lincoln, and the National Science Foundation. AS also received travel support from Apple and the Taiwanese Society for Nutritional Psychiatry Research, as well as research funding from General Motors Company, OpenAI, Apple, Google, and Nippon Electric Company. L.K.B. reports institutional support from Delta Airlines; as well as honorariums from Washington State University., A.J.K.P. has received research funding from Versalux, Beacon, and Delos; he is co-founder and co-director of Circadian Health Innovations PTY LTD. S.W.L. has received consulting fees from Absolute Rest, Ashurst Risk Advisory Pty Ltd, Hintsa Performance AG, and Monash University, and has consulting contracts with Akili Interactive, Apex 2100 Ltd, Consumer Sleep Solutions, Illumalife, KBR Wyle Services, Light Cognitive, and Timeshifter Inc. He has received honoraria and travel or accommodation expenses from Clifton College and CloserStill Media; and travel or accommodation expenses (no honoraria) from Energy Performance Lighting, IESANZ, the Royal Society of Medicine, and Wiley; and royalties from Oxford University Press and Monash University. He holds equity in iSleep pty. He has received a Clinical Research Support Agreement and Clinical Trial Agreement with Vanda Pharmaceuticals Inc. He is an unpaid Board Member of the Midwest Lighting Institute (non-profit). He is a Visiting Professor at the University of Surrey, UK and an Honorary Professor at Monash University, Australia. He holds several pending (US20190366032; US20210162164 US20220151552) and one awarded (USD943612) patents. He has served as a paid expert in legal proceedings related to light, sleep and health. C.A.C. reports grants and contracts to Brigham and Women’s Hospital from Axsome Therapeutics, CDC Foundation, City of San Francisco, Dayzz Live Well, Delta Airlines, Jazz Pharmaceuticals PLC Inc, Puget Sound Pilots, and Regeneron Pharmaceuticals/Sanofi during the conduct of the study; reports personal fees from Axsome Therapeutics, Associated Professional Sleep Societies, Bryte Foundation, Clement Law Firm, Institute of Digital Media and Child Development, Klarman Family Foundation, Law Offices of James L Mitchell, Law Office of Yolanda Huang, Massachusetts Medical Society, National Council for Mental Wellbeing, National Sleep Foundation, Puget Sound Pilots, Rabb and Rabb LLC, Segal Law Firm, Shaked Law Firm, P.A., Simpson & Simpson, Tencent Holdings Ltd, Teva Pharma Australia, The Armstrong Firm, PLLC, Vanda Pharmaceuticals Inc, With Deep, Inc., and Zehl Law Firm during the conduct of the study; reports research/education support to Brigham and Women’s Hospital from Abbaszadeh Foundation, Alexandra Drane, Apnimed, Inc., Avadel Pharmaceuticals, Bryte Foundation, Casey Feldman Foundation, Cephalon, DR Capital, Eisai Co., LTD, f.lux Software, LLC, Idorsia Pharmaceuticals LTD, Mary Ann & Stanley Snider via Combined Jewish Philanthropies, Harmony Biosciences LLC, Jazz Pharmaceuticals PLC, Inc, Johnson & Johnson, NeuroCare, Inc., Optum, Peter Brown and Margaret Hamburg, Philips Respironics Inc, Regional Home Care, ResMed Foundation, San Francisco Bar Pilots, Sleep Number Corp., Stuart F. and Diana L. Quan Charitable Fund, Summus, Inc., Takeda Pharmaceutical Co., LTD, Teva Pharmaceuticals Industries Ltd, ResMed, Sanofi, Philips, Vanda Pharmaceuticals, and Whoop, Inc. during the conduct of the study; and is the incumbent of an endowed professorship provided to Harvard University by Cephalon Inc. during the conduct of the study; report serving as an expert witness in legal cases, including those involving Advanced Power Technologies, Aegis Chemical Solutions LLC, Amtrak, Bombardier, Inc., Casper Sleep Inc, C&J Energy Services, Delta Airlines/Comair, Enterprise Rent-A-Car, FedEx, Greyhound Lines, Inc., Puget Sound Pilots, Steel Warehouse, FedEx, Greyhound Lines, Product & Logistics Services LLC, San Francisco Sheriff’s Department, Schlumberger Technology Corp., Union Pacific Railroad, UPS, and Vanda Pharmaceuticals during the conduct of the study; reports having an equity interest in Vanda Pharmaceuticals, With Deep, Inc, and Signos, Inc. during the conduct of the study; and reports receiving royalties from McGraw Hill, Massachusetts Medical Society, and Philips Respironics for the Actiwatch-2 and Actiwatch Spectrum devices during the conduct of the study. C.A.C.: interests were reviewed and are managed by the Brigham and Women’s Hospital and Mass General Brigham in accordance with their conflict-of-interest policies. E.B.K.: consulting: Buck Institute for Research on Aging, Circadian Therapeutics, Guidepoint. Travel support: Lorentz Center (Netherlands), Sleep Research Society, Society for Research on Biological Rhythms. Other: unpaid scientific board member of Chronsulting; partner is founder, director, and chief scientific officer of Chronsulting.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s44323-026-00106-z.