Skip to main content
PLOS One logoLink to PLOS One
. 2021 Oct 20;16(10):e0258314. doi: 10.1371/journal.pone.0258314

Detecting variations in ovulation and menstruation during the COVID-19 pandemic, using real-world mobile app data

Brian T Nguyen 1,*, Raina D Pang 2, Anita L Nelson 3, Jack T Pearson 4, Eleonora Benhar Noccioli 4, Hana R Reissner 1, Anita Kraker von Schwarzenfeld 4, Juan Acuna 5,6
Editor: Alessio Paffoni7
PMCID: PMC8528316  PMID: 34669726

Abstract

Background

As war and famine are population level stressors that have been historically linked to menstrual cycle abnormalities, we hypothesized that the COVID-19 pandemic could similarly affect ovulation and menstruation among women.

Methodology

We conducted a retrospective cohort study examining changes in ovulation and menstruation among women using the Natural Cycles mobile tracking app. We compared de-identified cycle data from March-September 2019 (pre-pandemic) versus March-September 2020 (during pandemic) to determine differences in the proportion of users experiencing anovulation, abnormal cycle length, and prolonged menses, as well as population level changes in these parameters, while controlling for user-reported stress during the pandemic.

Findings

We analyzed data from 214,426 cycles from 18,076 app users, primarily from Great Britain (29.3%) and the United States (22.6%). The average user was 33 years of age; most held at least a university degree (79.9%). Nearly half (45.4%) reported more pandemic-related stress. Changes in average cycle and menstruation lengths were not clinically significant, remaining at 29 and 4 days, respectively. Approximately 7.7% and 19.5% of users recorded more anovulatory cycles and abnormal cycle lengths during the pandemic, respectively. Contrary to expectation, 9.6% and 19.6% recorded fewer anovulatory cycles and abnormal cycle lengths, respectively. Women self-reporting more (32.0%) and markedly more (13.6%) stress during the pandemic were not more likely to experience cycle abnormalities.

Conclusions

The COVD-19 pandemic did not induce population-level changes to ovulation and menstruation among women using a mobile app to track menstrual cycles and predict ovulation. While some women experienced abnormalities during the pandemic, this proportion was smaller than that observed prior to the pandemic. As most app users in this study were well-educated women over the age of 30 years, and from high-income countries, their experience of the COVID-19 pandemic might differ in ways that limit the generalizability of these findings.

Introduction

On March 11, 2020 the World Health Organization (WHO) declared the COVID-19 outbreak a global pandemic, calling upon all countries to take urgent and aggressive action to prevent further spread of the disease. The pandemic declaration led to increasingly strict and widespread stay-at-home orders, which contributed to population level concerns about the risk of not only infection and loss of life, but the potential irrecoverable loss of livelihood, as non-essential businesses were shut down and citizens deprived of income, while facing continued expenses. Persisting beyond a full year, the COVID-19 pandemic produced a global population-level stressor that continues to influence how people live.

Historically, population level stressors have been linked to menstrual changes. Examples include World War II from 1939–1945 [1], the Dutch Famine from 1944–1955 [2], and the Desert Storm War in 1996 [3]. These examples represent a range of stressors, whether they be physical vs. psychological, direct vs. indirect, or short vs. long-term. At the extremes of stress, a case series of women studied prior to being executed noted that almost all became amenorrheic [4]. In an observational study of Lebanese women exposed to wartime bombing from April 11–27, 1996, 35% experienced menstrual aberrations for 3 months, as compared to an unexposed group that reported abnormalities in 2.6% [5]. Abnormal menstrual patterns are also commonly reported among desert-dwelling hunter–gatherer women who face difficult living conditions [6]. Yet mortal stresses of this magnitude are not required to induce menstrual aberrations; chronic job-related stress can be a contributor [7, 8]. Further, in a study of Japanese college students, 15.8% reported a correlation between their school examinations and irregular menses [9]. The range of stressors that can influence one’s likelihood of menstrual irregularity is wide, suggesting that other mediating factors may play a role.

The impact of stress on the reproductive system is grounded in biology whereby stress-related glucocorticoids can inhibit the release of gonadotropin releasing hormone (GnRH), luteinizing hormone (LH), and estradiol (E2) from the hypothalamic-pituitary-gonadal axis, which is most easily observed as secondary amenorrhea. More subtle manifestations would include delays in ovulation, anovulation, and changes in both cycle and menstruation lengths.

The COVID-19 pandemic represents a unique stressor, independent of whether individuals were infected by the virus, because of its far-reaching psychologic, social, and economic consequences beyond physical alone. Consequently, reproductive age females may have experienced changes in their ovulatory and menstrual cycles during this time. No studies have yet examined the influence of stress linked to an ongoing global pandemic on ovulatory and menstrual changes among a reproductive age female population. The pandemic provides an opportunity to identify and characterize potential changes.

Natural Cycles is the first Food and Drug Administration (FDA) cleared and CE-marked mobile app that uses women’s records of menstruation and basal body temperatures to identify their fertile window and estimate appropriate times when they can have unprotected sexual intercourse and be reasonably certain that they will avoid pregnancy. As of the beginning of the COVID-19 pandemic in 2020, Natural Cycles had more than 1.5 million members using the app for contraception, making its database one of the largest collections of menstrual cycle data ever compiled. We conducted analyses using aggregated, real-world menstrual cycle data from app users both before and during the COVID-19 pandemic with the primary objective of detecting potential changes in ovulation, cycle length, and menstrual duration during the pandemic. The secondary objective of this study was to examine the association of any menstrual changes with self-reported perceived stress related to the pandemic.

Materials and methods

Data collection

Our data was comprised of real-world, sociodemographic, limited clinical, and menstrual cycle data submitted by users of the Natural Cycles pregnancy prevention and fertility tracking mobile application [10]. All users who contributed data agreed to make their data available for clinical investigation prior to starting application use. All data was de-identified by the Natural Cycles data management team (EBN) and stored within a closed database, which was transferred to the research team at the University of Southern California (USC; BTN, RDP, HR, ALN) for subsequent data analysis. This research plan was reviewed and approved by the USC Institutional Review Board, which classified the study as exempt, non-human subjects research (HS-20-00402).

Inclusion and exclusion criteria

We included users who registered an account for contraception prior to September of 2019 and who consented to release their de-identified data for research purposes. We excluded cycle data from users who were breastfeeding, who reported a pregnancy in the 12 months prior to registration within the data collection period, and who became pregnant during use of the app. We additionally excluded any user on a hormonal contraceptive method, as well as reporting at the time of registration, a diagnosed medical condition that could influence cycle regularity, such as polycystic ovarian syndrome, endometriosis, thyroid disorders, and perimenopausal symptoms. To account for any undiagnosed secondary amenorrhea or ovulatory disorders, as well as to improve our ability to detect pandemic-induced changes in ovulation and menstruation, we excluded users who recorded any cycle length lasting more than 90 days during the pre-COVID analysis frame. We further excluded users who did not contribute at least two cycles of data prior to the COVID-19 pandemic and at least two cycles during the COVID-19 data collection period.

With respect to cycle data, we included only complete cycles (i.e., cycles with a start and an end) collected from March through September 2019 (pre-COVID) and March through September 2020 (during COVID). All cycles at the beginning of the sampling frame started in March; all cycles included from the end of the sampling frame must have started in September even if the cycles did not end until later. We excluded cycle data starting from October 2019 through February 2020 due to ambiguity at the time about the scale and threat of infection among various populations of users. We did not include cycles in which the Natural Cycles algorithm did not have enough data to definitively determine whether an ovulation occurred within the cycle (i.e., not enough user data entered). To improve the precision of contributed cycle data, we further excluded cycles that were not validated by at least 10 basal body temperature entries.

Outcomes of interest

We were primarily interested in examining the proportion of users experiencing anovulation, abnormal cycle length, and prolonged menstruation among app users pre-COVID compared to during COVID. Anovulatory cycles were defined according to the Natural Cycles app’s proprietary algorithm. The algorithm identifies ovulation retrospectively based on the first day of menstruation and basal body temperatures, which may be supplemented by positive urinary LH tests. Basal body temperatures are recorded each morning using a thermometer sensitive to the hundredth place, and with measures excluded if the user reports any illness, alcohol intake, or changes in sleep that might influence basal temperatures. Users of Natural Cycles record basal temperatures for approximately 70% of the days; approximately 25% use LH tests. To reduce the risk of misidentifying ovulations, the algorithm reports ovulation by rising basal body temperature only if the average temperature from three consecutive calendar days is greater than the woman’s follicular phase average and her baseline average across all data entries, as well as consistent with her luteal phase average [11]. If no temperature rise is observed and the data quality and quantity is deemed sufficient, the cycle is flagged as anovulatory. Of note, users with stable measurements (e.g., small day-to-day variations in the same cycle phase), require fewer data points for the Natural Cycles algorithm to draw conclusions about changes in the basal body temperature. Cycles with low data quality (e.g., high day-to-day temperature variability) or data that is insufficient to detect or exclude an increase in the basal body temperature are excluded by the analysis. While missing data may affect the app’s ability to predict the exact day of ovulation, they are not expected to affect its determination of whether ovulation has occurred. Based on criteria from the International Federation of Gynecology and Obstetrics (FIGO), we defined abnormal cycle length as lasting less than 24 days or more than 38 days, and prolonged menstruation length as lasting greater than 8 days [12, 13].

Covariates

Natural Cycles implemented a query on May 11, 2020 of its users’ experience of pandemic-related stress. Users could provide a response to this item through June 11, 2020. We incorporated this measure to examine a stress-related pathway for menstrual abnormalities. The researchers asked users to rate two Likert-type items: (1) “Thinking about your stress level before the COVID-19 pandemic started, how stressed were you then?” (2) “How stressful is the COVID-19 pandemic to you now?” The Likert items were anchored from “Not at all stressed” (1) to “Extremely stressed” (5). We then calculated a COVID-related stress change score by subtracting the user’s report of stress before COVID from their stress rating during COVID. For ease of interpretation, we categorized perceived changes in stress using five categories: “much less (1),” “less (2), “unchanged (3),” “more (4),” and “much more (5).” In addition, we used sociodemographic data and reproductive histories provided by users at the time of app registration and updated during use of the app as covariates in our analysis. These covariates included: age, country of registration, education (from less than a high school degree to graduate degree), relationship status (e.g., in a relationship, engaged or married, it’s complicated, and single), history of pregnancy, and whether they have any children.

Data analysis

At the population level, we examined differences in average cycle lengths and menstruation lengths before and during the COVID-19 pandemic via paired t-tests. We calculated the proportions of users experiencing an anovulatory cycle before and during the pandemic and analyzed for differences via Chi-square tests of association. Given that individual users did not contribute the same number of cycles to the study, we compared differences in the proportions of abnormal menstrual parameters recorded by calculating for each user their total number of cycles with anovulation, abnormal cycle length, and prolonged menses for each sampling frame and divided them by the number of cycles contributed. We examined for statistically significant changes in these abnormal cycle parameters, pre-COVID and during COVID, via paired t-test.

Given our interest in COVID-related stress as a potential mediator of abnormal cycles, we conducted a subset analysis of data from individuals contributing stress ratings during the pandemic and examined its relationship categorically via Chi-square tests and continuously via paired t-tests. We examined the role of the user’s age, country of registration, education, relationship status, and history of pregnancy and/or children on any increase in mean anovulation, cycle length, and menstruation length among users via Chi-square tests. We included covariates associated with increased anovulatory cycles, abnormal cycle lengths, and abnormal menstruation length (at an alpha level of 0.05) in a separate multivariable logistic regression to determine each covariate’s adjusted odds of influencing the outcome of interest. All analyses were conducted in Stata (Version SE/14.2; College Station, TX).

Results

Our sampling frame included a total of 214,426 cycles of data from 18,076 individual users from over 60 countries worldwide (Table 1). Users were primarily from Great Britain (29.3%), the United States (22.6%), and Sweden (17.8%). The average user was 32.5 ± 5.8 years of age at the time of analysis, and most held at least a university degree (79.9%). Most users reported being in a relationship, engaged, or married (85.1%); only 25.6% of users reported ever being pregnant and 16.9% of users reported having at least one child.

Table 1. Characteristics of natural cycles users contributing at least two cycles of data prior to and during the COVID-19 pandemic (n = 18,076 users; N = 214,426 cycles).

Change in perceived stress from pre-COVID to during COVID pandemic, N = 10,294 Total users, N = 18,076
Less or much less, n = 2206 Unchanged, n = 3419 More or much more, n = 4669 n (%)
Age (%) [mean 32.5 ± 5.8 years] ***
<25 103 (21.4) 176 (36.6) 202 (42.0) 911 (5.0)
25–34 1479 (22.2) 2104 (31.6) 3071 (46.2) 11552 (63.9)
35–44 544 (19.9) 980 (35.8) 1215 (44.4) 4861 (26.9))
45+ 80 (19.1) 159 (47.9) 181 (43.1) 752 (4.2)
Country (%) ***
Great Britain 700 (22.5) 942 (30.2) 1475 (47.3) 5293 (29.3)
United States 436 (18.6) 717 (30.6) 1189 (50.8) 4091 (22.6)
Sweden 421 (22.3) 742 (39.3) 726 (38.4) 3223 (17.8)
Other 649 (22.0) 1018 (34.6) 1279 (43.4) 5469 (30.3)
Education (%) **
High school or less 219 (21.7) 385 (38.1) 407 (40.3) 1672 (11.2)
Vocational training 178 (21.3) 280 (33.5) 378 (45.2) 1322 (8.9)
University degree 1377 (21.1) 2139 (32.8) 3014 (46.2) 10701 (71.8)
PhD degree 156 (21.5) 215 (29.6) 355 (48.9) 1208 (8.1)
Relationship status (%) **
In a relationship 1092 (22.2) 1625 (33.1) 2198 (44.7) 8211 (52.6)
Engaged or married 611 (19.5) 1046 (33.3) 1483 (47.2) 5071 (32.5)
Single 232 (22.1) 367 (35.0) 451 (43.0) 1700 (10.9)
It’s complicated 109 (27.1) 119 (29.6) 174 (43.3) 630 (4.0)
Pregnancies (%)
None 1507 (21.8) 2241 (32.4) 3169 (45.8) 11479 (74.4)
At least one 490 (20.4) 842 (35.0) 1074 (44.6) 3949 (25.6)
Children (%) *
None 1747 (21.8) 2634 (32.9) 3637 (45.4) 13255 (83.2)
At least one 308 (19.2) 563 (35.1) 731 (45.6) 2686 (16.9)

*p<0.05

**p<0.01

***p<0.001.

Sum different from total number of users due to missing data.

The proportion of those reporting being very to extremely stressed rose from 46.2% pre-COVID to 61.1% during COVID. Nearly half of users (45.4%) reported more pandemic-related stress, with 33.2% reporting no change, and 21.4% reporting less stress compared to the pre-COVID period. Individuals between the ages of 25–34 years, from the United States, with a graduate level degree, who reported being engaged or married, and having a child were significantly more likely to report more stress during the pandemic than prior.

Users individually contributed approximately 6 cycles to each of the pre-COVID and during COVID sampling frames. With respect to cycle characteristics (Table 2), the average cycle length among users significantly decreased from 29.40 (95%CI 29.34–29.46) days pre-COVID to 29.16 (95%CI 29.10–29.22) days during COVID (p<0.001). The average menstrual duration significantly increased from 4.21 (95%CI 4.19–4.23) days to 4.32 (95%CI 4.30–4.34) days. The average incidence of anovulation and abnormal cycle length decreased significantly across cycles, from 2.9% (2.7%-3.0%) to 2.5% (2.3%-2.6%) and 8.7% (8.4%-8.9%) to 8.0% (7.8%-8.2%), respectively, while the average incidence of prolonged menstruations increased from 0.9% (0.8%-1.0%) to 1.0% (9.0%-1.1%).

Table 2. Menstrual characteristics and abnormalities prior to and during the COVID-19 pandemic (n = 18,076 users; N = 214,426 cycles).

Pre-COVID: During COVID: p-value
Mar-Sep 2019 Mar-Sep 2020
(n = 108,021 cycles) (n = 106,405 cycles)
Mean (95%CI) Mean (95%CI)
Cycle length 29.40 (29.34–29.46) 29.16 (29.10–29.22) <0.001
Menstrual duration 4.21 (4.19–4.23) 4.32 (4.30–4.34) <0.001
Proportions % (95%CI) Proportions % (95%CI)
Anovulatory cycles 2.9% (2.7%-3.0%) 2.5% (2.3%-2.6%) <0.001
Abnormal cycle lengths 8.7% (8.4%-8.9%) 8.0% (7.8%-8.2%) <0.001
Prolonged menses 0.9% (0.8%-1.0%) 1.0% (0.9%-1.1%) 0.002
Users (%) Users (%)
Anovulatory cycles No 16075 (88.9) 16432 (90.9) <0.001
Yes 2000 (11.1) 1644 (9.1)
Abnormal cycle lengths No 12937 (71.6) 13266 (75.13) <0.001
Yes 5138 (28.4) 4810 (26.6)
Prolonged menses No 17449 (96.5) 17375 (96.1) 0.04
Yes 626 (3.5) 701 (3.9)

Abnormal cycle length: <24 days or >38 days; Prolonged menses: >8 days.

At the user level, the proportion of users experiencing any anovulatory cycles and any abnormal cycle lengths decreased from 11.1% to 9.1% (p<0.001) and 28.4% to 26.6% (p<0.001), respectively. The proportion experiencing increased menstrual duration increased from 3.5% to 3.9% (p = 0.04). We did not detect any statistically significant associations between changes in user ratings of stress before and during the pandemic and abnormal cycle parameters (Fig 1, Table 3).

Fig 1. Changes in the proportion of abnormal menstrual cycle parameters among users reporting more stress during than prior to the COVD-19 pandemic (n = 4,729).

Fig 1

Table 3. Users reporting more cycle abnormalities during the COVID-19 pandemic, by change in perceived stress (N = 10,293 users).

Less stress No change More stress p-value
n = 2214 (%) n = 3437 (%) n = 4729 (%)
Anovulatory cycles (n = 776) 175 (7.9) 244 (7.1) 357 (7.7) 0.74
Abnormal cycle lengths (n = 1942) 413 (18.7) 621 (18.2) 908 (19.5) 0.48
Prolonged menses (n = 370) 83 (3.8) 130 (3.8) 157 (3.4) 0.55

To explore factors associated with changes in the proportion of anovulatory cycles and abnormal cycle lengths after the pandemic, we conducted a multinomial logistic regression, setting no changes in the proportion of abnormalities as the reference point. We included those potential covariates significantly associated with changes in abnormal cycle parameters at the p<0.05 level. While changes in stress were not linked to changes in cycle parameters, we included this covariate given our interest in the potential role of subjective stress on menstrual changes (Table 4). Via this regression, only age and relationship status remained independently associated with any of the cycle outcomes. Users above the age of 45 years were more likely to report more anovulatory cycles, abnormal cycle lengths, and prolonged menses. Individuals who were in a relationship or were engaged or married were less likely than their complicated and single counterparts to experience more anovulatory cycles.

Table 4. Factors linked to more versus fewer cycle abnormalities during the COVID-19 pandemic (n = 18,076 users; N = 214,426 cycles).

Anovulatory cycles Abnormal cycle lengths Prolonged menses
Fewer More Fewer More Fewer More
aOR(95%CI) aOR(95%CI) aOR(95%CI) aOR(95%CI) aOR(95%CI) aOR(95%CI)
Age
<25 Ref Ref Ref Ref Ref Ref
25–34 0.49* 0.46* 0.71* 0.79 0.98 0.80
(0.36–0.66) (0.32–0.64) (0.55–0.92) (0.59–1.06) (0.50–1.93) (0.46–1.40)
35–44 0.38* 0.44* 0.65* 1.17 1.24 0.97
(0.27–0.53) (0.31–0.65) (0.49–0.87) (0.86–1.60) (0.60–2.54) (0.53–1.76)
45+ 0.79 1.57* 1.64* 3.51* 2.23 2.50*
(0.49–1.27) (1.00–2.50) (1.11–2.43) (2.37–5.19) (0.93–5.38) (1.24–5.02)
Education
High school or less Ref Ref Ref Ref Ref Ref
Vocational training 1.20 1.16 1.13 0.88 0.54 1.14
(0.87–1.65) (0.83–1.63) (0.89–1.45) (0.59–1.06) (0.27–1.0) (0.68–1.92)
University degree 1.03 0.87 1.02 0.93 0.89 1.18
(0.81–1.31) (0.67–1.13) (0.85–1.23) (0.78–1.12) (0.59–1.35) (0.79–1.76)
PhD degree 0.82 0.84 0.87 1.01 0.76 0.73
(0.57–1.20) (0.56–1.24) (0.74–1.27) (0.77–1.31) (0.41–1.42) (0.39–1.37)
Relationship status
In a relationship 0.81 0.75* 0.90 0.95 1.04 1.25
(0.64–1.03) (0.58–0.97) (0.75–1.08) (0.79–1.15) (0.67–1.61) (0.83–1.86)
Engaged or married 0.74* 0.72* 0.92 0.88 0.94 0.99
(0.57–0.96) (0.54–0.95) (0.75–1.12) (0.72–1.08) (0.59–1.52) (0.64–1.53)
It’s complicated 1.09 0.94 1.00 0.93 0.51 1.18
(0.74–1.62) (0.60–1.45) (0.73–1.38) (0.67–1.30) (0.19–1.35) (0.62–2.27)
Single Ref Ref Ref Ref Ref Ref
Children
None Ref Ref Ref ref Ref Ref
One or more 0.84 0.84 1.00 1.01 0.89 0.98
(0.64–1.09) (0.63–1.11) (0.96–1.30) (0.84–1.21) (0.58–1.38) (0.67–1.42)
Subjective COVID-19 related stress
Less 0.91 1.02 1.11 1.10 1.10 0.92
(0.74–1.12) (0.81–1.29) (0.91–1.25) (0.93–1.28) (0.77–1.56) (0.67–1.27)
Unchanged Ref Ref Ref Ref Ref Ref
More 0.98 1.07 1.08 1.12 0.82 0.88
(0.83–1.16 (0.88–1.29) (0.95–1.23) (0.98–1.28) (0.60–1.12) (0.68–1.14)

* p<0.05.

Discussion

The COVID-19 pandemic created an environment of global stress and an unprecedented opportunity to explore the influence of a potential chronic stressor on menstrual cycle parameters. No studies to date have analyzed large-scale, daily self-reported and biologically verified menstrual cycle data during a pandemic. Based on observations from previous historical stressors, we hypothesized that female menstrual cycle parameters would exhibit more abnormalities, inclusive of anovulation, during the COVID-19 pandemic. In this analysis of more than 200,000 cycles contributed by more than 18,000 app users with no recorded history of amenorrhea, we noted a statistically significant decrease in cycle length and increase in duration of menses at the population level, though these remained clinically unchanged at 29 and 4 days, respectively. At the individual user level, approximately 7.7% and 19.5% of users recorded more anovulatory cycles and abnormal cycle lengths following the pandemic, respectively; 3.4% recorded prolonged menses.

For some women, a new finding of amenorrhea, menorrhagia, or the inability to predict ovulation can be distressing and may lead to healthcare visits, laboratory workups, and imaging studies that may be unable to definitively determine an etiology. Contrary to expectation however, a larger proportion of mobile app users recorded fewer abnormalities during the pandemic than prior (9.6% and 20.1% recorded fewer anovulatory cycles and abnormal cycle lengths, respectively). Even when examining the cycle characteristics of women self-reporting more (32.0%) and markedly more (13.6%) stress following the pandemic, we did not observe any independent association of stress with cycle abnormalities. Rather than among women reporting high stress, we noted more abnormal cycles among women greater than the age of 45 years during the pandemic, which may suggest the stress-related sensitivity of the hypothalamic-pituitary axis during the perimenopausal state [14].

With respect to there being fewer abnormal parameters recorded during the pandemic, these findings may in part be explained by the sociodemographic characteristics of our sample population which may have provided some protection from both the physical and psychosocial effects of the pandemic. Our mobile app-using population was comprised primarily of healthy, college-educated women in their 30s, in relationships, and from majority Caucasian countries. Even during the first months of the pandemic, many individuals represented by the majority demographic in this sample were able to transition from commuting to working from home [15, 16], which based on analyses of global Twitter data, was perceived as a positive change among nearly 75% of individuals who additionally endorsed sentiments of trust, anticipation, and joy about the change during the pandemic [17]. Given the time and opportunity, some individuals may have engaged in exercise and better health habits, such as more regular sleep [18, 19], which taken together, might explain reductions in menstrual abnormalities during the pandemic.

In addition, we noted the protective effect of being in a relationship on the incidence of anovulation. While the COVID-19 pandemic may be exacerbating global gender inequality with more women transitioning to unemployment and reducing their work hours, women using the Natural Cycles app are primarily well-educated and from developed countries where one might expect more egalitarian gender role attitudes. In these households, gender roles may have been changing [20], with male partners spending more time at home and sharing both housework and childcare-related burdens [21]. These findings suggest that while the pandemic increased users’ feelings of stress overall, that their health awareness, as well as their stable, protected working and living conditions, may have provided greater resilience and ability to control the situation [22].

In our multivariable logistic regression, we directly examined reports of marked increases in perceived stress during COVID-19, finding no association with abnormal menstrual parameters both before and after controlling for sociodemographic characteristics. Only 56.9% of users responding to the pandemic-related stress item, with the consequent possibility of selection bias; however, this sub-analysis still captured data from more than 10,000 users. For this population, general stress provoked by the COVID-19 pandemic and its preoccupations may not have been sufficient to elicit an adrenal and adaptive neuroendocrine response to acute stress or chronic stress. We note as well that our chosen time points may not have been sufficiently long enough to observe stress-related physiologic effects which might not have been observed during the early months of the pandemic. Alternatively, our surface-level measurement of self-reported stress may have been insensitive to true distress or the behavioral determinants of stress-induced hypothalamic dysfunction [23]; our 2-item assessment of pandemic-related stress was likely biased towards inflated stress-reporting. While an ideal measure of change in stress would have repeated the question at two time points, we anticipated that user responses to the above two questions would still reflect perceived stress, as related to the pandemic. Yet even in studies where validated measures, such as the State-Trait Anxiety Inventory, are used, the relationship between the psychosocial status of women and aberrations in their menstrual cycles cannot always be demonstrated [24]. Future studies could collect serial stress assessments to evaluate the effects of chronic stress, as well as serum cortisol levels or diagnoses of COVID-19.

We also note that the proportion of cycles with prolonged menstruations increased during the pandemic, while the proportion of individuals with prolonged menses decreased, suggesting that those developing prolonged menses likely experienced recurrent prolonged menses. We are unable however, to make any inferences about the clinical significance of this finding as we did not quantify bleeding. Future versions of the app may expand its capabilities to help quantify bleeding and future analyses may examine the incidence of irregular spotting among users as a bothersome, early sign of menstrual irregularity.

The findings from this study are broadly limited by self-reported data. However, we believe that these data are collected from a group of individuals who are using the Natural Cycles app for its intended purpose of pregnancy prevention, rather than for the purposes of this research, such that these data are expected to be reliable. Mobile app-based reporting of menstrual data is preferred to conventional paper diary recording methods [25]. Further, previous publications on the menstrual cycle characteristics of more than 120,000 women using the Natural Cycles app to prevent or plan a pregnancy noted their individual contribution of approximately 9 cycles of data [26], suggesting user compliance with and acceptability of the method. Natural Cycles’ recording of daily menstrual and basal body temperature diaries thus provided us with robust data for exploring female reproductive health and physiology in the setting of widespread socioenvironmental change—the COVID-19 pandemic. Future studies may be improved by the incorporation of real-time body temperature data collected by wearable devices, such as the Oura ring [27].

Conclusion

The COVD-19 pandemic did not induce population-level changes to ovulation and menstruation among women using a mobile app to track menstrual cycles and predict ovulation. While some women experienced abnormalities during the pandemic, this proportion was smaller than that observed prior to the pandemic. As most app users in this study were well-educated women over the age of 30 years, and from high-income countries, their experience of the COVID-19 pandemic may differ in ways that limit the generalizability of these findings.

Acknowledgments

Dr. Brian T. Nguyen would like to acknowledge the support of his wife, Amy Li, whose predictable menstrual cycles led to the arrival of their newborn daughter, Charlotte Li Nguyen, during the COVID-19 pandemic.

Data Availability

The data underlying this study are publicly available at doi.org/10.5061/dryad.0cfxpnw2k.

Funding Statement

This is an investigator-initiated study by BTN, HRR, and ALN who have no commercial affiliation with Natural Cycles. While JA is an advisor for Natural Cycles, he has not received any salary or financial support for participation in this research. Natural Cycles provided support in the form of salaries for authors JTP, EBN, and AKvS, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the “author contributions” section.

References

  • 1.Whitacre FE, Barrera B, Briones TN, Suaco PS, Paz A. War amenorrhea: A clinical and laboratory study. J Am Med Assoc. 1944. Feb 12;124(7):399–403. [Google Scholar]
  • 2.Elias SG, van Noord PAH, Peeters PHM, den Tonkelaar I, Kaaks R, Grobbee DE. Menstruation during and after caloric restriction: The 1944–1945 Dutch famine. Fertil Steril. 2007. Oct;88(4 SUPPL.):1101–7. doi: 10.1016/j.fertnstert.2006.12.043 [DOI] [PubMed] [Google Scholar]
  • 3.Petit NF. Dysfunctional uterine bleeding in active-duty women: scope of the problem and management options. Womens Health Issues. 1996. Nov 1;6(6):358–61. [PubMed] [Google Scholar]
  • 4.Drew FL. The epidemiology of secondary amenorrhea. J Chronic Dis. 1961. Oct 1;14(4):396–407. doi: 10.1016/0021-9681(61)90138-2 [DOI] [PubMed] [Google Scholar]
  • 5.Hannoun AB, Nassar AH, Usta IM, Zreik TG, Abu Musa AA. Effect of War on the Menstrual Cycle. Obstet Gynecol. 2007. Apr;109(4):929–32. doi: 10.1097/01.AOG.0000257170.83920.de [DOI] [PubMed] [Google Scholar]
  • 6.VAN DER WALT LA, WILMSEN EN, JENKINS T. Unusual Sex Hormone Patterns among Desert-Dwelling Hunter-Gatherers*. J Clin Endocrinol Metab. 1978. Apr 1;46(4):658–63. doi: 10.1210/jcem-46-4-658 [DOI] [PubMed] [Google Scholar]
  • 7.Fenster L, Waller K, Chen J, Hubbard AE, Windham GC, Elkin E, et al. Psychological Stress in the Workplace and Menstrual Function. Am J Epidemiol. 1999. Jan 15;149(2):127–34. doi: 10.1093/oxfordjournals.aje.a009777 [DOI] [PubMed] [Google Scholar]
  • 8.Hatch MC, Figa-Talamanca I, Salerno S. Work stress and menstrual patterns among American and Italian nurses. Scand J Work Environ Health. 1999. Feb 2;25(2):144–50. doi: 10.5271/sjweh.417 [DOI] [PubMed] [Google Scholar]
  • 9.Yamamoto K, Okazaki A, Sakamoto Y, Funatsu M. The Relationship between Premenstrual Symptoms, Menstrual Pain, Irregular Menstrual Cycles, and Psychosocial Stress among Japanese College Students. J Physiol Anthropol. 2009. May 31;28(3):129–36. doi: 10.2114/jpa2.28.129 [DOI] [PubMed] [Google Scholar]
  • 10.Berglund Scherwitzl E, Lundberg O, Kopp Kallner H, Gemzell Danielsson K, Trussell J, Scherwitzl R. Perfect-use and typical-use Pearl Index of a contraceptive mobile app. Contraception. 2017. Dec 1;96(6):420–5. doi: 10.1016/j.contraception.2017.08.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Bull JR, Rowland SP, Scherwitzl EB, Scherwitzl R, Danielsson KG, Harper J. Real-world menstrual cycle characteristics of more than 600,000 menstrual cycles. npj Digit Med. 2019. Dec;2(1). doi: 10.1038/s41746-019-0152-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Fraser IS, Critchley HOD, Munro MG, Broder M. Can we achieve international agreement on terminologies and definitions used to describe abnormalities of menstrual bleeding? Hum Reprod. 2007. Mar 1;22(3):635–43. doi: 10.1093/humrep/del478 [DOI] [PubMed] [Google Scholar]
  • 13.Munro MG, Critchley HOD, Fraser IS, Permanente K. The two FIGO systems for normal and abnormal uterine bleeding symptoms and classification of causes of abnormal uterine bleeding in the reproductive years: 2018 revisions. Int J Gynecol Obs. 2018;143:393–408. [DOI] [PubMed] [Google Scholar]
  • 14.Rehman HU, Masson EA . Neuroendocrinology of female aging. Vol. 2, Gender Medicine. Elsevier; 2005. p. 41–56. doi: 10.1016/s1550-8579(05)80008-7 [DOI] [PubMed] [Google Scholar]
  • 15.Brynjolfsson E, Horton J, Ozimek A, Rock D, Sharma G, TuYe H-Y. COVID-19 and Remote Work: An Early Look at US Data. Cambridge, MA; 2020 Jun.
  • 16.Baker MG. Who cannot work from home? Characterizing occupations facing increased risk during the COVID-19 pandemic using 2018 BLS data. medRxiv. medRxiv; 2020. p. 2020.03.21.20031336. [Google Scholar]
  • 17.Dubey AD, Tripathi S. Analysing the Sentiments towards Work-From-Home Experience during COVID-19 Pandemic. J Innov Manag. 2020. Apr 28;8(1):13–9. [Google Scholar]
  • 18.Koushkie Jahromi M, Gaeini A, Rahimi Z. Influence of a physical fitness course on menstrual cycle characteristics. Gynecol Endocrinol. 2008;24(11):659–62. doi: 10.1080/09513590802342874 [DOI] [PubMed] [Google Scholar]
  • 19.Knell G, Robertson MC, Dooley EE, Burford K, Mendez KS. Health Behavior Changes During COVID-19 Pandemic and Subsequent “Stay-at-Home” Orders. Int J Environ Res Public Health. 2020. Aug 28;17(17):6268. doi: 10.3390/ijerph17176268 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Alon T, Doepke M, Olmstead-Rumsey J, Tertilt M. The Impact of COVID-19 on Gender Equality. Cambridge, MA; 2020 Apr.
  • 21.Reichelt M, Makovi K, Sargsyan A. The impact of COVID-19 on gender inequality in the labor market and gender-role attitudes. Eur Soc. 2021. Feb 19;23(S1):S228–45. [Google Scholar]
  • 22.Vinkers CH, van Amelsvoort T, Bisson JI, Branchi I, Cryan JF, Domschke K, et al. Stress resilience during the coronavirus pandemic. Eur Neuropsychopharmacol. 2020. Jun 1;35:12–6. doi: 10.1016/j.euroneuro.2020.05.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Berga SL. Stress-induced anovulation. In: Stress: Physiology, Biochemistry, and Pathology Handbook of Stress Series, Volume 3. Elsevier; 2019. p. 213–26. [Google Scholar]
  • 24.Sanders KA, Bruce NW. Psychosocial stress and the menstrual cycle. J Biosoc Sci. 1999;31(3):393–402. doi: 10.1017/s0021932099003934 [DOI] [PubMed] [Google Scholar]
  • 25.Jacobson AE, Vesely SK, Haamid F, Christian-Rancy M, O’Brien SH. Mobile Application vs Paper Pictorial Blood Assessment Chart to Track Menses in Young Women: A Randomized Cross-over Design. J Pediatr Adolesc Gynecol. 2018. Apr 1;31(2):84–8. doi: 10.1016/j.jpag.2017.09.009 [DOI] [PubMed] [Google Scholar]
  • 26.Bull JR, Rowland SP, Scherwitzl EB, Scherwitzl R, Danielsson KG, Harper J. Real-world menstrual cycle characteristics of more than 600,000 menstrual cycles. npj Digit Med 2019 21. 2019. Aug 27;2(1):1–8. doi: 10.1038/s41746-019-0152-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Maijala A, Kinnunen H, Koskimäki H, Jämsä T, Kangas M. Nocturnal finger skin temperature in menstrual cycle tracking: ambulatory pilot study using a wearable Oura ring. BMC Women’s Heal 2019 191. 2019. Nov 29;19(1):1–10. doi: 10.1186/s12905-019-0844-9 [DOI] [PMC free article] [PubMed] [Google Scholar]

Decision Letter 0

Alessio Paffoni

Transfer Alert

This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present.

3 Jun 2021

PONE-D-21-15151

Detecting variations in ovulation and menstruation during the COVID‐19 pandemic, using real‐world mobile app data

PLOS ONE

Dear Dr. Nguyen,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The manuscript presents some interesting data.

Before re-evaluating the possibility of publication, it is necessary that the authors provide feedback to the comments of the reviewers.

I also ask the authors to provide more details on the method the app uses to determine ovulation.

There is a comment the authors made claiming that their method has a Pearl index of 6.9, which they claim is comparable to hormonal contraceptives. This statement seems to be an advertisement for their app, and, as it is not strictly relevant to the content of the article, I think that has to be removed from the paper.

Please submit your revised manuscript by Jul 18 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Alessio Paffoni, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include your tables as part of your main manuscript and remove the individual files. Please note that supplementary tables (should remain/ be uploaded) as separate "supporting information" files

3.We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4.Thank you for stating the following in the Financial Disclosure section:

"The authors received no specific funding for this work."

We note that one or more of the authors are employed by a commercial company: Natural Cycles

a) Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

b) Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc.  

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and  there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

5. Please amend either the abstract on the online submission form (via Edit Submission) or the abstract in the manuscript so that they are identical.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Detecting variations in ovulation and menstruation during the COVID-19 pandemic, using real-world mobile app data.

The authors present a very large data set, collected through the app Natural Cycles, which provides a firm basis for real world conclusions. The data herein details changes in menstrual functionality amongst women during the COVID pandemic, comparing to data pre pandemic. The aim and relevance of the data collected is clear and necessary, however there are queries amongst how the data is collected through the app to determine ovulation. This needs to be clarified in more detail within the manuscript. Please see specific comments below.

Abstract: The first sentence of the abstract needs to be reworded or include more commas, as it stands this is not a very clear start to the manuscript.

The authors state they restricted the data set to regular cycling women yet do not include the clear, readily used, definition of eumenorrheic. The authors should detail which definition they have used in the abstract. Further comments on this below.

Ethics statement: The authors have stated that it was clarified that this was a human exempt study, could the authors make it explicitly clear that whilst this study was exempt that ethical approval was still sought?

Introduction

Line 123: ‘The events set…’ this depicts that it was the lockdowns that caused the concerns about the pandemic alone, in fact there were many different situations and events on going across the globe and collectively that raised the levels of concern. I would suggest re structure to ensure this comes through in the text.

Line 128: …’ability to reproduce’ I can see the authors need to introduce this concept to the text at this point as to the purpose of their manuscript however the placement of this here does not fit. The ability to reproduce was certainly not an immediate outcome of the pandemic nor do we yet have the scientific data to suggest this is a consequence of the long COVID for example. I would suggest either removing or expanding here on how the pandemic directly affects the ability of people to reproduce.

Line 139: …’transmitted generationally’ please provide a reference here that demonstrates the affects of stress on the menstrual cycle can be transmitted between generations.

Line 147: typo.

Line 157-161. The last sentence of this section needs to be re worded, it is not coherent in its current format. This sentence is making quite assumptive statements too I would suggest being more direct in making your point and include references.

Line 177: Reasonably certain? Can the authors provide clarification or quantification to this?

Methods:

The main issue here is the lack of clarity on determining ovulation. This is one of the main aims/outcomes of the study yet there are no specific details on how ovulation was detected through the app. There are some suggestions of basal body temperature, but no detail. This is imperative to be provided. Were all users testing temperature at same time of day? Were they using the same device? Did they all test the temperature regularly? If not how often? Without such information it makes the data seem quite questionable as to how the results can be deemed valid and/or appropriate.

Line 239. The definition of normal cycles often used in research is 21-35days long, please could the authors explain why they chose 24 days as the shortest cycle to include?

Line 254-255: The covariates stated are all reasonable but again no information of the questions as to how this was collected or categorised. Please provide such detail.

Results

The display of data is very poor. Figure 1 in particular has different coloured text/font/size. The key does also not correspond with the data being shown. Please revise thoroughly.

The symbols used in the tables to display significance, it is not clear which category this is referring to compared to which other category. Please revise.

Table IV is also quite confusing as to whether this is referring to categories of stress or not. I would suggest changing to a clearer display method or altering the table/better legends.

Line 293: include the significance value.

Lines 299-301 The authors state a significant change in cycle length yet looking at the numbers this changes by 0.5 day, the authors need to clarify the length of cycles users inputted as it seems somewhat strange that a user would have a cycle length of 29.5 days.

Line 301: Please include here the number of ‘average incidence’ of ovulation.

Line 319: Indicates that more variable cycles came from older participants, yet the authors have not discussed later that this could be linked to peri menopause as well. I would suggest mentioning this as a possibility within the discussion.

Discussion:

As per above there needs to be more a more detailed discussion of the possible limitations which should include the ovulation reliability, possibility of peri menopause, as well as the possible bias to the data set as already included. I would suggest a specific limitation section should be included as the whole basis of the data collection is reliant upon people taking their own ovulation testing and truthfully inputting the data. There are many studies that have discussed the inaccuracies of self-report data but none have been acknowledged or references. Please include such.

Line 368: In a May 2020…Please re word.

A clearer conclusion/future work section needs to be included, the manuscript ends somewhat abruptly.

Reviewer #2: This paper is quite interesting, and addresses a very commonly asked question by women during the pandemic. The authors do discuss the limitations of their paper, as far as the study group (Highly educated, presumably well off, young women, in general) but they have a considerable data base. And their negative findings are very interesting. As an ancillary question, which has nothing to do with the current study: I would ask, perhaps from a sociologist or psychologist, why this group of women in 2019 (going back to their control group) were experiencing so much stress? If I read the numbers correctly, almost half (46.2%) of the women studied stated that they were either very or extremely stressed in 2019. Why? But I think the study as is is quite succinct and helpful.

Reviewer #3: This is a very important and time-sensitive topic that has received a lot of media attention for anecdotal report of menstrual changes during the COVID-19 pandemic. It is great to have data to start to answer some of these questions. This manuscript contains valuable information but needs editing and some clarifications to make it suitable for publication.

Overall comments:

1. The entire paper is too long and wordy. It would benefit from editing, especially in the Abstract, Introduction, and Discussion. More details below.

2. Better clarification is needed about the definition of anovulation. While it appears that you are using definitions of ovulation/anovulation as defined by a confidential algorithm, it is confusing because regular menses and cycle length are typically used as proxies for ovulatory cycles, and yet anovulation and abnormal cycle length and prolonged menses are presented as completely different concepts here.

3. Similarly, you seem to imply that regular menstrual cycles was an inclusion criterion but then report anovulatory cycles in this cohort. I believe that you are excluding amenorrhea or “very irregular” cycles but need to clarify these concepts and definitions better.

Abstract:

-Much too long. The abstract should not be longer than 300 words and this seems much longer.

-Subheadings are typically Background, Methods, Results, Conclusions. There is no summary/conclusions in your abstract.

-Limitations are not included in the abstract.

-Have never seen an Extended Abstract before. Was this specifically requested? If not, please remove. Your abstract should speak for itself and lead readers to read the entire paper.

Introduction:

-Too long, should not be more than 2-3 paragraphs. Needs editing.

-Last paragraph should be moved to Methods.

-Section should end with a clear statement of the objective of this study.

Methods

-The Inclusion/Exclusion criteria section is long-winded and confusing. Needs to be tightened up and clearer. The most confusing part is the exclusion of those with any cycle >90 days “to account for any undiagnosed amenorrhea or ovulatory dysfunction.” This needs to be clarified. I believe what you mean is that you excluded anyone with secondary amenorrhea, as defined by any cycle >90 days. You did not exclude anyone with irregular cycles, as those would have been anyone with cycles <24 or >38 days, per the FIGO definitions you cite on the next page. As such, the line in the Outcomes section about “… the proportion of users experiencing anovulation, abnormal cycle length, and prolonged menstruation among regularly cycling app users …” seems contradictory. The phrase “regularly cycling app users” should be eliminated throughout the manuscript, as you are not necessarily including those with regular cycles, just those who are not amenorrheic.

-Why were women with endometriosis excluded? What is the evidence for endometriosis affecting menstrual regularity?

-I assume that women using this app are not on any hormonal birth control method, but that should be explicitly stated.

-The last 3 sentences of the Inclusion/Exclusion section are redundant and should be removed.

-Outcome section: Please clarify the line “which may be supplemented by positive urinary LH tests.” Was this done for all women?

Results

-Be consistent in the text and tables about the numbering system for tables. You use Arabic numbers (e.g. 1, 2) in the text but Roman numerals (e.g. I, II) for the tables themselves.

-If you state that there is a “significant” difference in the text and include numbers/percentages for the groups, then you should also include p-values in the text. This is very inconsistent.

-There is no reason to include both Figure 1 and Table 3, as both give the same information. I think it is presented better in the figure.

Discussion

-This is too long. The purpose of the Discussion is to summarize and interpret your most significant findings and compare them to prior data. You should not rehash all the Results.

-You take almost 2 full pages at the end of the Discussion to make the same point. 1) This should be significantly shortened. 2) You must also consider the possibility that routine stress, as experienced by many during COVID-19, is different than the stress experienced during, for example, war and that COVID-19 might not have caused significant physiologic stress and might not have had any affect on menses and ovulation.

-The final paragraph is out of place as the concluding paragraph. Shorten to 1-2 sentences as additional limitations and add a true Conclusions final paragraph to the paper that summarizes the importance of your findings and next steps.

Tables

-Consistent numbering, per comment above

-Why does Table 2 have an * in 1 place? What does this mean? Needs to be defined under table.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Mary Jane Minkin, MD

Reviewer #3: Yes: Beth I. Schwartz, MD

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 Oct 20;16(10):e0258314. doi: 10.1371/journal.pone.0258314.r002

Author response to Decision Letter 0


25 Jun 2021

Please see the separately attached reviewer response for a point-by-point address of each correction.

Attachment

Submitted filename: COVID Cycles 2021_06_09 - PLOSONE Review Response.docx

Decision Letter 1

Alessio Paffoni

13 Jul 2021

PONE-D-21-15151R1

Detecting variations in ovulation and menstruation during the COVID‐19 pandemic, using real‐world mobile app data

PLOS ONE

Dear Dr. Nguyen,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

Both reviewers agree that the text has improved. However, some aspects remain that require attention by the authors as specified below.

==============================

Please submit your revised manuscript by Aug 27 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Alessio Paffoni, PhD

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: No

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you to the authors for address the majority of the queries put forward by both reviewers. The manuscript provides a lot more detail in its current format. I do still have some concerns/queries that need to be addressed which I have provided in the comments below (see marked comment 2).

Original comments: Methods: The main issue here is the lack of clarity on determining ovulation. This is one of the main aims/outcomes of the study yet there are no specific details on how ovulation was detected through the app. There are some suggestions of basal body temperature, but no detail. This is imperative to be provided. Were all users testing temperature at same time of day? Were they using the same device? Did they all test the temperature regularly? If not how often? Without such information it makes the data seem quite questionable as to how the results can be deemed valid and/or appropriate.

Original response: We have provided more detail about how basal body temperatures are recorded as follows: “Anovulatory cycles were defined according to the Natural Cycles app’s proprietary algorithm. The algorithm identifies ovulation retrospectively based on the first day of menstruation and basal body temperatures, which may be supplemented by positive urinary LH tests. Basal body temperatures are recorded each morning using a thermometer sensitive to the hundredth place, and with measures excluded if the user reports any illness or changes in sleep that might influence basal temperatures. Users of Natural Cycles record basal temperatures for approximately 70% of the days. To reduce the risk of misidentifying ovulations, the algorithm reports ovulation by rising basal body temperature only if the average temperature from three consecutive calendar days is greater than the woman’s follicular phase average and her baseline average across all data entries, as well as consistent with her luteal phase average [15]. If no temperature rise is observed and the data quality and quantity is deemed sufficient, the cycle is flagged as anovulatory. Cycles with low data quality or many missing data points are excluded by the analysis.”

Comment 2: Thank you for providing this important detail. Could the authors please clarify the distribution of reporting further. The authors state that basal temperatures are recorded approximately 70% of the days, it is imperative to provide detail on this, what days were reported more than others? Did it vary between participants using the app? If certain days were continually omitted in some cases then this would contribute somewhat to the outcomes of the analysis.

Comment 2: The authors also state that cycles will low data quality or any missing data points are excluded by the analysis. Again the detail is required here. What was the authors definition of low quality data (is this linked to the statement prior to this or something else?) and provide exactly how many missing data points that then meant the cycle was excluded.

Original comments: Ref (line 239) is from 2007 there should be more recent ref used to support the use of 24 days.

Original response: Line 239. The definition of normal cycles often used in research is 21-35 days long, please could the authors explain why they chose 24 days as the shortest cycle to include? • We used criteria from FIGO (https://pubmed.ncbi.nlm.nih.gov/17362717/), which are based on more recent population estimates at the 5th and 95th percentile. Had we used 21 days, we would have significantly underestimated frequent menstruations and potentially been less likely to find a difference.

Comment 2: The reference the authors have provided is dated 2007, almost 14 years ago. There are many updated references that should be provided for clarification of their use of 24 days as the shortest cycle length to include. Please update.

Comment 2: Figure 1 is still very poor. I would suggest removing the background lines. Ensure all text is black font to align with the manuscript.

Original comment: Lines 299-301 The authors state a significant change in cycle length yet looking at the numbers this changes by 0.5 day, the authors need to clarify the length of cycles users inputted as it seems somewhat strange that a user would have a cycle length of 29.5 days.

Original Response: The cycle lengths are averaged from the number of cycles that are contributed over the study periods, which is why the averages are fractions of whole numbers.

Comment 2: My comment has still not been fully addressed here I apologise if I was not clear originally. My query is regarding the reporting of the cycle through the app. Generally, participants would report their cycle lengths in whole days. Is it feasible for a participants/users cycle to be reported for example as 29.5 days if they reported precisely in the day when bleeding started/ stopped and began again for the next cycle? Is this something that is feasible through the app? I fully appreciate and understand that the 0.5 increase would be an average.

Original comment: Discussion: As per above there needs to be more a more detailed discussion of the possible limitations which should include the ovulation reliability, possibility of peri menopause, as well as the possible bias to the data set as already included. I would suggest a specific limitation section should be included as the whole basis of the data collection is reliant upon people taking their own ovulation testing and truthfully inputting the data. There are many studies that have discussed the inaccuracies of self-report data but none have been acknowledged or references. Please include such.

Original response: We note that the reviewer is very focused on a biased critique of the app being unable to accurately predict ovulation. We note that ovulation in all the possible ways in which it can be measured non-invasively are soft markers. Even the gold standard of using ultrasound cannot determine if a ruptured follicle has released an egg. To suggest that our research would be improved via a study that demanded transvaginal ultrasound and routine LH kit testing would be infeasible at our sample size. Ovulation prediction is limited, which is why our investigation additionally includes other menstrual parameters aimed at increasing our ability to detect changes related to the pandemic. The reviewer’s comments do not acknowledge that this data set is one of largest to collect these data and that any data collection performed in a clinical setting would be subject to even greater limitations. Any paper-collected self-report would be even less reliable and more likely to reflect recall bias, which our methods avoid by requiring daily data entry. Abiding by the reviewer’s request would turn a strength of our methodology into a weakness.

Original response: That being said, we have added the following broad limitation acknowledgement in a separate limitations section as follows: “The findings from this study are broadly limited by self-reported data. However, we believe that these data are collected from a group of individuals who are using the Natural Cycles app for its intended purpose of pregnancy prevention, rather than for the purposes of this research, such that these data are expected to be reliable.”

Comment 2: I have already acknowledged the large data set in the very first comment to the authors. Nor have I ever suggested an alternative more invasive method be used such as transvaginal US. I have focused on the ovulation due to the previous lack of detail provided, therefore making it difficult to understand as a reader how an app would predict ovulation. The information that has been added has certainly now helped the manuscript. However, it is still important to acknowledge the limitations, as with all original research. The reporting of basal body temperature, despite being daily, is still reliant on self-report as it is trusting participants will not falsify any data, as is the same with all self report data. It does not mean to say your data is not reliable it is just something that cannot be guaranteed. Therefore, I would suggest adding to your acknowledgment of this with a reference regarding the reliability of self-report data.

Reviewer #3: This paper is significantly improved but still needs some edits to make it suitable for publication. Specific comments below.

Abstract: Much better! Only comment is that I would reword the last sentence both here and in the Discussion to read "... their experiences of the COVID-19 pandemic might differ ..."

Introduction: Significantly improved. You still need to end this section with a clearer sentence on the objective of the study. Most papers end with a line that literally states "The objective of this study is to ..."

Methods:

-Page 6, Lines 123-125: This seems unnecessary and like an ad for the app.

-Page 6, Inclusion/Exclusion Criteria: Were cycles of users who became pregnant during use included?

-Page 8, Lines 171-172: This was stated earlier and is redundant.

Results:

-Relationship status in Table 1 seems to need a +.

-It is unusual to present 95% CI for means, instead of SD. Please present that in the Methods and address why.

-I continue not to think that both Figure 1 and Table 3 are both necessary, especially as there are no differences and this can be just addressed in the text.

-Page 13, Lines 243-247: Creation of the model should be in the Methods section. However, I do not see that you actually created a model. Table 4 is just a list of significant associated risk factors and is not a model.

Discussion:

-My biggest concern is that you take 1.5 pages of a 2.5 page discussion on a theory that defends your lack of expected results. It is ok to mention this but not to take up this much space on it. Instead, spend some time discussing the implications of your actual results.

-Page 16, Lines 284-289: I don't understand how this is relevant.

-Page 17, Lines 312-315: Too much rehashing. Would rewrite as "Only 56.9% of users .... bias; however, this sub-analysis ..."

-Page 18, Limitations: You could consider the possibility that the chosen time points were not enough to capture changes in menses and ovulation, as it might take more time to see the physiologic effects on stress, as COVID-19 did not start until mid-March 2020 and most thought it would be very temporary at that time.

Conclusion:

-Page 19, Line 350: Per my comment in the Abstract, would change this to "may"

-Suggest ending with a line "Future research is needed to ..."

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: Yes: Beth I. Schwartz, MD

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 Oct 20;16(10):e0258314. doi: 10.1371/journal.pone.0258314.r004

Author response to Decision Letter 1


2 Sep 2021

Please see the separately attached reviewer response for a point-by-point address of each correction.

Attachment

Submitted filename: COVID Cycles 2021_09_01 - PLOSONE Review Response.docx

Decision Letter 2

Alessio Paffoni

20 Sep 2021

PONE-D-21-15151R2Detecting variations in ovulation and menstruation during the COVID‐19 pandemic, using real‐world mobile app dataPLOS ONE

Dear Dr. Nguyen,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================The revised manuscript has been appreciated. Minor revisions are needed at this point. I believe that table 3 and Figure 1 can remain in the final text.

Please submit your revised manuscript by Nov 04 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Alessio Paffoni, PhD

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for addressing my points.

The only remaining point is regarding the statement 'many missing data points' when discussing the data that is included for final analysis. Please could the authors change this precise statement to make more quantifiable and perhaps in relation to the '10 temperature readings' that are stated to have been required earlier. This is my only remaining query.

Reviewer #3: Thank you for your thoughtful and substantial edits. The paper is now in very good shape! Only 2 comments:

1) Would recommend slight edits to the Limitations section. This seems to be a continuation of the paragraph above, rather than a clear statement of the limitations of the study, which does not clearly begin until >1 paragraph later. You could even just remove the Limitations subheading, and it would read better.

2) I continue to note that I do not think that both Figure 1 and Table 3 are both necessary, especially as there are no differences and this can be just addressed in the text. However, this should not stop the paper from being published in its current form.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: Yes: Beth I. Schwartz, MD

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 Oct 20;16(10):e0258314. doi: 10.1371/journal.pone.0258314.r006

Author response to Decision Letter 2


21 Sep 2021

Dear editors and reviewers,

Thank you for your favorable review of our manuscript, “Detecting variations in ovulation and menstruation during the COVID-19 pandemic, using real-world mobile app data.” We acknowledge the reviewers’ comments and have made revisions as appropriate with responses below. Please find our review responses highlighted in yellow.

Reviewer #1: Thank you for addressing my points. The only remaining point is regarding the statement 'many missing data points' when discussing the data that is included for final analysis. Please could the authors change this precise statement to make more quantifiable and perhaps in relation to the '10 temperature readings' that are stated to have been required earlier. This is my only remaining query.

Thank you for your query about the ambiguity in our manuscript. However, the explanation of how cycles are excluded is not as simple to explain as to provide a cutoff or threshold value. The quantity of missing data that is acceptable is a function as well of the stability of the user’s data as well. To help the reader understand this and reduce the ambiguity in the manuscript we have revised as follows (we have removed mention of “many missing data points”:

“If no temperature rise is observed and the data quality and quantity is deemed sufficient, the cycle is flagged as anovulatory. Of note, users with stable measurements (e.g., small day-to-day variations in the same cycle phase), require fewer data points for the Natural Cycles algorithm to draw conclusions about changes in the basal body temperature. Cycles with low data quality (e.g., high day-to-day temperature variability) or data that is insufficient to detect or exclude an increase in the basal body temperature are excluded by the analysis.”

Reviewer #3: Thank you for your thoughtful and substantial edits. The paper is now in very good shape! Only 2 comments:

1) Would recommend slight edits to the Limitations section. This seems to be a continuation of the paragraph above, rather than a clear statement of the limitations of the study, which does not clearly begin until >1 paragraph later. You could even just remove the Limitations subheading, and it would read better.

- Thank you for your comment, for which we’ve removed the “Limitations” subheading to help with the flow of the discussion section.

2) I continue to note that I do not think that both Figure 1 and Table 3 are both necessary, especially as there are no differences, and this can be just addressed in the text. However, this should not stop the paper from being published in its current form.

- Per editor, Fig 1 and Table 3 may remain in final text.

Attachment

Submitted filename: COVID Cycles 2021_09_21 - PLOSONE Review Response.docx

Decision Letter 3

Alessio Paffoni

24 Sep 2021

Detecting variations in ovulation and menstruation during the COVID‐19 pandemic, using real‐world mobile app data

PONE-D-21-15151R3

Dear Dr. Nguyen,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Alessio Paffoni, PhD

Academic Editor

PLOS ONE

Acceptance letter

Alessio Paffoni

12 Oct 2021

PONE-D-21-15151R3

Detecting variations in ovulation and menstruation during the COVID-19 pandemic, using real-world mobile app data

Dear Dr. Nguyen:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Alessio Paffoni

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Attachment

    Submitted filename: COVID Cycles 2021_06_09 - PLOSONE Review Response.docx

    Attachment

    Submitted filename: COVID Cycles 2021_09_01 - PLOSONE Review Response.docx

    Attachment

    Submitted filename: COVID Cycles 2021_09_21 - PLOSONE Review Response.docx

    Data Availability Statement

    The data underlying this study are publicly available at doi.org/10.5061/dryad.0cfxpnw2k.


    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES