Working 9 to 5, not the way to make an academic living: observational analysis of manuscript and peer review submissions over time

Abstract

Objective

To determine whether researchers are submitting manuscripts and peer reviews to BMJ journals out of hours and whether this has changed over time.

Design

Observational study of research manuscripts and peer reviews submitted between 2012 and 2019 for which an author’s address could be geocoded.

Setting

Online BMJ submission systems for two large general medical journals.

Main outcome measures

Manuscript and peer review submissions on weekends, on national holidays, and by hour of day (to determine early mornings and late nights). Logistic regression was used to estimate the probability of manuscript and peer review submissions on weekends or holidays.

Results

The analyses included more than 49 000 manuscript submissions and 76 000 peer reviews. Little change over time was seen in the average probability of manuscript or peer review submissions occurring on weekends or holidays. The levels of out of hours work were high, with average probabilities of 0.14 to 0.18 for work on the weekends and 0.08 to 0.13 for work on holidays compared with days in the same week. Clear and consistent differences were seen between countries. Chinese researchers most often worked at weekends and at midnight, whereas researchers in Scandinavian countries were among the most likely to submit during the week and the middle of the day.

Conclusion

The differences between countries that are persistent over time show that a “culture of overwork” is a literal thing, not just a figure of speech.

Introduction

The “publish or perish” mantra is well established in academic circles, but do researchers have to sacrifice leisure time to achieve this ideal? Academic productivity—or the fear of the lack of it—is a focus for many senior university managers, but the data show an explosion over time in the number of articles in health and medical journals,¹ with early career researchers publishing more since 1980 and a general increase in the number of authors per paper.² Although universities benefit from the prestige conferred by a rise in the rankings, some people have questioned whether this is a desirable outcome for individual academics and academic culture.³ The pressure to publish creates a vicious cycle whereby publishing more papers has the potential to push the university up in the rankings, which, in turn, puts more pressure on academics to keep working to maintain their university’s place in the rankings (and perhaps their own jobs). At the same time, universities in many countries have seen a massive increase in student numbers and teaching loads that squeeze the available time for research.⁴

Anecdotally at least, academics and medical students are working longer hours to accommodate the demands of research and publishing.^{5 6} Critics of higher education have long complained that research is being pushed to the margins by teaching and administrative tasks. Hang around in any academic department’s kitchen and you will hear complaints about overflowing inboxes, demanding students, and requests from administrators. A casual glance at the so called “academic quit lit,”⁷ in which departing academics list all the reasons they do not want to stay in the profession, suggests that some academics are drowning under administrative tasks.

The constant connectivity of our digital world leads to what Gregg called the “presence bleed,” whereby little difference exists between home and work in the capacity to do work.⁸ The ability to work anytime and anywhere can make setting boundaries difficult. The temptation to overwork is exacerbated in many countries by the precarious nature of academic labour and the accompanying anxiety that people feel to produce or lose their job.⁹ The academic overwork problem has consequences for individuals, but also for the publishing system as a whole. Erickson suggested that academic overwork is eroding the “gift economy” of peer reviewing that is necessary to keep the publishing system moving.¹⁰

For all this talk, little empirical research has been done on whether academics are actually working longer hours; what does exist is only retrospective self report, via interview or survey.¹¹ This paper sets out to start to fill this gap by looking at the timing of manuscript and peer review submissions to The BMJ and BMJ Open. If research writing and reviewing is being squeezed into leisure time, we would expect to see an increase over time in submissions of manuscripts and peer reviews on weekends and holidays.

Methods

This was an observational study using routine data from the journal submission systems of The BMJ and BMJ Open, two large international general medical journals based in London. We analysed the transmission dates and times of research manuscripts and peer reviewers’ reports between 1 January 2012 (when the ScholarOne submission system was implemented) and 5 April 2019 (2651 days).

We included research papers and research methods and reporting articles for The BMJ and original research and protocols for BMJ Open. We limited the submissions to first version manuscript submissions, so excluding resubmissions. We excluded manuscripts transferred between the two journals to avoid double counting (1.7% (918/52 744) for manuscripts and 2.1% (1706/81 874) for peer reviews). For the peer reviewers’ data, we included reviews of research papers, research methods and reporting articles, and protocols. We excluded reviews by The BMJ’s statistical advisers, as they are paid and so may have a different pattern of working hours (1.2%; 1007/83 177). One author (SS) used their names to excluded these reviewers before the data were anonymised. We also excluded reviews by patients and the public (0.4%; 296/82 170).

The variables were the date and time of transmission (in Eastern Standard Time), the submitting author’s or peer reviewer’s address (city, state, and country), and the article type. We transformed dates and times to local times by geocoding the addresses and then extracting the time zone. We used the Google Geocoding API and the R library “ggmap” for geocoding,¹² and added the time zones by using the R library “geonames.”¹³ We excluded 0.02% (12/52 756) of manuscripts and 4.3% (3709/86 886) of peer reviews with missing address data, as well as a further 0.1% (38/50 110) of manuscripts and 0.8% (612/78 114) of peer reviews with partially complete address data that could not be geocoded. To have sufficient data to examine differences between countries, we excluded countries with fewer than 100 manuscripts (3.3%; 1716/51 826) or peer reviews (2.6%; 2054/80 168).

Outcome definitions

Ninety eight per cent of the data came from countries that use a Saturday to Sunday weekend. In some Muslim majority countries, the weekend is Friday to Saturday. We used data on public holidays from Nager.Date, which has data for 100 countries.¹⁴ We examined only national holidays and excluded regional holidays (for example, the Picnic Day holiday in the Northern Territory of Australia).

Statistical methods

We wrote a study protocol and had pre-planned analyses (available at https://github.com/agbarnett/weekends). Our primary aim was to examine changes over time, and we anticipated that we would see an increase over time in weekend and holiday work because of the growing pressure on researchers. We also expected to see differences between countries. We ran separate models for The BMJ and BMJ Open because we thought that important differences might exist between the characteristics of researchers—for example, more clinical researchers for The BMJ—which could have influenced typical working hours.

We tested for an increase in the probability of weekend manuscript and peer review submissions over time by using logistic regression. We grouped the data by consecutive weeks (Monday to Sunday), and the dependent variable was the number of weekend manuscripts/reviews that week, with a denominator of the total number of manuscripts/reviews that week. This approach meant that we examined a relative change in the probability of weekend manuscripts or peer reviews and not an absolute change that could simply happen because of a change over time in the absolute number of manuscripts or peer reviews submitted. The logistic regression models included country specific intercepts to model between country differences in the weekend probability. In alternative models, we included country specific trends to examine between country differences in trends. We assessed the addition of country specific trends by using the deviance information criterion, with the best model chosen using the lowest criterion.¹⁵

For holidays, we used a similar approach to the weekend analysis, but we compared holidays with other days in the same week that were not holidays. We excluded weeks without holidays from this analysis.

To examine submissions early in the morning and late at night, we originally used a dichotomous approach with out of hours defined as 6 pm to 7 am. However, this simple dichotomous approach could miss important differences between countries with differing work hours, so we instead examined the number of manuscripts or peer reviews for each hour of the 24 hour clock. We modelled an overall pattern common to all countries plus a smooth difference for each country, an approach successfully applied in time series.¹⁶ We used a Poisson regression model of the number of manuscript or peer review submissions in each hour in each country. We modelled the smooth difference for each country by using a sinusoid that allowed a single peak in submissions at any time during the 24 hour clock, with a nadir 12 hours later.¹⁷ We summarised the results across countries by plotting the timing of the peak against the height of the peak expressed as a probability ratio. The full details of this regression model are in the supplement.

We did not use a sample size calculation but instead included all the available data. We created an initial set of results by using randomly generated submission dates and times, with the aim of making any changes to the statistical methods before viewing the real data.

The regression models used a Bayesian paradigm because this gives 95% credible intervals that have a 95% probability of containing the true estimate, whereas classical 95% confidence intervals do not.¹⁸ We used non-informative normal priors for regression estimates and non-informative γ priors for inverse variances. The data management and analyses used R (version 3.6.0). The code, data, and checks of convergence for the Bayesian estimates are available at https://github.com/agbarnett/weekends.

Patient and public involvement

This study was an analysis of routine non-clinical data. Patients and the public were not involved in setting the research question, designing the study, the conduct of the study, the interpretation of the results, or our dissemination plans. Dissemination to participants is not applicable.

Results

The final analyses used 49 464 submissions and 76 678 reviews (see flow diagrams in supplement). Peer reviews were more often submitted on weekends than were manuscripts. The average probability of a manuscript being submitted at the weekend for both journals was 0.14, and for a peer review it was 0.18 (table 1). Peer review submissions during holidays had average probabilities of 0.13 (The BMJ) and 0.12 (BMJ Open), which were higher than the probabilities for manuscripts of 0.08 (The BMJ) and 0.10 (BMJ Open).

Table 1.

Mean probabilities of manuscripts and peer reviews being submitted on weekends and holidays

Outcome	Type	Journal	Mean (95% CI)
Weekend	Manuscripts	BMJ	0.14 (0.12 to 0.15)
		BMJ Open	0.14 (0.13 to 0.15)
	Peer reviews	BMJ	0.18 (0.16 to 0.20)
		BMJ Open	0.18 (0.17 to 0.20)
Holidays	Manuscripts	BMJ	0.08 (0.07 to 0.10)
		BMJ Open	0.10 (0.08 to 0.11)
	Peer reviews	BMJ	0.13 (0.11 to 0.15)
		BMJ Open	0.12 (0.11 to 0.13)

Holiday probabilities are relative to other days in week with at least one holiday.

We observed no difference over time in the probability of manuscripts or peer reviews being submitted on weekends or holidays (table 2). Most of the differences were close to zero with narrow credible intervals, indicating little change over time.

Table 2.

Estimated changes per year in probability of manuscripts and peer reviews being submitted on weekends and holidays

Outcome	Data	Journal	Mean difference (95% CI)
Weekend	Manuscripts	BMJ	–0.2 (–1.3 to 0.9)
		BMJ Open	0.1 (–1.0 to 1.2)
	Peer reviews	BMJ	0.2 (–1.3 to 1.8)
		BMJ Open	0.4 (–0.9 to 1.8)
Holidays	Manuscripts	BMJ	–0.0 (–0.3 to 0.2)
		BMJ Open	–0.0 (–0.3 to 0.2)
	Peer reviews	BMJ	–0.0 (–0.3 to 0.3)
		BMJ Open	0.0 (–0.2 to 0.2)

Results are absolute probability difference (current year minus previous year) and 95% credible interval.

Figure 1 shows the differences between countries in probabilities for manuscript submissions to The BMJ on weekends and holidays, indicating a large variation between countries. The lowest mean probabilities for weekend submissions were in India and the highest in China. For holidays, the lowest mean probability was in Canada and the highest in Belgium. The credible intervals are narrower for countries with more data.

Fig 1.

Country specific probabilities for manuscript submissions to The BMJ on weekends and holidays. Dots are means, and horizontal lines are 95% credible intervals. The dotted vertical line is the study average. Additional plots for BMJ Open and for peer reviews are in the supplement

Figure 2 summarises the differences between countries for the combinations of the two journals, manuscripts or peer reviews, and weekends or holidays. China had high probabilities of manuscript and peer review submissions on weekends (0.22 to 0.23), but lower probabilities on holidays (0.08 to 0.12). The Scandinavian countries Norway, Denmark, Finland, and Sweden had some of the lowest probabilities of working on weekends (0.10 to 0.17). Italy had relatively high probabilities of working on the weekends (0.12 to 0.20) but low probabilities for the holidays (0.08 to 0.12). Belgium had the highest average probability of working during the holidays (0.09 to 0.18).

Fig 2.

Each country’s probability of submission on weekends and holidays. Countries are ordered according to their average probability. Plot includes only countries with more than 100 submissions or peer reviews for all eight outcomes

We observed a strong diurnal pattern in submission of manuscripts and reviews (fig 3). The peak time for submissions was the end of the working day (3-5 pm). A small local peak occurred at midday, which may correspond to people working during lunch. China and Japan had the highest probabilities for submitting manuscripts and peer reviews late at night (fig 4). Manuscripts from China were 86% higher than the average during the hours of midnight to just before 1 am, and peer reviews were 57% higher during the same time. The three Mediterranean countries France, Italy and Spain together with Brazil had relatively late peak times (mostly after 6 pm), but with probability ratios much smaller than those in Japan and China (all <25% increase). Denmark had a relatively high probability of submission of manuscripts during the middle of the day (57% higher than the average).

Fig 3.

Percentages of manuscripts and peer reviews by hour of day and journal

Fig 4.

Estimates of what time countries are most likely to submit manuscripts or peer reviews compared with average. X axis is peak time and y axis is probability ratio, which gives relative change from average

Discussion

Our primary aim was to examine the change over time in out-of-hours work. We hypothesised that a growing pressure on researchers would mean a growing trend in unsociable work hours. However, the lack of any trends over time was remarkably consistent, indicating stable working patterns since 2012. The levels of out-of-hours work were high, however, with average probabilities of 0.14 to 0.18 for work on the weekends and 0.08 to 0.13 for holidays. Digital technology enables academic writing to happen anywhere, anytime, meaning that work can easily invade leisure time. The constant ability to work may be contributing to burnout.¹⁹

A positive side to being able to work at home is that researchers may be time shifting important tasks to quieter times and spaces, away from their noisy office, as a recent study of academic work suggests.²⁰ The creative work of writing a paper may be easier to do at the weekend and may be something that many researchers enjoy. A recent study found that doctors in Germany often did continuing medical education at the weekend, early in the morning, and late at night.²¹

We observed large and interesting differences between countries (fig 1, fig 2, and fig 4). The country someone lives in was the strongest predictor of whether they will work out of hours, suggesting that a “culture of overwork” is a literal thing, not just a figure of speech. People in countries known to have family friendly working conditions and unionised academic workforces were less likely to work out of hours. China was a clear outlier in terms of weekends and late nights and is known to have a hard working academic culture.²² Clear incentives exist for researchers to “publish or perish” in China, as hiring and promotion are based on the quantity of papers in journals with high impact factors, and financial bonuses are given for published papers.^{23 24} At a national level, targets exist for international league tables that are largely driven by the quantity rather than the quality of research.²³

Policy implications

The higher probability of peer reviews (compared with manuscripts) being submitted during weekends and holidays suggests that some researchers may feel unable to complete peer reviews at work. Research institutes and universities should acknowledge that peer reviewing manuscripts is an expected part of a researcher’s job and not something that researchers need to do in their own time. The assessment and reward structures at universities should include peer review as a key measure.²⁵

Although journals have less responsibility towards researchers than do employers, they should recognise their part in the culture of overwork. Journals could state that no expectation exists for submissions to occur out of hours, and this message would apply equally to authors, reviewers, and editors. We are not convinced that such a solution would be effective, but we suggest it because if peer review is something that becomes measured and acknowledged, this task is likely to become yet another metric for academics to try to achieve.

Limitations of study

Our study has some limitations. Firstly, submission of a manuscript or peer review is the final step in an often long process; even when it is submitted out of hours, the bulk of the thinking and writing may have been done during working hours. Secondly, potential sources of measurement error for the submission times exist that we cannot quantify. Some authors may have been outside their country when they submitted their manuscript or peer review (for example, at a conference), meaning that they would be out of their time zone. Researchers may have inaccurately completed their address or may not have updated their address after moving country. Some researchers have multiple affiliations, and their primary time zone may differ from the one they are working in.

Thirdly, we did not find an increase over time, but a large increase in weekend work may have occurred before 2012, during the time of the exponential increase in the volume of research published.¹ We examined a relative increase in out-of-hours work, and we would expect an increase in the absolute number of out-of-hours peer review and manuscript submissions if the total number of publications continues to increase over time. Fourthly, we did not have the age or sex of the authors or peer reviewers, so we could not examine differences by these characteristics. We also did not have the researchers’ names, so we could not examine a within researcher change over time in out-of-hours work. Finally, The BMJ and BMJ Open are large general medical journals, but they will not be representative of all researchers or of all researchers in health and medicine. Hence, we should be careful about generalising to other areas of research.

What is already known on this topic

• University ranking tables are being used as a management tool to encourage researchers to “publish or perish”

• Anecdotally, researchers are working longer hours to accommodate the demands of research and publishing

• Many complaints are made about academic and clinical overwork

What this study adds

• The “weekend” is a misnomer for many researchers

• Large differences exist between countries in working hours, with China as a clear outlier in working weekends and late nights

Web extra.

Extra material supplied by authors

Supplement