Cross-sectional study of the relationship between women’s representation among editors and peer reviewers in journals of the British Medical Journal Publishing Group

Abstract

Objectives

To investigate whether there is an association between women’s representation as peer reviewers and editors of medical journals.

Methods

In this cross-sectional study, the gender of editors and peer reviewers of journals of the British Medical Journal Publishing Group (BMJ-PG) in 2020 was determined based on given names. Trends over time were analysed for the BMJ between 2009 and 2017.

Results

Overall, this study included 47 of the 74 journals in the BMJ-PG. Women accounted for 30.2% of the 42 539 peer reviewers, with marked variation from 8% to 50%. Women represented 33.4% of the 555 editors, including 19.2% of the 52 editors-in-chief. There was a moderate positive correlation between the percentage of women as editors and as reviewers (Spearman correlation coefficient 0.590; p<0.0001). The percentage of women as editors, excluding editors-in-chief, was higher when the editor-in-chief was a woman than a man (53.3% vs 29.2%, respectively; p<0.0001). Likewise, the percentage of women as peer reviewers was higher in journals that had a woman as editor-in-chief in comparison with a man (32.0% vs 26.4%, respectively; p<0.0001). There was a slight increase in the percentage of women as peer reviewers from 27.3% in 2009 to 29.7% in 2017 in the BMJ.

Conclusions

Women account for less than one in three peer reviewers of medical journals. Women’s representation as peer reviewers is higher in journals with higher percentage of women as editors or with a woman as editor-in-chief. It is, thus, imperative to address the persisting gender gap at all levels of the publishing system.

Strengths and limitations of this study.

• This study included journals from a large and renowned family of journals, which enable including over 40 000 peer reviewers and 500 editors.

• Gender identification based on pronouns for editors enabled considering non-binary gender, even if no their/theirs pronouns were used.

• By relying on a binary assignment of gender based on given names for reviewers, this study failed to account for non-binary gender or gender identities that did not match that of the given name.

• This study used journals from a single publishing family, which might not be representative of all medical journals.

• It is impossible to ascertain whether the observed correlation between women’s representation among editors and peer reviewers is causal.

Introduction

Women’s under-representation in the publishing system, including in medicine, is well-established, with stark gender inequalities among authors of scientific papers, particularly at senior levels.^{1 2} Overall, women account for 20%–40% of first authors and for 15%–30% of last authors.^3–5 Women are also under-represented among editors-in-chief of medical journals and more widely in scientific editorial boards.^{6 7}

Peer reviewers play a pivotal role in the publishing process and exert a strong influence on what research eventually gets published and in what calibre of journal. Peer reviewers also have an important role in ensuring scientific publications adhere to reporting standards and guidelines, particularly those for the incorporation of sex and gender analyses.⁸ Since women as authors are more likely to report sex-disaggregated and gender-disaggregated analyses, women as peer reviewers may also be more likely than men to ensure that sex and gender are adequately handled in medical papers.⁹ Gender inequality among peer reviewers may, thus, have detrimental consequences for progress in medical knowledge and, ultimately, population health. However, the inclusion of women as peer reviewers of medical journals has received less attention, probably due to the lack of detailed publicly available data on peer reviewers.

In addition, although women’s representation among journal editors has been positively associated with women’s representation among authors, this association remains poorly understood for peer reviewers, particularly in medical journals.¹⁰ Indeed, the choice of peer reviewers is influenced by myriad factors, and hence it is uncertain to what extent gender influences editors’ decisions, either consciously or unconsciously.¹¹ Therefore, this study aimed to determine women’s representation among peer reviewers and editors of medical journals, and investigate whether greater women’s representation among editors correlated with greater representation as peer reviewers.

Methods

Data sources and definitions

Among the major families of journals, only the British Medical Journal Publishing Group (BMJ-PG) requires their journals to report annually a list of their contributing peer reviewers.¹² The BMJ-PG is a large family of journals, which covers most medical specialties, as well as other fields of research related to health services (eg, quality improvement and safety). As data for peer reviewers were not publicly available for other publishers, or families of medical journals, they were not eligible for this study. We conducted a systematic search on Google for the list of peer reviewers for each of the journals in the BMJ-PG in 2020. Given names were extracted for all peer reviewers. For all journals of the BMJ-PG, apart the BMJ, data were available only for 2020, and we used those data to investigate current representation of women in the BMJ-PG overall.

In addition, we investigated trends over time in women’s representation using data available for peer reviewers in the BMJ for 2009, 2010 and 2013–2017. Data were not available for the BMJ after 2017. For comparison, the list of peer reviewers in 2010, 2012, 2014, 2016, 2018 and 2020 for two leading medical journals (The New England Journal of Medicine (NEJM) and Journal of the American Medical Association (JAMA)) was also reviewed and given names of reviewers extracted. These two journals were not included in the analysis of BMJ-PG journals.

We used the “genderizeR” package for R to predict the gender of the peer reviewers based on their given names. This software collects data from the internet and includes 38 659 given names from 242 countries across the globe.¹³ A two-step approach was used to determine gender based on given names.¹⁴ First, given names were extracted from full names using a specific feature of the GenderizeR package. Second, the gender of the vector of given names was classified as either woman or man using another feature of the package. When given names could not be recognised and extracted from full names by the software, those reviewers were considered as ‘missing’ and excluded from all analyses.

For each journal, data for editor-in-chief, deputy editors, assistant editors and associate editors were extracted. These are defined as ‘editors’ throughout the manuscript. Their gender was determined based on pronouns and photographs available on the journal website or professional affiliations. Other members of editorial boards (eg, advisory editors, statistical advisors, emeritus editors) were excluded.

Data on the impact factor and CiteScore for 2020 were extracted from the journal website. CiteScore is a measure reflecting the yearly average number of citations of articles published in that journal. This metric was launched in December 2016 by Elsevier as an alternative to the generally used impact factors calculated by Clarivate Analytics and published in the Journal Citation Reports. CiteScore is based on the citations recorded in the Scopus database rather than in Journal Citation Reports, and those citations are collected for articles published in the preceding 4 years instead of 2 or 5. We used these two metrics to assess impact because impact factor was not available for 21 journals, of which 15 had a CiteScore available.

Data analysis

We computed the percentage of women among peer reviewers and editors overall and for each journal. We plotted the association between the percentage of women as peer reviewers and editors, stratified by gender of the editor-in-chief. We computed the Spearman correlation coefficients between the percentage of women as editors and the percentage of women as peer reviewers and between the percentage of women as peer reviewers and the journal impact factor and CiteScore. We compared the percentage of women among peer reviewers and editors according to the gender of the editor-in-chief using Fisher’s exact test. All data analyses used R V.4.0.2 (R Core Team, 2020).

Patient and public involvement

Patients and the public were not involved in this study.

Results

The BMJ-PG publishes 74 journals, of which 47 were included in the analysis because reviewers’ names were not available for 27 journals (online supplemental table S1).

Women as peer reviewers

Overall, women accounted for 30.2% of the 42 539 peer reviewers in 2020 (table 1). There was marked variation in women’s representation across journals (median 31.3%, IQR 24.5% to 38.5%), ranging from 8% for The Journal of the International Society of Arthroscopy, Knee Surgery and Orthopaedic Sports Medicine to 50% in Medical Humanities. No journal had more than 50% women reviewers. Women’s representation among peer reviewers in the BMJ-PG was higher than in the JAMA (28.1%) and the NEJM (18.9%).

Table 1.

Representation of women among peer reviewers and editors of medical journals

BMJ journals	Reviewers (n)	% Women	% Missing	Editors (n)	% Women	Gender of EIC	CiteScore	Impact factor
Annals of the Rheumatic Diseases	529	23.1	0.4	12	25.0	Man	28.7	19.1
BMJ Case Reports	7179	23.1	1.1	11	27.3	Woman	NA	NA
BMJ Global Health	1325	41.1	0.8	16	25.0	Man	5.5	5.6
BMJ Health & Care Informatics	133	34.1	0.8	17	35.3	Man	1.9	NA
BMJ Leader	162	47.8	1.9	14	35.7	Man	1	NA
BMJ Neurology Open	85	32.9	0.0	8	25.0	Man	NA	NA
BMJ Open	13 041	36.4	1.3	14	50.0	Man	3.7	2.7
BMJ Open Diabetes Research & Care	1038	30.8	0.9	8	0.0	Man	3.3	3.4
BMJ Open Ophthalmology	278	30.1	0.7	29	34.5	Man	2.5	NA
BMJ Open Quality	42	39.0	2.4	8	87.5	Woman	1.1	NA
BMJ Open Respiratory Research	340	24.6	1.8	3	0.0	Men (2)	4	NA
BMJ Open Science	43	37.2	0.0	18	44.4	Woman	NA	NA
BMJ Open Sport & Exercise Medicine	309	33.4	0.3	39	33.3	Man	3.5	NA
BMJ Paediatrics Open	356	35.0	0.6	26	46.2	Man	2.5	NA
BMJ Simulation & Technology Enhanced Learning	180	44.4	0.0	12	58.3	Woman	1.4	NA
BMJ Supportive & Palliative Care	417	48.3	0.7	29	34.5	Men (2)	4.8	3.6
British Journal of Ophthalmology	1113	24.5	0.3	3	0.0	Man	7.3	4.6
British Journal of Sports Medicine	693	28.5	0.1	15	40.0	Man	19.2	13.8
Drug and Therapeutics Bulletin	64	31.3	0.0	12	33.3	Man	NA	NA
Emergency Medicine Journal	767	26.5	0.0	6	50.0	Woman	3.4	2.8
European Journal of Hospital Pharmacy	203	40.5	1.5	16	37.5	Man	1.6	1.7
Evidence-Based Medicine	271	33.3	1.5	11	63.6	Man	3.2	NA
Evidence-Based Mental Health	64	35.9	1.6	12	25.0	Man	8.6	8.5
Frontline Gastroenterology	220	19.5	0.0	11	9.1	Man	3.2	NA
General Psychiatry	167	25.7	0.0	10	10.0	Man	4.5	NA
Gut	1307	20.2	0.8	17	5.9	Man	35.6	23.1
Heart	970	23.0	0.4	17	23.5	Woman	9	6.0
Injury Prevention	282	38.6	1.8	7	57.1	Woman	3.7	2.4
Integrated Healthcare Journal	35	37.1	0.0	2	0.0	Man	NA	NA
Journal of Clinical Pathology	441	30.9	1.8	10	30.0	Man	5.3	3.4
Journal of Epidemiology & Community Health	548	40.7	1.5	22	27.3	Men (2)	6.3	3.7
Journal of Investigative Medicine	366	24.9	0.3	27	18.5	Man	3.9	2.9
Journal of Medical Ethics	726	38.7	0.4	8	62.5	Man	4	2.9
Journal of Medical Genetics	504	38.3	0.0	6	33.3	Man	9.7	6.3
Journal of Neurointerventional Surgery	788	11.2	0.5	16	12.5	Man	8.2	5.8
Journal of Neurology, Neurosurgery, and Psychiatry	1126	19.1	0.7	8	12.5	Man	13.5	10.3
Medical Humanities	198	50.5	1.0	5	60.0	Woman	1.5	NA
Occupational and Environmental Medicine	440	40.6	0.0	15	33.3	Man	6.8	4.4
Open Heart	365	19.2	0.3	13	23.1	Man	3.1	NA
Postgraduate Medical Journal	429	24.9	1.6	12	16.7	Man	3.3	2.4
Practical Neurology	118	16.2	0.8	6	0.0	Men (2)	3.1	NA
Regional Anaesthesia and Pain Medicine	405	21.4	1.0	12	8.3	Men (2)	7.9	6.3
RMD Open	424	32.9	1.2	8	50.0	Man	6.1	5.1
The Journal of ISAKSOS Medicine	165	8.0	1.2	3	33.3	Man	NA	NA
Tobacco Control	519	40.9	1.2	8	75.0	Woman	10.9	6.6
Trauma Surgery & Acute Care Open	140	26.1	1.4	10	50.0	Man	1.3	NA
The BMJ	3224	29.5	0.8	15	80.0	Woman	6.9	38.9
Overall	42 539	30.2	0.9	555	33.4	19.2%
External comparators
NEJM	695	18.9	0.3	19	36.8	Man	80.6	91.2
JAMA	2880	28.1	0.2	32	31.3	Man	NA	56.3

All data refer to 2020 apart from data for the BMJ, which are from 2017 as this was the last year available.

EIC, editor-in-chief.

Women as editors

Overall, women represented 33.4% of the 555 editors, including 19.2% of the 52 editors-in-chief in 2020 (table 1). There were five journals with more than one editor-in-chief, all of which had two men as editors-in-chief. There were 5 journals with no woman among their editors and 12 journals in which women’s representation was equal or above 50% (table 1). Among those 12 journals, 7 had a woman as editor-in-chief. The highest women’s representation was 88% in BMJ Open Quality.

Association between women as editors and peer reviewers

There was a moderate positive correlation between the percentage of women as editors and as reviewers (Spearman correlation coefficient 0.590; p<0.0001) (figure 1). The percentage of women as editors, excluding editors-in-chief, was higher when the editor-in-chief was a woman than a man (53.3% vs 29.2%, respectively; p<0.0001). Likewise, the percentage of women as peer reviewers was higher in journals that had a woman as editor-in-chief in comparison with a man (32.0% vs 26.4%, respectively; p<0.0001).

Figure 1.

Representation of women as peer reviewers and editors according to the gender of the editor-in-chief. Points represent individual journals and size is proportional to the total number of peer reviewers. The colour of the points represents the gender of the editor-in-chief (turquoise for women and coral for men). Black line represents linear regression line. Spearman correlation coefficient was 0.590.

Trends over time and by impact metrics

The percentage of women as peer reviewers increased slightly from 27.3% in 2010 to 29.7% in 2017 in the BMJ, from 23.9% in 2010 to 28.1% in 2020 in JAMA and from 16.9% in 2010 to 18.9% in 2020 in the NEJM (figure 2 and online supplemental table S2). The impact factor of the journals varied between 1.7 for the European Journal of Hospital Pharmacy and 38.8 for the BMJ, and the CiteScore ranged from 1 for BMJ Leader to 35.6 for Gut (table 1). The impact factors of the NEJM and JAMA were 91.2 and 56.3, respectively. The CiteScore of the NEJM was 80.6, and there was no CiteScore for JAMA. There was a non-significant negative correlation between the impact of the journal and the percentage of women as peer reviewers (online supplemental figure S1). The Spearman correlation coefficient was −0.288 (p=0.068), when using CiteScore, and −0.343 (p=0.087), when using impact factor. There was a modest negative correlation between the impact of the journal and the percentage of women as editors when using CiteScore (Spearman correlation coefficient −0.310, p=0.049), but not when using impact factor (Spearman correlation coefficient −0.152, p=0.459).

Figure 2.

Trends in representation of women as peer reviewers The dots represent the percentage of women as peer reviewers for each available year and journal. The colours of the lines represent different journals: British Medical Journal (BMJ), The New England Journal of Medicine (NEJM) and Journal of the American Medical Association (JAMA).

Discussion

In this study of women’s representation among peer reviewers of medical journals in the BMJ-PG, women accounted for 30% of peer reviewers in 2020, with variation from 8% to 50% and no evidence of a meaningful change between 2009 and 2017 in the BMJ. Women were also under-represented among editors, where they accounted for 33% of the editors and 19% of editors-in-chief. Twelve journals (25%) had 50% or more women editors, and five journals had no women editors. Women’s representation among peer reviewers was higher in journals with a higher representation of women as editors, or with a woman as editor-in-chief, as well as in journals with lower impact factor.

Our finding that women account for less than one in three peer reviewers is in keeping with previous studies, which used different methods and samples of journals. In the Frontiers family of journals, women accounted for only 28% of 43 000 peer reviewers between 2007 and 2015.¹⁵ More recently, women were found to represent 21% of 740 000 peer reviewers across 145 journals in various fields of research, including physical, biomedical and social sciences.¹⁶ Women’s representation as peer reviewers was 25% in journals related to biomedicine and health, 21% in life sciences, 16% in physical sciences and 38% in social sciences and humanities. Although the latter study had access to privileged information provided by publishers, it was based on a sample of journals selected by the publishers, which may not have been a random sample. Notwithstanding, the limitations of ascertaining gender based on given names, the consistency of our findings with those of different publishers and journal families supports the validity of the conclusion that women are under-represented as peer reviewers. Furthermore, as we included more recent data, the lack of progress towards gender equity is disappointing.

The underlying reasons for women’s under-representation as peer reviewers of medical journals are likely manifold. First, bias, even if unconscious, may influence editors’ decision to invite a man rather than a woman to peer review a manuscript. Our findings that men are disproportionately represented as editors, and that this is associated with a lower representation of women as peer reviewers in comparison to men, support the possibility of such gender affinity bias. Indeed, a previous study demonstrated editors have substantial same-gender preference when selecting peer reviewers irrespective of whether they are women or men.¹⁷ Likewise, having women as editors-in-chief has been associated with increased representation of women in peer review.¹⁸ Second, considering that peer reviewers are usually senior researchers or leaders in their fields,¹⁸ the long-standing under-representation of women in senior academic roles may leave editors with seemingly little choice but to invite men to peer-review manuscripts.¹⁹ This is supported by our finding that women’s representation as peer reviewers was lower in journals with higher impact factor, which are more likely to acquire peer reviewers who are leading experts in their field. Third, it is possible women face barriers that prevent them from accepting invitations to take part in the peer-review process due to competing demands. Deeply entrenched gendered roles in our contemporary societies mean women still bear the brunt of homemaking, childcare, other unpaid care roles.^{20 21} Furthermore, women undertake a greater share of internal service in academic institutions (eg, activities related to faculty governance, faculty recruitment, evaluation and promotion, student admissions and scholarships, programme supervision, development and marketing, internal awards) in comparison to men.²² Taken together, these unpaid commitments reduce women’s availability to engage with scholarly activities with unscheduled and tight deadlines, such as peer review. Although a recent study showed a minimal difference between women and men’s acceptance of peer review invitations (37% for women vs 41% for men), there was a significant decline during the COVID-19 pandemic in acceptance rates for women, but not for men, in health and medicine journals.²³ This strengthens the argument that the greater burden of caring and family responsibilities posed on women, which was exacerbated during the pandemic, may jeopardise women’s ability to engage with peer review.

The findings of this study have important implications. The wider benefits of gender equality for science and medicine have been compelling demonstrated for men as well as women.^{24 25} Indeed, a research community that is more inclusive, diverse and representative, and works to ensure that everyone counts, is more likely to generate research that is universally beneficial and not limited by inequalities.²⁶ Peer reviewers share with editors the role of gatekeepers of science and evidence. Besides scrutinising and evaluating the quality and integrity of manuscripts, they often influence the content. Ultimately, peer reviewers support editors in determining whether manuscripts are published or not and in which class of journal. Therefore, disproportionate representation of men among peer reviewers and editors could have deleterious consequences on the research that is published as well as its reach and impact on the scientific community and general public. Lack of gender diversity means evidence published in the highest impact journals might be swayed in favour of topics, or methods that are preferred by men and framed from their point of view, thus failing to account for the important perspective and priorities of women. On the other hand, women’s under-representation as peer reviewers may be both a symptom and a cause of broader under-representation in senior positions in academia and journals as taking part in the peer-review process can be a career milestone and a stepping stone to leadership roles.^{27 28}

Although it is unclear how to resolve the long-standing gender gap in the publishing system, particularly in medical sciences, taking small yet steady steps in the right direction and monitoring their effects is a positive approach.²⁰ First, editors should be mindful of the inherent properties of software tools available to help them find suitable peer reviewers.²⁹ Those tools draw on databases of authors and use matching algorithms, which means they are inherently bound to replicate or expand the gender gap in authorship. For instance, Reviewer Finder is a matching algorithm that returns researchers who have a publishing profile similar to that of the manuscript author(s).³⁰ As men are disproportionately represented among authors of papers across many scientific fields, matching is likely to lead to similar gender gaps in potential peer reviewers, unless algorithms are pre-set to suggest a gender balanced pool of peer reviewers. Second, publishers should ensure they have clear policies promoting gender equality (eg, gender quotas) in their editorial boards. Men appear, in general, less aware of gender bias in academia than women, yet hold the majority of leadership positions in publishing, which may exacerbate unrecognised biases if clear policies are not in place.^{31 32} However, evidence from a researcher-led journal suggests improving women’s representation (eg, by gender quotas) may not be enough to stem deep-rooted gender bias observed along the editorial process.³³ For instance, senior editors and authors were more likely to select men than women as reviewing editors, even after correcting for the gender imbalance in the pool of reviewing editors available.³³ Third, publishers should provide training to editors and other editorial staff on diversity and unconscious gender bias to counteract its effects. Although equality and diversity training is no magic wand to address long-standing gender inequalities,³⁴ it may have benefits on cognitive, behavioural and attitudinal/affective learning, especially when complemented by other initiatives targeted to both awareness and skills development, and conducted over a significant period of time.³⁵ Fourth, to improve transparency and accountability, publishers should consider adopting open peer review (ie, publishing the names of the reviewers and the content of the review with the article) or making the names of their peer reviewers publicly available, for instance, as an overall acknowledgement not linked to specific contributions. However, this is not a silver bullet to fix gender inequalities. Even in journals with open peer review as standard policy, women represented only 28% of peer reviewers.¹⁷ In addition, open peer review, if not properly implemented, may exacerbate inequities. Scientists, especially women, have witnessed a sharp rise in harassment, abuse (eg, threatening emails, calls and comments on social media) and attacks on credibility during the COVID-19 pandemic.³⁶ Open peer review could fuel this further by publicly exposing reviewers names and the content of their appraisals. Concerns about deleterious professional and personal consequences of open peer review may discourage women to engage with the process. This, in turn, may result in increased difficulty in finding peer reviewers, and hence strategies will need to be implemented to limit the risk to researchers who reveal their identity during a critical peer review.³⁷ Finally, all of us have a key role to play in promoting gender equality within our teams, working groups, institutions, by exposing unfair gender gaps and addressing overt or concealed gender discrimination and bias.³⁸

Limitations

This study has some limitations to acknowledge. First, we used a binary definition of gender of peer reviewers, which relied on predicting and assigning gender based on given names. Therefore, we did not account for non-binary gender or gender identities that did not match that of the given name and acknowledge that this method does not reflect the true diversity of the medical research community. Pronouns were used to determine gender of editors, and no they/them pronouns were present. However, it is still possible that non-binary gender identification was not reflected by the pronouns used on public websites. Ideally, future research should aim to investigate gender gaps based on self-identified gender, as has been done elsewhere.³⁹ Second, the genderizeR package could not assign a gender to all peer reviewers because the given name could not be classified as belonging to a woman or a man. However, we adopted a two-step approach to maximise the efficiency of the package, and hence the minimal percentage (<1%) of missing data is unlikely to have had a material impact on our key findings.¹⁴ Third, we used journals from a single publishing family, which might not be representative of all medical journals. Results for two leading journals from different publishers, together with previous reports from other journal families, suggest our findings might overestimate women’s representation among peer reviewers of medical journals.^{15 16} Fourth, it is possible that our findings were affected by the COVID-19 pandemic. However, trends over time investigated for the BMJ suggested women’s under-representation is a long-standing issue. Fifth, we cannot ascertain whether the observed correlation between women’s representation among editors and peer reviewers is causal. Sixth, we could not estimate how many manuscripts were reviewed by each individual, and it is uncertain whether this would have swayed the gender distribution in favour of women or men.

Conclusions

Women account for less than one in three peer reviewers in BMJ-PG journals with no evidence of improvement between 2009 and 2017 in the BMJ. No journal had more than 50% women reviewers. Better representation of women as editors was correlated with representation as peer reviewers, thus suggesting increasing women’s representation as editors and peer reviewers may be one among many necessary steps in the pursuit of gender equity in editorial and publishing systems.

Data availability statement

Data are available upon reasonable request. All data are available upon request from the corresponding author.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Not applicable.