Gender climate in rheumatology peer review

Key message.

• Women are underrepresented in the rheumatology peer review process and editorial board memberships.

Dear Editor, To ensure the rigour and relevance of rheumatology research, it is imperative to minimize bias in the peer review process. However, since most academic journals operate blind peer review, it is hard to check for bias in the peer review process [1]. To explore possible gender bias in rheumatology journals’ peer review, we used publicly available data from Publons—a service for reviewers to record and verify their peer review. In this review we focused on high-performing reviewers to ensure we analysed data for those who regularly update their records.

Searching by ‘rheumatology’, we identified 1868 reviewers registered on Publons. Aiming for a sample size representing high-performing peer reviewers with a confidence level of 95% and a margin of error of 5%, we extracted data from 338 reviewers’ profiles. These included those recognized as ‘top in the last 12 months’ and other reviewers according to the number of reviews in decreasing order. We extracted names, top peer reviewer status, editorial board member status, institutional affiliation and reviewer and publication records. Additionally, we determined region based on institutional affiliation and inferred binary gender using Gender API, a name-to-gender inference platform. Gender API infers gender based on the extensive database of validated names in different countries and cultural contexts. When available, the inferred genders were verified using personal pronouns and photos used on social media or institutional web profiles (291 of 338). Statistical testing for gender differences was performed using unpaired t-tests and chi-squared tests with a level of significance of P < 0.05 (Supplementary Data S1 and Supplementary Fig. S1, available at Rheumatology Advances in Practice online).

Data for a total of 329 reviewers were analysed (Table 1). This included reviewers from all major regions. Women were significantly underrepresented in our sample of top peer reviewers compared with men (67.5% men vs. 32.5% women). Similarly, women were less likely to be editorial board members (20.6%). The majority of reviews in our sample were performed by men (70.8% men vs. 29.2% women). We did not find statistically significant gender-based differences in reviewer productivity, as reflected in the average number of reviews and top peer-reviewer awards. Likewise, there were no statistically significant gender differences in reviewer qualifications, as reflected in the average H-index, although women were underrepresented in editorial board memberships.

Table 1.

Characteristics of a sample (N = 329) of high-performing rheumatology peer reviewers on Publons by gender, July–November 2022

Characteristics	Male (n = 222)	Female (n = 107)	P-value
Proportion of total peer reviewers, %	67.5	32.5	–
Peer reviews, n (%)	20 602 (70.8)	8499 (29.2)	–
Average number of peer reviews per reviewer	92.80	79.43	0.45
Average H-index	16.59	15.91	0.74
Reviewers with editorial board membershipa, n (%)	71 (32.0)	22 (20.6)	0.03
Top peer reviewer statusb, n (%)	21 (9.5)	7 (6.5)	0.37

^aAny reviewer with an editorial board membership for at least one journal.

^bAs determined by Publons.

The potential limitations of our analysis include the sampling of rheumatology reviewers from Publons and inferring genders retrospectively. Our sample represents only those high-performing rheumatology reviewers who actively use Publons. While the proportion of researchers archiving their reviewer performance using Publons is unknown worldwide, underrepresentation from Central Asia is known, and we acknowledge that as a limitation of the study. However, it is noteworthy that Central Asia’s rheumatologist pool is also limited, and we hope this offsets some of the underestimates [2]. Reviewer registration with Publons has been known to be skewed towards covering some journals or publishers, highly productive reviewers and the male gender. Hence we do not rule out a possibility of self-selection bias among them. Given that currently there is no other source of data on rheumatology reviews worldwide, we cannot estimate how representative our sample is of the entire population of rheumatology reviewers. Finally, we do not rule out errors while inferring genders, such as accounting for non-binary genders, and a margin of error of up to 1.8% using Gender API [3].

Notwithstanding these potential limitations, our analysis provides the first insights into global peer review in rheumatology journals. The fact that male reviewers performed more than two-thirds of reviews in our sample indicates that there may be a significant gender bias in rheumatology journals’ peer review, given that female and male reviewers can be equally qualified and productive. Our findings align with previous research that shows that women are underrepresented as editors and editorial board members of rheumatology journals [1, 4], speakers and moderators at rheumatology conferences [5], first authors on rheumatology guidelines and recommendations [6] and senior authors on rheumatology research articles [6, 7].

The inequitably larger pool of male reviewers could result from gender preference. Having a pool of reviewers that lacks diversity can result in similar evaluations. This could result in selecting articles with authors similar to the editorial and reviewer pool [1]. Previous research has shown that gender equity in peer review can improve the quality of evaluations and shape the direction of research in academia [7]. Gender equity adds to the diversity and may enhance ways to evaluate research and bring impartiality.

We call on the editors of rheumatology journals to take measures to investigate whether gender bias in peer review exists in their journals and, if so, address it. These may include adding to the peer review registration and submission forms a field for gender, publishing data on gender balance in peer reviews, broadening the network of potential peer reviewers and inviting more peer reviewers of the underrepresented gender and providing adequate recognition to peer reviewers. Additional efforts by rheumatology professionals and institutions aimed at increasing the visibility of female role models; promoting gender-sensitive clinical practice, research and training; and providing support in pursuing academic careers to female rheumatologists are also necessary to broaden the pool of qualified peer reviewers [8].

Supplementary material

Supplementary material is available at Rheumatology Advances in Practice online.

Data availability

All data are included in the manuscript and as supplementary material and further details available from corresponding author upon reasonable requests.

Authors’ contributions

L.G. was responsible for conceptualization. R.S.T. and T.K. were responsible for data curation. R.S.T. and P.O. were responsible for the formal analysis. P.O. was responsible for the methodology. L.G. and P.O. were responsible for validation. R.S.T. was responsible for writing the original draft. All authors were responsible for reviewing and editing the manuscript.

Funding

No specific funding was received from public, commercial or not-for-profit bodies to carry out the work described in this article.

Disclosure statement: The authors have declared no conflicts of interest.