A systematic review of interventions to support the careers of women in academic medicine and other disciplines

Abstract

Objective

To summarise quantitative evaluations of interventions designed to support the careers of women in academia of any discipline.

Method

A systematic search of English entries in PubMed, CINAHL and Google Scholar was conducted in September 2017. Methodological quality of the studies was independently assessed by two authors using the Joanna Briggs Institute quality appraisal checklists. Meta-analysis was not possible due to heterogeneity in methods and outcomes; results were synthesised and displayed narratively.

Results

Eighteen eligible studies were identified, mostly evaluating programmes in academic medicine departments. The most common interventions were mentoring, education, professional development and/or networking programmes. All programmes took a ‘bottom-up’ approach in that women were responsible for opting into and devoting time to participation. Study quality was low overall, but all studies reported positive outcomes on at least one indicator. Most often this included improvements in self-rated skills and capabilities, or satisfaction with the programme offered. Results regarding tangible outcomes were mixed; while some studies noted improvements in promotion, retention and remuneration, others did not.

Conclusions

This review suggests that targeted programmes have the potential to improve some outcomes for women in academia. However, the studies provide limited high-quality evidence to provide information for academic institutions in terms of the best way to improve outcomes for women in academia. The success of an intervention appears to be undermined when it relies on the additional labour of those it is intending to support (ie, ‘bottom-up’ approaches). As such, academic institutions should consider and evaluate the efficacy of ‘top-down’ interventions that start with change in practice of higher management.

Strengths and limitations of this study.

• This review builds on extensive literature regarding the contributors to gender inequality in academic medicine and other disciplines, and synthesises current evidence regarding interventions to address this inequality.

• Included studies are limited to those reporting quantitative outcomes only, with a focus on tangible outcomes such as retention, promotion, research grant success and salary.

• Meta-analysis was not possible due to heterogeneity in methods and descriptive presentation of results.

• Methodological quality of the studies was low overall, making it difficult to determine the program elements most important for success.

Introduction

There is increasing focus on gender inequality in academia and the under-representation of women in senior academic positions internationally.^{1 2} While a trend towards parity among all academic staff has been noted in many countries,^3–5 figures decline sharply with seniority. In the UK, only 18% of Professors of Medicine are women,⁶ and in Australia women represent less than one-third of academic staff above the senior level (Senior Lecturer).³ Similar imbalances are reported in Germany, the USA and, to a smaller extent, Nordic countries.^7–10 Women of colour are particularly affected, accounting for only 7.5% of all full-time faculty in the USA¹¹ and 0.9% of Professors in the UK.⁵ Progress on addressing this problem is slow. The proportion of female full Professors in Medicine in the USA has only grown by 7% (to 17% total) in the last three decades.^{4 12 13} Women remain under-represented on editorial boards, have fewer publications in high-ranking medical journals, are less likely to be awarded research funding and receive less funding in relative terms.^{14 15} They are also more likely to leave academia.¹⁶

Suggested reasons for this under-representation include formal and informal gendered hiring practices that begin at the point of recruitment.¹⁷ For example, fewer women are involved in the selection process of new staff,¹⁷ and women are less likely to benefit from networks with more senior (and usually male) faculty, a key barrier to promotion.¹⁷ Women are considered less capable and suited for leadership than their male counterparts,^{18 19} and when exhibiting traits that are considered signs of leadership when demonstrated in men (eg, assertiveness, competitiveness), women can attract penalisation or loss of credibility.²⁰ Women also more often engage in equalising behaviour that is not valued or tied to career advancement.¹² Women with children are disadvantaged in a variety of ways particularly if they take time away from work to care for their children. This includes reduced research publication outputs and chances of being recruited, lower perceived credibility and difficulties with ‘catching up’ to their male counterparts on return to work.^{8 21–23} While women are usually aware of these inequalities, they report sensing a complacency among upper management and fear being considered a ‘trouble maker’ if they take assertive action (and the damage to career that can result).²⁴

Given the complexity and wider causes of gender inequality in academia, interventions to improve conditions are likely to be similarly complex. Identification and implementation of evidence-based intervention programmes that improve parity are essential considering the rights of women to equal opportunity and the noted benefits of staff diversity.²⁵ The under-representation of women in senior academic positions is a waste of public investment in women’s capital, deprives research of women’s perspectives and stunts the economic growth of the sector as a whole.^{26 27} Several approaches to addressing the problem have been trialled, but whether these contribute to quantifiable benefits for women (such as promotion or remuneration) is not well understood.

The aim of this systematic review is to identify intervention programmes intended to support the careers of women in academia, to describe and synthesise the quantifiable outcomes of these programmes and to identify the most efficacious programme elements. This information can then be used to guide academic institutions in programme design and delivery.

Method

We drafted a protocol for this review ‘a priori’ and inclusion criteria were developed prior to commencing the search. We report according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, and a checklist of PRISMA items is presented in the online supplementary data S1.

Eligibility criteria

Studies were eligible for inclusion in the review if

• The population of interest was women in any field in academia;

• The study evaluated an intervention specifically intended to improve outcomes for these women;

• Some form of quantitative evaluation was conducted; and

• Outcomes included self-rated attitudes/efficacy or concrete measures of success (eg, funding, promotion, publications).

Studies were excluded when

• The intervention population was not specific to women, for example, where institutions looked to increase ethnic, gender and socioeconomic diversity in tandem. Studies that assessed existing opportunities for career development, rather than evaluating a specific programme, were also excluded;

• The study did not conduct a formal evaluation of a programme or used qualitative or narrative outcomes, which were beyond the scope of this review. Included studies were limited to quantitative studies to identify quantifiable outcomes on objective measures; and

• Only a conference abstract was available with insufficient detail about study methods or results.

Data sources and searches

We searched PubMed (1966 onwards) and CINAHL (1981 onwards) for English-language studies published any time to September 2017. The search strategy (available in online supplementary data S2) was deliberately broad in an effort to gather all eligible studies. Reference lists of all included studies were hand-searched for additional records. We also searched grey literature via Google and Google Scholar.

Study selection and data extraction

Two authors (KEL and MC) reviewed titles, abstracts and full-text papers for eligibility. Authors resolved disagreement by discussion or, where necessary, a third author (IP) offered their view. Another author (IO) was responsible for extracting data using a standardised data sheet that was piloted with three papers and revised. All data extraction was verified by MC, and disagreement was resolved via discussion. Extracted data included study design, participants, intervention details, comparators, outcome assessed and method, and relevant statistical data.

Quality assessment

Two people (KEL and MC) independently assessed the methodological quality of each study using the Joanna Briggs Institute (JBI) battery of quality appraisal tools.²⁸ This battery was chosen because it offers a range of checklists suitable for a variety of study designs, which is useful given the breadth of study types included here. The tools also rank well in systematic evaluations of quality assessment tools.²⁹ An amended version of the eight-item Checklist for Analytical Cross-Sectional Studies was used for observational and post-test only design studies, removing item four (‘Were objective, standard criteria used for measurement of the condition?’) because the ‘condition’ was central to every study. Pre-test and post-test studies or controlled studies were appraised using the nine-item JBI Checklist for Quasi-Experimental Studies. Both checklists include yes/no questions assessing study reproducibility (maximum score 7 and 9, respectively).

Data synthesis

Due to significant heterogeneity regarding interventions, comparison groups, outcomes of interest, outcome measurement and statistical analysis, it was not possible to conduct meta-analysis.

Results

The search of the electronic databases revealed 2790 citations (figure 1). A search of reference lists and the grey literature revealed four further relevant studies. Of the citations identified, 61 full-text articles were reviewed and 18 were found to meet all inclusion criteria (see table 1).

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart describing the process of study selection.

Table 1.

Details of interventions assessed in included studies

Study, country	Country	Programme type	Length of programme	Details
Richman et al 38	USA	Multicomponent (ELAM)	1 year	Senior women faculty invited to participate in the Executive Leadership in Academic Medicine for women programme (ELAM), including three sessions including two week-long residential sessions and one annual meeting of the American Medical Colleges. Programme elements include Networking Lectures, panel discussions, in-depth case studies, role playing and small group work Individual assessment Career counselling Intersession assignments
McDade et al 37	USA	Multicomponent (ELAM)	1 year	See above
Stewart et al 33	USA	Peer education	Not specified	Creation of a faculty committee of full professors in science and engineering (three women, four men), trained to understand contributors and consequences of gender inequality in academia. Brainstormed methods to improve hiring practices in their department over 20 hours of meetings. Dissemination via 26 workshops with their department staff (details not specified)
Jagsi et al 39	USA	Professional development grant	2 years	Junior women academics at instructor or assistant professor level and responsible for the care of children invited to apply for US$30 000 of faculty funding for 2 years, to be used for professional development
Seritan et al 42	USA	Multicomponent	Ongoing	Women faculty invited to participate in the Society of Women in Academic Psychiatry group. Intervention included Peer mentoring Online collaboration on scholarly projects Events with special guests Liaison with interdisciplinary teams Professional development regarding research resources, collaborative writing, leadership and negotiation skills
Gardiner et al 30	Australia	Peer mentoring	Ongoing	Women academics at lecturer level with mixed teaching and research roles invited to be mentored by a senior faculty member. Education workshops on mentoring provided to both mentors and mentees Dyads decided on the frequency, length and content of their meetings (based on the suggested 1 hour per month)
Dannels et al 36	USA	Multicomponent (ELAM)	1 year	See above
Files et al 34	USA	Peer mentoring	1 year	Junior women invited to form a peer mentoring group who met weekly to monthly. Each given 25 hours of time to participate Senior women faculty invited to form facilitator mentor group—available to junior faculty on an as-needed basis. Also met together once a month Whole group met once a month Three phases: Skill acquisition and enhancement (lectures, workshops, information sharing) Skills application (writing an article)—reviewed by facilitators, available for brainstorming Development of group research protocol
Von Feldt et al 47	USA	CV review	One session	Women assistant professors invited to submit CV for review by a mentor of their choice (in a different department). Mentor provided feedback regarding CV, promotion and resources CV reviews occurred at a conference during a break
Dutta et al 31	UK	Peer mentoring	1 year	Women academics at senior lecturer level or below invited to be mentored by a senior faculty member of their choice Mentors received training in developmental mentoring
Carnes et al 45	USA	Peer education	2.5 hours	Delivery of a 2.5-hour bias literacy workshop in individual departments, to improve awareness of gender bias and its consequences in academia. Four elements: Generating recognition of a need to change and desire to act Providing tools to engage in new behaviour Helping envision a link between action and desired outcome Facilitate deliberate practice
Varkey et al 40	USA	Peer mentoring	1 year	Women faculty at either instructor or assistant professor rank were invited to join a peer mentoring group with others at similar rank Five groups formed, each with a facilitator experienced and trained in group development. Group mentoring sessions occurred at least once monthly
Bauman et al 46	USA	Multicomponent	1 year	Women faculty invited to participate in the Women in Medicine and Health Sciences programme, including Opportunities for creating networks, interacting and collaborating with one another Networking events (meeting leaders, founding women’s events) Internal career development (mentorship clinics, leadership clinics) External career development (public speaking, salary negotiations) Work/life balance education (elder care workshops) Leadership and mentoring opportunities Professional development workshops and lectures
Valantine et al 43	USA	Multicomponent	Ongoing	School of Medicine implemented Provost’s Advisory Committee on the Status of Women Faculty. Interventions included recruitment of diverse faculty, faculty awards, professional development programmes, intensive mentoring, skill building workshops, professional networking programme
Helitzer et al 41	USA	Multicomponent (ELAM and others)	ELAM: 1 year Others: 3 to 4 days	ELAM: see above Early Career Women Faculty Professional Development Programme (EWIM): professional development seminars for women at instructor, lecture or assistant professor level Mid-Career Women Faculty Professional Development Programme (MidWIM): professional development seminars for women at associate professor or professor level
Levine et al 32	USA	Multicomponent	10 months	All women faculty invited to participate in the Leadership Programme for Women Faculty, including nine half-day workshop sessions including networking, role-play and reflective practice. Modules covered Collaboration skills Networking skills Presentation skills Influence of gender on communication styles Agreement and conflict management Decision-making Facilitating group decision-making Leadership skills
Chang et al 35	USA	Multicomponent (ELAM and others)	ELAM: 1 year Others: 3 to 4 days	ELAM: see above EWIM: see above MidWIM: see above
Girod et al 44	USA	Peer education	20 min	Standardised presentation by Medicine department leaders summarising research literature on implicit bias, particularly in reference to gender and leadership Includes Data about the existence and effects of unconscious gender bias Tips for overcoming bias in hiring processes

CV, curriculum vitae.

Characteristics and quality of included studies

Studies were published between 2001 and 2016 and all but two^{30 31} took place in the USA. Fifteen were implemented in academic medicine departments, while the other three^{30 32 33} were implemented across multiple departments (including medicine). Most samples were relatively small (with as few as four participants³⁴) except in one study where several thousands of faculty members over time were included in analyses.³⁵ Interventions were provided for senior faculty (Assistant/Associate Professor and above) in four studies,^36–38 junior faculty in five studies,30 31 34 39 40 all female faculty in five studies32 35 41–43 and all faculty (male and female) in four studies.^{33 44–46} Interventions targeting junior faculty were typically focused on peer mentoring, career development skills and support to return to work after having children, while programmes for senior faculty involved upskilling in administrative leadership and conflict management. Programmes delivered to all faculty (men and women) involved education about the impact of gender bias and techniques to improve equity or, in one case,⁴⁶ providing better support to new fathers. None of the studies specified the ethnicity of their sample.

The methodological quality of study designs was low to moderate overall. Only three studies included a comparison group using controlled before and after^{30 36} or cohort design.³⁵ Members of the comparison group were either matched on level and time at the University^{30 35} or drawn from those who had applied to be part of the programme but had not been accepted.³⁶ Of the 15 studies that did not include a comparison group, four used an observational design with the whole faculty or institution,33 38 43 46 four conducted evaluations of programme participants at the end of the intervention only39 41 42 47 and seven implemented pre-programme and post-programme measures.31 32 34 37 40 44 45 Several methodological limitations were noted during critical appraisal (table 2), including that eight studies did not use validated tools to measure their outcomes and nine did not include adequate description of participants. None of the observational or post-test only studies identified or appropriately controlled for relevant confounds, and six (of 10) before-and-after and controlled studies did not provide enough detail about data loss at follow-up. Most studies did not include repeat follow-up to assess for sustained effects of the intervention. JBI Checklist scores ranged from 2/7 (28.5%) to 6/9 (66.7%).

Table 2.

Results of quality appraisal of included studies

Study	Were the criteria for inclusion in the sample clearly defined?	Were the study subjects and the setting described in detail?	Was the exposure measured in a valid and reliable way?*	Were confounding factors identified?	Were strategies to deal with confounding factors stated?	Were the outcomes measured in a valid and reliable way?†	Was appropriate statistical analysis used?‡	Score
Modified analytical cross-sectional checklist (observational and post-test only studies)
Richman et al 38	N	N	N	N	N	Y	Y	2/7
Stewart et al 33	N	N	Y	Y	N	Y	Y	4/7
Jagsi et al 39	Y	N	Y	N	N	Y	Y	4/7
Seritan et al 42	N	Y	N	N	N	N	Y	2/7
Von Feldt 47	N	N	Y	N	N	N	Y	2/7
Bauman et al 46	N	N	N	N	N	Y	Y	2/7
Helitzer et al 41	N	Y	Unclear	N	N	Y	Y	3/7
Valantine et al 43	N	N	N	N	N	Y	Y	2/7

Study	Is it clear in the study what is the ‘cause’ and what is the ‘effect’ (ie, there is no confusion about which variable comes first)?	Were the participants included in any comparisons similar?	Were the participants included in any comparisons receiving similar treatment/care other than the exposure or intervention of interest?	Was there a control group?	Were there multiple measurements of the outcome both before and after the intervention/exposure?	Was follow-up complete and if not, were differences between groups in terms of their follow-up adequately described and analysed?	Were the outcomes of participants included in any comparisons measured in the same way?	Were outcomes measured in a reliable way?†	Was appropriate statistical analysis used?‡	Score
Quasi-experimental checklist (pre-test and post-test or controlled studies)
McDade et al 37	Y	Y	Y	N	N	N	Y	N	Y	5/9
Gardiner et al 30	Y	Unclear	Unclear	Y	N	N	Y	Y	Unclear	4/9
Dannels et al 36	Y	N	Y	Y	N	N	Y	Y	Y	6/9
Files et al 34	Y	Y	Y	N	N	Y	Y	N	Y	6/9
Dutta et al 31	Y	Y	Y	N	N	N	Y	Y	Y	6/9
Carnes et al 45	Y	Y	Y	N	N	N	Y	N	N	4/9
Varkey et al 40	Y	Y	Y	N	N	N	Y	N	Y	5/9
Levine et al 32	Y	Y	Y	N	N	Y	Y	N	Y	6/9
Chang et al 35	N	N	Y	Y	Unclear	Y	Y	Y	Y	6/9
Girod et al 44	Y	Y	Y	N	N	Y	Y	Unclear	Y	6/9

*The ‘exposure’ was considered intervention attendance/adherence.

†Validated tool used (with figures provided) or reliable figures like rank.

‡Studies employing descriptive analysis only were considered appropriate.

Interventions

Three studies^36–38 evaluated the efficacy of the Hedwig van Amerigen Executive Leadership in Academic Medicine for women programme (ELAM) with overlapping samples. ELAM is an American 1-year leadership training programme for senior academic women with coaching, networking and mentoring opportunities. Participants meet for a week at the beginning and end of the year, once in between this time, and submit assignments regularly. The curriculum aims to improve skills in paradigms of leadership, financial management, strategic planning, emerging issues in academic medicine, communication, personal dimensions of leadership and career advancement strategies.³⁸ An additional two studies compared outcomes from ELAM with less intensive multicomponent interventions involving professional development seminars only.^{35 41} Four studies evaluated other multicomponent interventions, which involved two or more of peer mentoring, education and skill building sessions, networking and improved access to resources (such as child care or grant funding).32 42 43 46 These interventions were delivered over 10 months,³²1 year⁴⁶ or on an ongoing basis.^{42 43} One-year peer mentoring programmes were evaluated in four studies.30 31 34 40 Three studies evaluated the impact of peer education sessions about implicit and explicit bias against women in academia, with strategies to address this bias and improve hiring practices.^{33 44 45} The final two studies evaluated the efficacy of provision of a US$30 000 grant to enable research and professional development for women responsible for the care of children³⁹ and a short curriculum vitae (CV) review and feedback session.⁴⁷

Efficacy of interventions

The included studies reported on a range of outcomes that could be categorised into (1) self-reported skills and capabilities, (2) gender bias, (3) satisfaction with the programme and (4) tangible outcomes including faculty representation, retention, rank and remuneration. All of the included studies reported positive outcomes on at least one measure. A snapshot summary of outcomes appears in table 3 and detailed findings in table 4.

Table 3.

Summary of outcomes

Study	Intervention	Positive outcome reported by authors
		Self-reported skills	Bias	Representation, promotion, retention, remuneration	Satisfaction with programme	Satisfaction with career, well-being
Multicomponent
Richman et al 38	ELAM			Yes
McDade et al 37	ELAM	Yes			Yes
Seritan et al 42	Other multicomponent			Yes (representation); no (rank)	Yes
Dannels et al 36	ELAM	Yes		Yes
Bauman et al 46	Other multicomponent			Yes		Yes
Valantine et al 43	Other multicomponent			Yes		Yes
Helitzer et al 41	ELAM and other multicomponent	Yes
Levine et al 32	Other multicomponent	Yes		No
Chang et al 35	ELAM and other multicomponent			Yes
Peer education
Stewart et al 33	Peer education			Yes
Carnes et al 45	Peer education		Yes
Girod et al 44	Peer education		Yes
Peer mentoring
Gardiner et al 30	Peer mentoring	Yes		Yes		No
Files et al 34	Peer mentoring	Yes		Yes
Dutta et al 31	Peer mentoring	Yes				Yes
Varkey et al 40	Peer mentoring	Yes
Other
Von Feldt et al 47	CV review				Yes
Jagsi et al 39	Professional development grant			Yes	Yes

CV, curriculum vitae; ELAM, Executive Leadership in Academic Medicine programme.

Table 4.

Included study outcomes

Study	Design (time point/s)	Intervention	Participants	Department/s	Outcomes	Results
Richman et al 38	Observational (first programme in 1995, observation period to 2000)	ELAM	85*	Medicine and dentistry	No of participants in senior administrative positions (chair, vice chair, division chief, assistant and associate dean)	Participants with senior administrative positions increased from 47.1% to 70.6%†
McDade et al 37	Within group pre-test and post-test (baseline, 11 months after programme completion)	ELAM	79*	Medicine and dentistry	67-item Likert-scale questionnaire to measure self-rated capability on 10 leadership constructs: Knowledge of leadership, management and organisational theory Environmental scanning Financial management Communication Networking and coalition building Conflict management General leadership skills Assessment of strengths and weaknesses Acceptance of demands of leadership Career advancement sophistication Additional post-ELAM questions regarding satisfaction with programme	Improved self-rated leadership capability on all 10 domains post-ELAM, all statistically significant to P<0.001. MD=0.84–1.91 Overall satisfaction with ELAM programme
Stewart et al 33	Observational (1–2 years after implementation of workshops)	Peer education	Whole departments (129 total)	Medicine, basic science, engineering, literature, arts	No of women faculty in all specified departments (rank not specified)	Increase in women faculty from 15.7% in 2000/2001 to 31.3% in 2002/2003 (P<0.05)
Jagsi et al 39	Post-test only (9 years after programme formation)	Professional development grant	121 total applicants 40 recipients 30 survey participants	Medicine	Retention among all recipients Promotions among all recipients Perceived efficacy, optimism among survey participants (qualitative only)	90% retention (compared with 68% non-awarded)† 55% promotion (compared with 30.9% non-awarded)† Programme satisfaction very high overall
Seritan et al 42	Post-test only (1 year after programme formation)	Multicomponent	All department (n not reported) 14 provided qualitative feedback	Psychiatry	No of women in department Rank of women staff Qualitative feedback (collected in a meeting)	Increase in women faculty from 29% to 33% in 1 year† No change in rank representation† Meeting participants reported a sense of community, belonging and empowerment
Gardiner et al 30	Controlled before and after (baseline, end of programme, 6 years later)	Peer mentoring	Intervention group: 22 (mostly at lecturer level with mixed teaching and research roles) Control group: 46 (similar academic standing)	All	Retention Promotion Grant income Publications Subjective career outcomes Perceptions of mentoring	Retention: 86% (intervention) vs 67% (control)† Promotion: 68% (intervention) vs 43% (control)† Annual grant income: US$6983 (intervention) vs US$2441 (control)† Perceived capacity: significantly higher perceived capacity in intervention than control group (P<0.01) No significant differences in career satisfaction, job satisfaction, career planning, work-related distress or work-related morale (all P>0.05)
Dannels et al 36	Controlled before and after (baseline, 4 years after programme)	ELAM	ELAM group: 53 Comparison group: 172‡ Non-ELAM group: 25 (not accepted to ELAM)	Medicine	34-item Likert-scale questionnaire of self-rated capability on eight constructs: Knowledge of leadership theory Environmental scanning Financial management Communication Conflict management Diversity competence Reconciliation to demands of leadership Leadership positioning Demographic data regarding: Academic rank Promotion	Rank of chair or above: 63.5% (ELAM), 22.5% (comparison), 37% (non-ELAM) (all P>0.05) Women at full professor rank increased over time in all programmes†: ELAM 44.8% (2001) to 69.8% (2006) Comparison 55.4% (2001) to 68.6% (2006) Non-ELAM 46.1% (2001) to 48% (2006) Self-rated capabilities significantly higher in ELAM than other groups in seven domains (all P<0.05); not statistically significant for diversity competence
Files et al 34	Within group pre-test and post-test (before and 10 months into programme)	Peer mentoring	4 (ranked at instructor level with no peer-reviewed papers)	Medicine	Publications Promotion 26-item Likert-scale questionnaire of academic skills and career satisfaction. Key indicators: Satisfaction with accomplishments Achievement of necessary skills Belief in necessary writing skills	3 co-authored peer-reviewed manuscripts All achieved promotion in academic rank from instructor to assistant professor 30% improvement in key indicators†
Von Feldt et al 47	Post-test only (2 weeks after session)	CV review	93 CV review sessions 61 survey participants	Medicine	26-item questionnaire of experience and satisfaction with sessions	50%–66% of participants agreed that the experience was helpful and productive 51% believed that they had at least 60% likelihood of being promoted to associated professor
Dutta et al 31	Within group pre-test and post-test (baseline, 6 months, 1 year)	Peer mentoring	46 mentoring pairs 44 survey participants (rank senior lecturer level or below)	Psychiatry	Validated scales measuring Job satisfaction Job-related anxiety Job-related depression Self-esteem Self-efficacy Work–family conflict Qualitative perceptions of benefits	Job satisfaction: no significant differences at 6 months or 1 year Job-related anxiety: no differences at 6 months, significant improvement at 1 year (MD=0.40, P<0.01) Job-related depression: no significant differences at 6 months or 1 year Self-esteem: significant improvements at 6 months (MD=2.22, P<0.01) and 1 year (MD=2.62, P<0.01) Self-efficacy: significant improvements at 6 months (MD=0.91, P=0.02) and 1 year (MD=1.07, P<0.01) Work–family conflict: no difference at 6 months, significant improvement at 1 year (MD=−1.52, P=0.04) Key benefits identified: Confidence and assertiveness Support and encouragement Space to reflect on careers and goals
Carnes et al 45	Within group pre-test and post-test (immediately before and after workshop)	Peer education	167 faculty members, 53 administrative staff attended workshops 134 completed post-workshop evaluation	Medicine	Increase in self-reported knowledge on workshop content; plan to incorporate workshop elements into practice	Improved knowledge about† Expectancy bias: 77.8% Prescriptive gender norms: 68.6% Role congruity/incongruity: 72.6% Reconstructing credentials: 91.4% Stereotype priming: 80.6% Stereotype threat: 72.1% Strategies for deliberate practice of non-biased behaviours: 82.9% Plan to incorporate workshop elements into practice: 87%
Varkey et al 40	Within group pre-test and post-test (baseline, 12 months)	Peer mentoring	19 (rank either instructor or assistant professor)	Medicine	25-item questionnaire assessing self-efficacy, academic skills and career goals	Significant improvement in satisfaction with academic achievement, networking skills, skills necessary for academic success, skills necessary for publishing a research paper, and knowledge of how to access a mentor (all P<0.05) No significant difference in skills related to developing clinical research projects Nine manuscripts submitted, six published, a clinical research project completed
Bauman et al 46	Observational (programme established in 2000, observation from 2001 to 2011)	Multicomponent	Whole department; n not reported	Medicine	No of women faculty over time Retention rate	Increase in women faculty from 138 (2001) to 235 (2011)† Increase in women chairs from 1 (2001) to 5 (2011)† Majority of faculty women either satisfied or very satisfied with their careers
Valantine et al43	Observational (programme implemented in 2001; observation from 2004 to 2010, surveys 2003 and 2008)	Multicomponent	Whole department, 234 at baseline	Medicine	Observational: No of women faculty Rank of women faculty Surveys: Likert-scale questionnaire of quality of life (no of items not reported)	No of women facility increased from 234 (27.7%) to 408 (33.5%)† Increase in representation at assistant, associate and full professor ranks noted† Quality of life in women faculty significantly increased after the programme (P<0.05)
Helitzer et al 41	Post-test only (survey sent in 2010 to all programme attendees since 1988)	ELAM and other multicomponent (EWIM and MidWIM)	845* ELAM=127 EWIM=306 MidWIM=252 Multiple=160	Medicine	Participants provided with a list of 16 academic skills and asked to report whether skills were acquired, improved or not improved as a result of the programme. Skills were Achieving work/life balance Communication CV/executive summary development Finance Human resources Interpersonal Interview preparation Leadership Leading meetings Managing difficult discussions Public relations Mentoring Negotiation Networking Planning for promotion Planning for next career stage	Most participants reported an overall gain in skills: 95% (EWIM), 93% (MidWIM) and 99% (ELAM) Attainment of 11 skills significantly more common in ELAM than other programmes (all P<0.05) No difference between programmes in achieving work/life balance, interview preparation, managing difficult discussion, mentoring, planning for promotion (all P>0.05)
Levine et al 32	Within group pre-test and post-test (baseline, end of 10-month programme)	Multicomponent	95	Medicine, surgery, basic sciences, nursing, engineering, public health	11-item questionnaire assessing self-rated skill in 11 leadership domains: Developing a mission statement Working in teams Crucial conversations Dealing with difficult behaviour Public speaking Understanding influencing style Understanding gender-based communication differences Negotiation style and skills Understanding gender differences in decision-making Enhancing decision-making Influencing decision-making in groups Also, asked how often they had negotiated before and during programme	Significant improvement in all domains (all P<0.05) aside from working in teams (possibly due to ceiling effects) No increase in salary or promotion negotiation following the programme
Chang et al 35	Observational with matched comparison groups (observation 1988–2008)	ELAM and other multicomponent (EWIM and MidWIM)	Programme participants=3268* Women non-participants=17 834 Male faculty=40 319	Medicine	Retention	Programme participants recorded significantly longer retention than women non-participants and men overall (P<0.001) and at each academic rank Results retained after controlling for age, tenure status, degree and department type (except for Full Professor level, where retention rates were the same between men and programme participants) Type of programme not associated with retention (P>0.05) Participation in more than one programme associated with longer retention (P<0.001)
Girod et al 44	Within group pre-test and post-test (immediately before and after video presentation)	Peer education	281 (163 male, 118 female)	Medicine	General perception about bias: survey of agree/disagree statements about gender bias and stereotypes (no of items not reported) Explicit bias: survey asking respondents to explicitly rate the effectiveness of men and women as leaders (no of items not reported) Implicit bias: Brief Implicit Association Test, which assesses strength of association between two pairs of concepts (male/female and leader/follower) (no of items not reported)	Increase in perception of personal bias (P<0.01) Increase in perception of bias in academic medicine (P<0.001) Increase in perception of societal gender bias (P<0.05) No significant change in explicit bias (P>0.05) Decrease in implicit bias (P<0.002); remained after controlling for gender, age, race (P<0.01) No interaction effect with age, gender, race

*Used overlapping samples.

†Statistical significance not reported.

‡Matched to ELAM group by academic rank, department chair status, race/ethnicity, degree type, discipline, department, age, medical school, award ranking.

CV, curriculum vitae; ELAM, Executive Leadership in Academic Medicine programme; MD, mean difference.

Self-reported leadership capability and skills

Eight studies measured self-rated leadership capabilities and skills and all reported positive results following intervention. Four of these studies evaluated programmes that involved a number of components. One controlled before-and-after study reported that those enrolled in the ELAM programme rated their leadership capability more highly after the programme than matched control participants or those not accepted into the ELAM programme. However, the authors did not specify or control for reasons for non-admission to the programme.³⁶ Similarly, McDade et al ³⁷ reported significant improvements in all 10 domains of leadership capability following participation in the ELAM programme but did not include a comparison group. The benefits of multicomponent programmes were not unique to ELAM; Helitzer and colleagues⁴¹ found that nearly all participants in any one of three multicomponent programmes (including ELAM) reported an overall gain in leadership skills, while Levine and colleagues³² found that participants in their multicomponent programme (including nine half-day workshops and networking) reported post-intervention improvements in most reported leadership domains including developing a mission statement, dealing with difficult behaviour and influencing decision-making.

Peer mentoring programmes were also universally linked to improved self-rated skills. Dutta et al ³¹ reported that self-rated well-being, self-esteem and self-efficacy all improved following a 1-year peer mentoring programme, while Varkey et al ⁴⁰ noted significant improvements in writing, networking, critical appraisal and other skills. However, none of these studies of peer mentoring included a comparison group. Of the eight studies reporting on self-rated skills, only two explored whether benefits extended to tangible outcomes, with conflicting results.^{32 36}

Gender bias

Both Girod et al ⁴⁴ and Carnes et al ⁴⁵ noted a significant increase in knowledge of gender bias and awareness of one’s own bias in academic medicine following a once-off peer education session. Carnes et al ⁴⁵ additionally noted that 87% of workshop participants indicated an intention to change their practice as a result, but did not assess whether this occurred. Neither of these studies reported whether the education translated to improved conditions for women faculty.

Satisfaction with programme

Four studies reported that there were high levels of participant satisfaction with an intervention. This included development of a sense of community and empowerment⁴² and increased career satisfaction⁴⁶ following participation in a multicomponent intervention including peer mentoring, networking and professional development, and appreciation of peer feedback and practical skills in manuscript preparation.³⁴ Von Feldt and colleagues⁴⁷ reported that a brief CV review session was moderately successful with 50%–66% of participants reporting the session to be helpful and productive. None of these studies established whether these benefits extended to tangible outcomes.

Representation, promotion, retention and remuneration

Eleven studies reported the impact of interventions on career outcomes for women including faculty representation, promotion, retention and pay. Seven were evaluations of multicomponent programmes, and five of these reported positive outcomes. In addition to self-rated leadership capabilities, Dannels et al ³⁶ noted a significant improvement in rank among participants of ELAM relative to other programmes. Similarly, Chang et al ³⁵ compared 20-year retention rates between ELAM or other (less intensive) multicomponent programme participants, non-participating women faculty and matched male faculty. Programme participants at all levels recorded significantly longer retention at their institution than non-participating women and equivalent or longer retention when compared with men. These effects remained even after controlling for age, tenure status, degree and department type (clinical, basic science, other). Three other uncontrolled studies reported increased representation of women in senior positions concurrent with implementation of ELAM³⁸ or other multicomponent interventions,^{43 46} but did not establish whether the increase was statistically significant or attributable to programme participation. Similarly, Seritan et al ⁴² noted an overall 4% increase in women faculty 1 year after implementing their multicomponent programme including peer mentoring, online collaborations and professional development, but did not report the statistical significance of this. However, there was no change in rank representation among participating women. This is consistent with Kazemi and colleagues,⁴⁸ who reported that despite increased self-reported capability (see above) there were no significant increases in salary or promotion following their multicomponent programme (though they did not include a comparison group).

Both studies reporting on tangible outcomes following mentoring programmes reported positive results but were of low methodological quality. One reported that all four participants were promoted within 1 year (but did not compare this to non-participating faculty),³⁴ while the other reported higher rates of retention, promotion and grant income compared with non-participating women (but did not report the statistical significance of this).³⁰

Jagsi and colleagues³⁹ reported that 55% of women responsible for the care of children who had received an internal financial grant were promoted within a year, compared with 31% who had not received the grant. Grant recipients reported feeling more optimistic about their careers and that the grant helped them to overcome impediments related to taking time off to care for children. Authors also noted that subsequent funding received by grant participants far exceeded institutional cost of the programme.

Finally, Stewart et al ³³ reported that female faculty doubled in the 2 years following establishment of a committee of full professors in science and engineering who received extensive education and delivered workshops to other staff about gender bias in academia and strategies to improve recruitment of female staff. However, the authors note several factors that could account for this change aside from the intervention (but were not controlled for in analysis), including that the departments included in the analysis were all led by members of the committee. In addition, an institutional report was released during the follow-up period criticising the academic climate of the University and a departmental grant was subsequently made available for individual departments to address inequalities.

Discussion

To our knowledge, this is the first review to identify and synthesise quantitative results of programmes that had operated in academic institutions to improve career outcomes for women. Eighteen eligible studies were identified, evaluating a range of structured programmes typically including peer mentoring, education and skill development, and networking opportunities. Study quality was low to moderate overall. All studies reported positive outcomes on at least one indicator. Most often these were related to self-rated skills and capability, or satisfaction with the interventions. Eleven of the studies included in the review reported how these benefits translated to concrete outcomes like promotion, retention, grant success and pay, with mixed results.30 32–34 36 38 39 42 43 46

Main findings

The ELAM programme was evaluated in five studies, though four of these were conducted at the same University and used overlapping samples.35 37 38 41 The benefits of ELAM outside its original institution and transferability to other disciplines cannot be established from existing literature. Nonetheless, the programme had a positive effect on the rank, retention and/or self-rated capabilities of women faculty in all studies. Benefits of ELAM may be greater than those achieved by other multicomponent programmes, with one study demonstrating a greater increase in self-rated skills after ELAM than from two other programmes comprising professional development seminars only.⁴¹ Authors attribute these added benefits to the greater length and intensity of ELAM, and intentional community building with other female staff. A more recent study using the same database of participants, however, did not find any difference in retention between the same programmes.³⁵ This may reflect that the benefits of more intensive programmes do not extend to tangible outcomes.

Mentoring was a central component of almost all multicomponent programmes and was evaluated separately in four studies. Although the nature and content of mentoring varied, positive effects were noted in all studies including improvement in perceived skills and self-esteem,^{31 40} and in one study higher retention and promotion rates.³⁰ The mutual benefits available to universities by investing in junior women academics were exemplified by one study that evaluated a professional development grant programme for women responsible for the care of children. Authors remarked at the significant return on investment generated by the scheme, as recipients had achieved an income of more than US$51 million in grants and investments after the faculty invested just over US$2 million.³⁹ This is congruent with evidence to suggest that diversity among staff contributes positively to an organisation’s financial health.²⁵ Recipients of the grant noted that its existence had a positive impact on the institutional culture, as it promoted optimism about the potential to balance an academic career with child rearing.

The importance of tailoring an intervention to the life and career stage of the woman was well acknowledged during intervention design in all studies. Programmes intended for junior faculty typically addressed known barriers to career progression including lack of role models and mentors, fewer networks and difficulty managing family and work responsibilities.⁸ On the other hand, programmes for more senior women focused on developing skills in administrative leadership and overcoming the systemic barriers that prevent women from reaching leadership positions.³⁸ Very few of the studies discussed the impact of rank on the experience and outcomes of programme participation, and this is a valuable avenue for future research.

Despite the positive outcomes reported in most studies, more work is required to inform design of high-quality and efficacious programmes to support the careers of women in academia. Few studies evaluated their intervention of interest in depth, establishing the most efficacious components, environmental factors that impacted outcomes or sustainability of benefits. Most were designed without a suitable control or other comparison group, and it is not possible to know whether results are attributable to the intervention or other factors. Effects noted in observational studies in particular could reflect other political, economic, cohort or organisational change. Most studies enrolled relatively small samples, with limited power to detect meaningful differences. Additionally, few of the included studies addressed the impact of self-selection and the pre-existing differences between those who participated in the intervention and those who did not. While participation in a given programme is known to vary according to personal preferences, formal and informal support, perceived efficacy and other barriers,⁴⁹ few studies reported on these factors. Importantly, the included studies did not discuss the factors that impeded programme participation in the first place and results may therefore reflect only those willing and able to take part. These factors may also affect participant success. Overall, comprehensive process evaluations are notably missing from the evidence base and will be beneficial to establish programme efficacy, acceptability, sustainability and transferability in the future.

All of the interventions were implemented in academic medicine departments, which historically contain more men than women⁴⁴ and in which academics often split their time with clinical positions. Many of the programmes trialled here (including ELAM) were specifically designed for academic medicine and may not be similarly efficacious in other areas. None of the three studies that included more than one faculty compared outcomes between them. In addition, all but two studies were conducted in the USA where academic staff are less likely (than UK academics, for example) to be unionised, are more often subject to a University-specific pay scale and are able to receive tenure.⁵⁰ These structural differences may also affect success. Low-income and middle-income countries were not represented at all. The importance of intersecting oppressions (ie, gender, race, disability and so on) was also not addressed in any study, though this may reflect our review design and exclusion of studies programmes addressing multiple disadvantages (eg, for women of colour). These factors are likely to be crucial predictors of success and should be addressed during intervention design or by statistical controlling in future studies.

It was noted that the interventions trialled in the included studies generally implemented a ‘bottom-up’ approach in which women were responsible for opting in and participating. Such interventions may be unsuccessful where the woman feels her efforts would be futile or met with hostility. Few programmes with a ‘top-down’ approach, in which senior management are tasked with improving conditions for their disadvantaged colleagues, were identified in this review. This approach was successful when used: Stewart et al ³³ attribute the doubling of female staff over 2 years in their study to an institution-wide change in culture starting with senior management and full professors. The Athena Scientific Women’s Academic Network (SWAN) Charter is a widely used example of a ‘top-down’ intervention, involving evaluation and accreditation standards to recognise organisations that perform well in terms of employment practices that support equality.⁵¹ Qualitative reports about Athena SWAN have noted a high level of satisfaction with the programme and that participating organisations had changed their procedures to reflect the key principles of the charter.^{52 53} Whether this work is translating to short-term or long-term improvements in senior academic representation is yet to be reported and should be explored in future research. Some qualitative evaluations of Athena SWAN have noted concerns that women are still burdened with much of the responsibility to implement it.²⁶ Extant institutional financial support programmes and re-entry fellowships that support women returning from care leave are also good candidates for trial.

Limitations of review

Interpretation of the results of this review should take into consideration that a lack of comparable studies meant that a quantitative meta-analysis was not possible in this case. Publication bias in favour of significant findings is possible, and the lack of homogeneity in methods and outcomes of studies included in this review left us unable to formally test for this. That all included studies reported a positive outcome on at least one indicator (or did not report statistical significance) increases the risk of bias.

Conclusion

This review builds on research to date that has predominantly examined the extent of and potential contributors to gender inequality in academia. It provides some insight into the potential for low-cost interventions to address these factors and provide mutually beneficial outcomes for women staff and institutions as a whole. However, the studies identified provide limited high-quality evidence to guide programme design. Rigorous assessment of programme efficacy and sustainability is required across countries and academic departments, with appropriate comparison groups and controlling for relevant confounders. Given the importance of reducing gender bias in academia, it is perhaps best to conclude here that something is better than nothing. The success of an intervention is undermined where it relies on the additional labour of those it is intending to support. Innovative programmes that provide incentives for those with the most power to remove barriers to women in academia will likely be required to create sustainable change.