Evidence Map: reporting of results by sex or gender in randomized controlled trials with women Veteran participants (2008-2018)

Abstract

Background:

Higher participation of women in randomized controlled trials (RCTs) has not led to significantly improved reporting of sex-stratified results. A recent evidence map of research on women Veterans revealed that many studies did not report results by sex or gender.

Objective:

Compare characteristics of RCTs with women Veteran participants that did or did not report results by sex or gender. Assess how sex and gender are addressed in research with women Veterans.

Methods:

We extended the prior evidence map with a systematic search for RCTs with women Veterans, published 2008–2018. We compared characteristics of RCTs that reported results by sex or gender with those of RCTs that did not, and reviewed methodology and reporting of sex/gender analyses.

Results:

In addition to 11 studies from the prior evidence map, we assessed 1,820 abstracts for relevance and ultimately included 45 unique RCTs. Five trials included only women and 40 included both men and women (median 14.3% women). Ten studies reported results by sex or gender. These trials were larger (median study size n=343.5 vs. n=125.5) and included a higher median proportion of female participants (16.8% vs. 11.2%) than studies without sex/gender results. Ten of 11 trials that tested pharmacologic or device interventions did not report results by sex or gender.

Conclusions:

Reporting of results by sex or gender remains low in Veteran research, but may improve with larger studies and increased recruitment of women Veterans into trials. Trials of pharmacologic or device interventions may be targets for future reporting requirements. Standardization could improve attention to sex and gender in methodology and reporting.

Introduction

Rationale.

Increased attention to the inclusion of women in medical research by the National Institutes of Health (2001; US Department of Health and Human Services, 1994) and National Academy of Medicine (Wizemann & Pardue, 2001) has led to higher participation by women in clinical trials (Office of Research on Women’s Health, 2017). Despite these advances, most published research results are not disaggregated by sex or gender (Avery & Clark, 2016; Phillips & Hamberg, 2016). Grouping male and female study subjects together without attention to potential sex and gender differences can lead to overgeneralization of study results (Clayton & Tannenbaum, 2016). This practice can overestimate benefits and underestimate harms for male or female patients due to biologic (sex) or sociocultural (gender) differences in response to the intervention.

The challenge of conducting research that is meaningfully applicable to women is intensified for studies related to Veterans’ health. Although women are the fastest growing segment of the Veterans Health Administration (VA) population, they represent an extreme minority of living Veterans (10%) and VA users (7%) (Frayne, 2014). Women Veterans are projected to comprise 15.9% of the Veteran population by 2040 (Department of Veterans Affairs, 2016), so high-quality research related to the health and healthcare of women Veterans will be critical for evidence-based practice.

In our recent evidence map of women Veterans’ health and healthcare research published 2008–2015, we found a significant increase in the amount of research conducted with women Veteran participants over time (Danan, Ensrud, et al., 2017; Danan, Krebs, et al., 2017). In that report, we included 440 articles that reported results by sex or gender, including 8 randomized controlled trials (RCTs). We excluded 424 articles that included women Veterans but did not report results by sex or gender or included very few women; as a result, we were unable to examine study characteristics associated with reporting results by sex or gender.

Objectives.

The main purpose of the current study was to compare characteristics of RCTs that included women Veterans and did or did not report results by sex or gender (Q1). We focused on RCTs due to the importance of intervention research in advancing women Veterans’ health. Secondary goals were to appraise how studies with sex/gender results adhere to currently proposed guidelines or best practices (Q2) and to assess how studies without sex/gender results address sex or gender, if at all (Q3). We aimed to identify targets for researchers and policymakers to improve the quality of future research involving women Veterans. Our key questions were:

Q1: How do RCTs that include women Veterans and report results by sex or gender differ from RCTs that include women Veterans but do not report results by sex or gender?

Q2: Among RCTs with women Veterans that report results by sex or gender, do sex/gender analyses and reporting adhere to currently proposed best practices?

Q3: Among RCTs with women Veterans that do not report results by sex or gender, how are sex/gender addressed in publications, if at all?

Methods

We used systematic review procedures to create a targeted evidence map comparing characteristics of RCTs published from 2008 to 2018 that included women Veterans and did or did not report results by sex or gender. A recent systematic review identified 5 key characteristics of evidence maps: (1) systematic process, (2) capture a broad field, (3) identify gaps or future research needs, and (4) produce a user-friendly product with a (5) visual depiction. (Miake-Lye, Hempel, Shanman, & Shekelle, 2016). Though initially developed in parallel, evidence maps are increasingly considered a type of scoping review, for which a new checklist (the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews, or PRISMA-ScR) can be applied (Tricco et al., 2018). We report this review according to the PRISMA-ScR checklist.

Protocol, information sources, and search strategy.

The current paper is a focused extension of a recent systematic review, for which we created a broad evidence map of the literature related to women Veterans’ health and healthcare from 2008 to 2015. Full methods for the prior review have been described (Danan, Ensrud, et al., 2017; Danan, Krebs, et al., 2017). Briefly, we searched MEDLINE (Ovid), CINAHL, and the VA Health Services Research and Development database for articles published between January 2008 and December 2015 using the MeSH terms: Women; Women’s Health; Women’s Health Services; Veterans; Veterans Health; and Hospitals, Veterans. That review included 440 articles that provided results by sex or gender, of which 8 were randomized controlled trials (RCTs) and 12 were secondary analyses of RCTs (3 of which described trials with male and female participants). The prior review identified and excluded 424 articles related to women Veterans’ health that did not provide results by sex or gender or included a very small proportion of women Veterans. These 424 previously excluded articles formed the basis for the current review. For the current review, we also updated our results by repeating the identical MEDLINE search for articles published January 2015 to May 2018 (Appendix 1).

Eligibility criteria.

The current review differed from the prior review by including only RCTs, excluding studies in which participants were VA clinicians or clinics, and eliminating the minimum threshold for number of women Veteran participants. Other study eligibility criteria were unchanged from the prior review. We excluded studies that were not related to health/healthcare, did not include women US Veterans, or included only active duty military. Studies with <75% Veterans (e.g., multi-site trials with one VA site) were excluded if they did not report results stratified by Veteran status.

Selection of sources of evidence.

Abstracts were dual-reviewed for exclusion criteria by an investigator (ED) and research associate (KU). Agreement on exclusion was 97% (Cohen’s κ=0.81) and conflicts were resolved by consensus. Trials or interventions that were uncontrolled (single-arm) or used non-random treatment assignment were identified (using abstracts or full-text if needed) but did not undergo data extraction. Each RCT was counted only once, using either the primary outcomes paper (with or without results by sex or gender) or a secondary analysis paper that provided results by sex or gender (if not presented in the parent trial). Agreement on classification as an RCT was 86% (κ=0.63) and conflicts were resolved by consensus.

Data charting process.

Original RCTs and secondary analyses meeting eligibility criteria underwent data abstraction for 12 study characteristics by one investigator or research associate (ED, EY, RK, or KU). Each article abstraction was checked by a second reviewer and conflicts were resolved by consensus. Study characteristics of interest were selected a priori based on our prior review and opinion of participating investigators, who have extensive professional experience with the women Veterans’ health and healthcare literature. Throughout the initial review process, group consensus was used to refine or adjust extracted characteristics as needed to address feasibility issues.

Data items.

The characteristics abstracted were publication year, healthcare topic (grouped under 4 headings: (1) mental health, (2) physical health, (3) healthcare organization and delivery, and (4) access, utilization, and post-deployment health), sample size (n randomized), number and proportion of women participants, study location(s) (single or multi-site, VA Cooperative Studies Program or VA Women’s Health Practice Based Research Network studies, non-VA/community based), time to longest follow-up (converted to days), intervention type (pharmacologic or behavioral), control type (active comparator, placebo, attention control, usual care, waitlist control), outcome type(s) (patient reported, objective clinical measure, resource utilization or cost), and funding source(s) (VA, DOD, NIH, industry, university, foundation, or not reported).

We recorded journal titles and cross-referenced them with an online list of journals that report adhering to the International Committee of Medical Journal Editors (ICMJE) recommendations for scholarly work (2017). The ICMJE recommendations state that “separate reporting of data by demographic variables, including age and sex… should be routine, unless there are compelling reasons not to stratify reporting.”

We determined whether results were reported by sex or gender for any outcomes. This was defined as sub-group results for women for the primary study outcome and/or an explicit assessment of the interaction between the treatment and sex or gender. Interrater agreement on the presence of results by sex or gender was 94% (κ=0.80). Conflicts were resolved by consensus.

Synthesis of results by key question (Figure 1):

Figure 1.

Sources of evidence for key questions

Q1: How do RCTs that report results by sex or gender differ from RCTs that do not? Across all included trials, we used descriptive statistics and visual graphing to describe trends in study characteristics. Among trials that included both male and female Veterans, we compared characteristics of those that did provide results by sex or gender with those that did not.

Q2: Among RCTs that report sex/gender results, do sex/gender analyses and reporting adhere to currently proposed best practices? The principal investigator (ED) reviewed studies classified as reporting results by sex or gender for attention to sex/gender in hypothesis, design, analysis, reporting, and acknowledgement of limitations (Table 2). Criteria for appraisal were adapted using the Sex and Gender Equity in Research (SAGER) guidelines (Heidari, Babor, De Castro, Tort, & Curno, 2016), with additional attention to relevant literature (Aulakh & Anand, 2007; Johnson, Greaves, & Repta, 2009; McGregor et al., 2016; Nieuwenhoven & Klinge, 2010). In the absence of widely accepted guidelines or requirements for reporting results by sex or gender, this assessment was intended to be descriptive.

Table 2.

Attention to sex/gender among 10 trials that reported results by sex or gender

	Article ID number
	1	2	3	4	5	6	7	8	9	10
Publication Year	2010	2015		2016		2017			2018
Hypothesis
Explicitly stated hypothesis		X	X
Suggested relationship or prior sex-specific findings cited	X				X				X
Study design
Explicitly an article about sex/gender differences		X			X
Oversampling or enhanced recruitment of women							X			X
Sex/gender-specific inclusion/exclusion criteria					X			X
Randomization stratified or blocked by sex			X
Sex/gender balanced between treatment arms	X		X	X	X	X	X	X		X
Statistical analysis
Power calculation for interaction								X
Interaction test (sex/gender by treatment group)	X	X	X	X		X	X	X	X
Reporting
Gender of patients lost/withdrawn post-randomization reported	X						X
Sex/gender analysis described in introduction or methods	X	X	X	X	X		X		X
Statistically significant sex/gender by treatment interaction	X	X				X
Any differential treatment effect by sex/gender reported	X	X			X	X			X
Full sex-disaggregated results reported for primary outcome	X	X			X	X			X	X
Limitations acknowledged
Small proportion of women limits generalizability				X			X			X
Sub-group analysis lacks power, interpret with caution, replicate	X				X			X

Q3: Among RCTs that do not report results by sex or gender, how are sex/gender addressed in publications, if at all? For studies that included men and women but did not report results by sex or gender, 2 reviewers identified and described mentions of sex or gender in the methods, results or discussion sections (Appendix 2). If articles referenced online supplemental materials, these were also reviewed for the presence of results by sex or gender (online tables) and/or sex-specific recruitment or eligibility criteria (online protocols).

Results

Selection of sources of evidence (Figure 2).

Figure 2.

Literature Flow Diagram: screening and selection of included articles

From the prior review (2008–2015), we identified 424 articles that did not report results by sex or gender (n=361) or included very few women (n=63). Our updated search (2015–2018) produced 1,461 citations. After removing duplicates, 1,820 abstracts from the combined searches underwent dual review and 1,668 were excluded. Next, we added 11 studies from the prior review that did report results by sex or gender for a total of 163 intervention studies, and then excluded 98 ineligible studies. Of 24 secondary analyses, 22 did not meet inclusion criteria. Two of the ineligible secondary analyses described parent trials that had not been previously captured by our search due to lack of keywords in the title or abstract. These parent trials were reviewed and included. The final sample included 45 trials: 43 papers reporting primary outcomes, and 2 secondary analysis papers focused on sex/gender differences in the original trial (Appendix 3).

Characteristics and results of sources of evidence

All included trials (n=45).

Of the 45 included trials, 5 included only women Veteran participants and 40 included both male and female Veteran participants (Table 1, Figure 3, Appendix 3). Twenty-six trials (58%) covered mental health topics, including PTSD (9), comorbid mental and physical health problems (6), substance abuse (5) and military sexual trauma (2). Fourteen trials (31%) covered physical health topics including chronic pain (4), diabetes (2), hypertension (2), and long-term care and aging (2). Three trials (7%) addressed healthcare organization and delivery (all telehealth interventions) and two studies addressed healthcare utilization and post-deployment health. These 45 articles were published in 36 distinct medical journals, 15 of which reported adherence to the ICMJE recommendations.

Table 1.

Characteristics of included trials

Trial characteristics	Women and men veteran participants		100% Women veteran participants (n=5) n(%) or median(IQR)
	NO results by sex or gender (n=30) n(%) or median(IQR)	Results by sex or gender (n=10) n(%) or median(IQR)
n randomized participants	125.5 (56.5–295.3)	343.5 (113.5–765.3)	86 (58.5–2813)
% women participants	11.2 (4.9–21.7)	16.8 (13–40.4)	100%
Time to longest follow-up (days)	182.4 (91.2–365)	228 (144.4–365)	182.4 (45.6–456.2)
Health care topic
Mental health	18 (60)	4 (40)	4 (80)
Physical health	9 (30)	4 (40)	1 (20)
Health care delivery	3 (10)	0 (0)	0 (0)
Access, Utilization, PDH	0 (0)	2 (20)	0 (0)
Study location(s)
Single site	20 (67)	4 (40)	3 (60)
Multi-site	10 (33)	3 (30)	1 (20)
VA Cooperative study	3 (10)	0 (0)	0 (0)
WH PBRN study	0 (0)	0 (0)	0 (0)
Non-VA or Community based	2 (7)	2 (20)	2 (40)
Intervention type
Pharmacologic	8 (27)	1 (10)	0 (0)
Behavioral	11 (37)	7 (70)	4 (80)
Health services	9 (30)	2 (20)	1 (20)
Device or Physical treatment	2 (7)	0 (0)	0 (0)
Control type
Active comparator	13 (43)	5 (50)	4 (80)
Placebo	6 (20)	0 (0)	0 (0)
Attention control	2 (7)	1 (10)	0 (0)
Usual care	6 (20)	3 (30)	0 (0)
Waitlist control	3 (10)	1 (10)	1 (20)
Outcome type(s)
Patient-reported measure	24 (80)	10 (100)	5 (100)
Objective clinical measure	11 (37)	4 (40)	1 (20)
Funding source(s)
VA	24 (80)	9 (90)	2 (40)
DOD	4 (13)	1 (10)	2 (40)
NIH/other government	3 (10)	2 (20)	1 (20)
Industry	3 (10)	0 (0)	0 (0)
University	0 (0)	0 (0)	0 (0)
Foundation	3 (10)	1 (10)	0 (0)
Not reported	0 (0)	0 (0)	1 (20)
Mention of sex/gender
Methods	12 (40)	8 (80)
Results	20 (67)	10 (100)
Discussion	14 (47)	9 (90)

Figure 3.

Evidence Map: randomized controlled trials with veteran participants, by proportion women, reporting of results by sex or gender, and publication year

Four of the 5 trials that included all women Veteran participants were small local trials (n=51–126) about mental health topics that were published since 2014. The fifth, a large study of mammography promotion (n=5500), was a multisite trial published in 2008. Only one of the 5, a substance abuse trial, received VA funding.

Across 40 trials that included male and female participants, there were a total of 1,928 women; the median proportion of female participants was 14.3%. One third of trials (13 of 40) included less than 10% women, and half (20 of 40) had 10–33% women; only 7 trials included more than 33% women. The number of trials published per year increased substantially in the latter half of our time period: from 1–3 per year (2008–2012) to 4–9 per year (2013–2018). (Figure 3).

Synthesis of results by key question:

Q1: How do RCTs that report results by sex or gender differ from RCTs that do not?

Trials that reported results by sex or gender (n=10).

Ten of 40 trials that included both men and women participants reported results by sex or gender (25%) (Table 1). Nine of 10 trials that reported results by sex or gender were published in 2015 or later (Figure 3). Trials that reported results by sex or gender were larger and included a higher proportion of women than those that did not. These articles were published in 10 different journals, half of which reported adherence to the ICMJE recommendations.

Trials that did not report results by sex or gender (n=30).

Several characteristics were observed more frequently among the 30 trials that did not report results by sex or gender (Table 1). They were more likely to be single-site studies. Notably, however, the 3 multisite VA Cooperative Studies Program trials did not report results by sex or gender. Most pharmacologic or device trials, all 6 studies with a placebo control, all 3 studies with industry funding, and 3 of 4 studies with foundation funding did not report results by sex or gender. Fourteen of 30 articles were published in journals that reported adherence to the ICMJE recommendations.

Q2: Among RCTs that report sex/gender results, do sex/gender analyses and reporting adhere to currently proposed best practices? The 10 trials that reported results by sex or gender paid varied attention to sex and gender with respect to hypothesis, design, analysis, reporting, and acknowledgement of limitations (Table 2).

Q3: Among RCTs that do not report results by sex or gender, how are sex/gender addressed in publications, if at all? The 30 trials that did not report results by sex or gender paid limited attention to sex and gender (Appendix 2).

Discussion

Summary of evidence.

In this evidence map of 45 RCTs with women Veteran participants published in the past decade, we found 10 studies that reported results by sex or gender, 30 that did not, and 5 studies with only female participants. To answer our first key question, we identified study characteristics that differed between RCTs that did or did not report results by sex or gender, including sample size, proportion female, and intervention type. Publication in a journal that reports adherence to ICMJE recommendations, or utilization of a multi-site research mechanism such as the VA Cooperative Studies program, did not predict reporting of results by sex or gender. For our second key question, we found that important features of appropriate sex and gender reporting were often missing from studies reporting results by sex or gender. For our third key question, we found that most studies that did not report results by sex or gender paid little attention to the role of sex or gender in published methods, results or discussion sections. We review the significance of each of these findings below, with suggestions for action by research stakeholders.

Participation of women Veterans in trials

Among 40 trials that included both men and women Veterans, only 14.3% of trial participants were women, a far smaller proportion than in published general population trials (37–41% women) (Avery & Clark, 2016; Geller, Koch, Pellettieri, & Carnes, 2011). However, participation by women Veterans in trials was double their representation among VA users (7%) and higher than their representation among living Veterans (10%) (Department of Veterans Affairs, 2016; Frayne, 2014). Only 1 of 13 trials that included less than 10% women reported results by sex or gender. In contrast, among 27 trials with at least 10% women, one-third reported results by sex or gender. Achieving higher participation rates by women is a necessary, though not sufficient, first step toward improving generalizability of Veterans’ health research.

Outside VA, funding agencies have contributed to increased enrollment of women in clinical trials. The NIH’s Advisory Committee on Research on Women’s Health requires the inclusion of women and publicly reports the proportion of women enrolled in NIH-funded research, which has approached 50% for the past decade (2008; 2017). The Canadian Institutes of Health Research has created a cross-cutting sex and gender platform within large research consortia to provide guidance to research teams (Duchesne, Tannenbaum, & Einstein, 2017).

VA Office of Research and Development (ORD), a major funder of Veterans’ health research and the sponsor of this special issue, has long required that “special efforts must be made when scientifically appropriate to include women Veterans” in research studies. While data on women’s participation in VA trials have been collected and reported to oversight groups such as Institutional Review Boards or Data Monitoring Committees, these data have not been publicly available. Since 2013, VA ORD has required that results from its registered clinical trials be submitted to ClinicalTrials.gov, including the number of men and women enrolled. This requirement was retroactively applied to trials that were initiated and or actively enrolling since 2007 (Huang, Altemose, & O’Leary, 2017). This public registry will enable VA and other interested stakeholders to track the inclusion of women in VA clinical trials over time.

In addition, since 2010, VA Health Services Research & Development has invested in the Women’s Health Research Network, which includes a Practice Based Research Network that supports investigators seeking to recruit more women in multi-site observational and intervention studies and a Research Consortium that supports conduct of women’s health research within VA. We did not identify any RCTs that reported using the Practice Based Research Network in this review. Given the long lag-time between planning and publishing multi-site studies, it is probably too soon to draw conclusions about effects of these initiatives on women Veterans’ research participation.

Reporting of results by sex or gender

The proportion of trials that reported results by sex or gender in this review (25%) was within the range of previously published reviews of non-Veteran-specific RCTs from a similar time frame (13–48%) (Avery & Clark, 2016; Geller, Adams, & Carnes, 2006; Geller et al., 2011; Phillips & Hamberg, 2016). Certain study types, such as those evaluating pharmacologic and device interventions, may be particularly important targets for sex and gender research equity (Carey et al., 2017). In 2001, the US General Accounting Office reported that 8 of 10 drugs removed from the market in the preceding years had more significant (and some potentially fatal) adverse effects for women than men (Heinrich, Gahart, Rowe, & Bradley, 2001). Since 2001, the NIH and FDA have required phase III pharmaceutical and device trials to incorporate sex-specific analyses (National Institutes of Health, 2001); a November 2017 amendment states these results must be reported on ClinicalTrials.gov (National Institutes of Health, 2017).

Among 11 pharmacologic or device studies, only 1 presented results by sex or gender and 7 included very few women. For example, a VA Cooperative Studies Program trial that evaluated prazosin for PTSD and reported a potentially practice-changing finding included only 7 women among 304 participants (Raskind et al., 2018). This is troubling because PTSD treatment needs may vary by sex, given evidence that military-related sexual trauma has a much stronger relationship with PTSD for women Veterans (Kimerling et al., 2010). Likewise, the efficacy and side effects of prazosin likely vary by sex; indeed, the prazosin study used different dosing protocols for men and women. Nonetheless, the publication did not describe under-representation of women as a limitation.

Although the ICMJE recommends routine reporting of sex-stratified results, we found little evidence that medical journal editors are enforcing this expectation (2017). Of 19 included trials published in 16 journals that state they follow the ICMJE guidelines, only 5 reported results by sex or gender. In 2016, the European Association of Science Editors proposed guidelines for Sex and Gender Equity in Research (SAGER) that can be incorporated into journal instructions to authors and peer-review forms (Heidari et al., 2016). However, the SAGER guidelines have not yet been widely implemented. The most widely used set of guidelines for randomized trial reporting, the CONSORT statement, makes no specific recommendation about sex/gender results reporting.

Without prioritization of sex/gender stratified reporting, competing reporting requirements and strict journal word limits may contribute to omission of sex/gender results. The single pharmacologic intervention trial in this review that reported results by sex or gender, led by an author of the current report (EK) and published in a journal with a 3000-word limit, included sex-stratified results in an online supplemental appendix (Krebs et al., 2018). The only in-text mention of the sex subgroup analysis was cut in the pre-publication editorial process. Because we did not speak with authors of other trials included in this review, we do not know how often omissions of sex/gender results are due to authors’ versus editors’ decisions.

Areas for improved attention to sex and gender

Across 10 trials that reported results by sex or gender, we identified targets for improvement with respect to study design, analysis and reporting. Few trials presented a hypothesis for the relationship between sex/gender and the intervention to justify the analysis and most trials did not block or stratify randomization by sex, nor report the sex of participants who withdrew early. Only one of the five trials that reported a non-significant sex/gender by treatment interaction also provided full results disaggregated by sex.

Sex-disaggregated outcomes should be routinely reported regardless of statistical test results because interaction tests for sub-group analyses are often underpowered and, conversely, because statistically significant findings can be due to chance. Further, routine reporting of sex-disaggregated results, regardless of findings, allows for meta-analyses by sex in the future and may reduce unnecessary repetition in research (Clayton & Tannenbaum, 2016). In the VA, where women represent an extreme minority of the population, techniques that combine data from multiple studies to overcome sample size limitations (Wizemann & Pardue, 2001) may be especially relevant.

Among 30 trials that did not report results by sex or gender, less than half remarked on sex/gender limitations in the discussion. Investigators that do not incorporate sex or gender analyses into the design, execution or analysis of their trial must at least critique this omission (Johnson et al., 2009).

Shortcomings we identified are not restricted to Veteran research; previous literature reviews have also found methodologic shortcomings in subgroup analyses by sex (Aulakh & Anand, 2007). Inattention to sex and gender reduces research equity, but poorly executed sex and gender analyses also have risks. For example, overinterpretation of post-hoc subgroup analyses as conclusive, rather than hypothesis generating, can lead providers to withhold potentially beneficial treatments from women (Aulakh & Anand, 2007) or overuse potentially harmful treatments (Rothwell, 2005). Most investigators lack training in sex/gender equity in research design, conduct, analysis and reporting. Although workshops and publications are available for individual investigators seeking to build this skillset (Johnson et al., 2009; Nieuwenhoven & Klinge, 2010), uniform requirements for training tied to research funding may reach more researchers (Duchesne et al., 2017). Within the VA, the Women’s Health Research Network can provide expert guidance for investigators to facilitate improved methodology and design (Rohrer, Gierisch, Fish, Blakeney, & Bastian, 2011), but investigators do not always take advantage of this resource.

Limitations.

Limitations of this review include search criteria specific to women, which led us to miss some RCTs that did not reference women in the abstract or MeSH terms. As a result, we may have overestimated the proportion of all RCTs that report results by sex or gender. Our analysis is limited by the information provided by study authors in published manuscripts or linked online materials. We restricted our reporting to published data, but some trials listed eligibility criteria related to sex/gender on ClinicalTrials.gov that were not mentioned in the published article. Finally, results apply only to RCTs. Our exclusion of single-arm pilots and implementation/evaluation projects created a more uniform cohort of included studies for comparison purposes, but omitted research that is also important to women Veterans’ health and deserves similar scrutiny.

Implications for Practice and Policy.

Several actions by research stakeholders could improve sex and gender sensitivity in Veteran research. Funders could set more specific expectations for women’s inclusion, track and publicly report women’s participation, and offer additional support to increase participation of women in multi-site VA trials. Journal editors could adopt and enforce sex/gender reporting guidelines, including a requirement to report results of clinical trials disaggregated by sex. Investigators should target a higher proportion of female study subjects and/or a larger overall study size in order to make reporting results disaggregated by sex/gender feasible. They should acknowledge the role of sex/gender in their study, including the limitations of not providing results by sex or gender. To improve the quality of reporting, investigators can seek training and use existing resources for the conduct, analysis, and reporting of sex- and gender-equitable research.

Conclusions.

In summary, this systematic review found that women Veterans are increasingly participating in RCTs, but reporting of results by sex or gender remains infrequent. In particular, trials with less than 10% women and trials testing pharmacologic and device interventions were unlikely to report results by sex or gender. Trials that did report results by sex or gender often omitted important details related to study design. Reporting of full sex-disaggregated results, regardless of statistical test outcomes, was rare. Implementing the suggested approaches will increase applicability of the knowledge gained from Veteran research to the care of women.

Appendix 1.

MEDLINE search strategy

1	Women/
2	Women’s Health/
3	Women’s Health Services/
4	Transgendered Persons/
5	(reproductive or reproduction or pregnan* or birth* or fertility or infertility or infertile or menstrual or menstruation or menses or urinary tract or sexually transmitted or hiv or cervical or ovarian or genital or gynecologic* or breast or congenital abnormalities).mp.
6	1 or 2 or 3 or 4 or 5
7	Veterans/ or Veterans Health/ or “United States Department of Veterans Affairs”/
8	(post deployment or post-deployment).mp.
9	(VA or VHA or VAMC or veteran*).mp.
10	Hospitals, veterans/
11	7 or 8 or 9 or 10
12	6 and 11
13	((women or LGBT or gay or lesbian or bisexual or female or transgender or transsexual or queer) and veteran*).ti,ab.
14	12 or 13
15	Limit 14 to (English language and yr=”2008-Current”)

Appendix 2.

Attention to sex/gender among 30 trials that did not report results by sex or gender

	Article ID number
	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40
Publication Year	2008		2009			2011		2012		2013			2014			2015			2016		2017						2018
Methods
Randomization stratified by sex/gender									X																X		X
Sex-specific eligibility criteria (pregnancy related)	X																									X				X
Sex-specific eligibility criteria (non-pregnancy related)																							X				X
Potential participant pool described as mostly male						X
Men and women eligible for study												X								X	X								X
Results
Proportion of male/female participants reported	X	X			X		X		X		X			X			X	X		X		X	X	X		X	X	X		X
Balance of sex/gender across treatment arms reported	X	X											X	X	X					X					X	X				X
Discussion
Mostly male population limits generalizability			X				X					X	X				X					X	X	X			X	X
Future research should study sex/gender effect	X																									X
Mostly male population (descriptive)																													X	X

Appendix 3.

All included trials

Publication Year	Author	Title	Journal	N	% women	Sex-specific results	Article ID #
2010	Baker, L. D.,Frank, L. L.,Foster-Schubert, K.,Green, P. S.,Wilkinson, C. W.,McTiernan, A.,Plymate, S. R.,Fishel, M. A.,Watson, G. S.,Cholerton, B. A.,Duncan, G. E.,Mehta, P. D.,Craft, S.	Effects of aerobic exercise on mild cognitive impairment: a controlled trial	Arch Neurol	33	51.5	Yes	1
2015	Gallegos, A. M.,Wolff, K. B.,Streltzov, N. A.,Adams, L. B.,Carpenter-Song, E.,Nicholson, J.,Stecker, T.	Gender Differences in Service Utilization among OEF/OIF Veterans with Posttraumatic Stress Disorder after a Brief Cognitive-Behavioral Intervention to Increase Treatment Engagement: A Mixed Methods Study	Womens Health Issues	273	12.8	Yes	2
2015	Sayer, N. A.,Noorbaloochi, S.,Frazier, P. A.,Pennebaker, J. W.,Orazem, R. J.,Schnurr, P. P.,Murdoch, M.,Carlson, K. F.,Gravely, A.,Litz, B. T.	Randomized Controlled Trial of Online Expressive Writing to Address Readjustment Difficulties Among U.S. Afghanistan and Iraq War Veterans	Journal of Traumatic Stress	1292	39.3	Yes	3
2016	Goodman, M.,Banthin, D.,Blair, N. J.,Mascitelli, K. A.,Wilsnack, J.,Chen, J.,Messenger, J. W.,Perez-Rodriguez, M. M.,Triebwasser, J.,Koenigsberg, H. W.,Goetz, R. R.,Hazlett, E. A.,New, A. S.	A Randomized Trial of Dialectical Behavior Therapy in High-Risk Suicidal Veterans	Journal of Clinical Psychiatry	91	33	Yes	4
2016	Vimalananda, V.,Damschroder, L.,Janney, C. A.,Goodrich, D.,Kim, H. M.,Holleman, R.,Gillon, L.,Lutes, L.	Weight loss among women and men in the ASPIRE-VA behavioral weight loss intervention trial	Obesity	481	15	Yes	5
2017	Erickson, Z. D.,Kwan, C. L.,Gelberg, H. A.,Arnold, I. Y.,Chamberlin, V.,Rosen, J. A.,Shah, C.,Nguyen, C. T.,Hellemann, G.,Aragaki, D. R.,Kunkel, C. F.,Lewis, M. M.,Sachinvala, N.,Sonza, P. A.,Pierre, J. M.,Ames, D.	A Randomized, Controlled Multisite Study of Behavioral Interventions for Veterans with Mental Illness and Antipsychotic Medication-Associated Obesity	Journal of General Internal Medicine	121	16.5	Yes	6
2017	Sudore, R. L.,Boscardin, J.,Feuz, M. A.,McMahan, R. D.,Katen, M. T.,Barnes, D. E.	Effect of the PREPARE Website vs an Easy-to-Read Advance Directive on Advance Care Planning Documentation and Engagement Among Veterans: A Randomized Clinical Trial	JAMA Internal Medicine	414	9.2	Yes	7
2017	ER Pedersen, L Parast, GN Marshall, TL Schell, C Neighbors	A randomized controlled trial of a web-based, personalized normative feedback alcohol intervention for young-adult veterans.	Journal of consulting and clinical psychology	784	17	Yes	8
2018	Grubbs, K. M.,Fortney, J. C.,Pyne, D.,Ray, J.,Hudson, T. J.	A Comparison of Collaborative Care Outcomes in Two Health Care Systems: VA Clinics and Federally Qualified Health Centers	Psychiatric Services	759	43.5	Yes	9
2018	Krebs, E. E.,Gravely, A.,Nugent, S.,Jensen, A. C.,DeRonne, B.,Goldsmith, E. S.,Kroenke, K.,Bair, M. J.,Noorbaloochi, S.	Effect of Opioid vs Nonopioid Medications on Pain-Related Function in Patients With Chronic Back Pain or Hip or Knee Osteoarthritis Pain: The SPACE Randomized Clinical Trial	JAMA	240	13	Yes	10
2008	Davis, L. L.,Davidson, J. R.,Ward, L. C.,Bartolucci, A.,Bowden, C. L.,Petty, F.	Divalproex in the treatment of posttraumatic stress disorder: a randomized, double-blind, placebo-controlled trial in a veteran population	J Clin Psychopharmacol	85	2.4	No	11
2008	Kilbourne AM, Post EP, Nossek A, Drill L, Cooley S, Bauer MS	Improving medical and psychiatric outcomes among individuals with bipolar disorder: a randomized controlled trial		61	8.2	No	12
2009	Abraira, C.,Duckworth, W. C.,Moritz, T.,Vadt Group	Glycaemic separation and risk factor control in the Veterans Affairs Diabetes Trial: an interim report	Diabetes Obes Metab	1791	2.9	No	13
2009	Edinger, J. D.,Olsen, M. K.,Stechuchak, K. M.,Means, M. K.,Lineberger, M. D.,Kirby, A.,Carney, C. E.	Cognitive behavioral therapy for patients with primary insomnia or insomnia associated predominantly with mixed psychiatric disorders: a randomized clinical trial	Sleep	81	13.6	No	14
2009	Bosworth, Hayden B,Olsen, Maren K,Dudley, Tara,Orr, Melinda,Goldstein, Mary K,Datta, Santanu K,McCant, Felicia,Gentry, Pam,Simel, David L,Oddone, Eugene Z	Patient education and provider decision support to control blood pressure in primary care: a cluster randomized trial	American heart journal	588	2	No	15
2011	Pyne, J. M.,Fortney, J. C.,Curran, G. M.,Tripathi, S.,Atkinson, J. H.,Kilbourne, A. M.,Hagedorn, H. J.,Rimland, D.,Rodriguez-Barradas, M. C.,Monson, T.,Bottonari, K. A.,Asch, S. M.,Gifford, A. L.	Effectiveness of collaborative care for depression in human immunodeficiency virus clinics	Arch Intern Med	249	2.8	No	16
2011	Wakefield, B. J.,Holman, J. E.,Ray, A.,Scherubel, M.,Adams, M. R.,Hillis, S. L.,Rosenthal, G. E.	Effectiveness of home telehealth in comorbid diabetes and hypertension: a randomized, controlled trial	Telemed J E Health	302	2	No	17
2012	Epstein, D. R.,Sidani, S.,Bootzin, R. R.,Belyea, M. J.	Dismantling multicomponent behavioral treatment for insomnia in older adults: a randomized controlled trial	Sleep	179	64	No	18
2012	Bastian, Lori A,Fish, Laura J,Gierisch, Jennifer M,Rohrer, Lesley D,Stechuchak, Karen M,Grambow, Steven C	Comparative effectiveness trial of family-supported smoking cessation intervention versus standard telephone counseling for chronically ill veterans using proactive recruitment	Comparative Effectiveness Research	471	8.5	No	19
2013	Kilbourne, A. M.,Goodrich, D. E.,Lai, Z.,Post, E. P.,Schumacher, K.,Nord, K. M.,Bramlet, M.,Chermack, S.,Bialy, D.,Bauer, M. S.	Randomized controlled trial to assess reduction of cardiovascular disease risk in patients with bipolar disorder: the Self-Management Addressing Heart Risk Trial (SMAHRT)	J Clin Psychiatry	126	17	No	20
2013	Kip, K. E.,Rosenzweig, L.,Hernandez, D. F.,Shuman, A.,Sullivan, K. L.,Long, C. J.,Taylor, J.,McGhee, S.,Girling, S. A.,Wittenberg, T.,Sahebzamani, F. M.,Lengacher, C. A.,Kadel, R.,Diamond, D. M.	Randomized controlled trial of accelerated resolution therapy (ART) for symptoms of combat-related posttraumatic stress disorder (PTSD)	Mil Med	57	19.3	No	21
2013	Suris, A.,Link-Malcolm, J.,Chard, K.,Ahn, C.,North, C.	A randomized clinical trial of cognitive processing therapy for veterans with PTSD related to military sexual trauma	J Trauma Stress	86	84.9	No	22
2014	Darouiche, R. O.,Al Mohajer, M.,Siddiq, D. M.,Minard, C. G.	Short versus long course of antibiotics for catheter-associated urinary tract infections in patients with spinal cord injury: a randomized controlled noninferiority trial	Arch Phys Med Rehabil	55	5.5	No	23
2014	Yehuda, R.,Pratchett, L. C.,Elmes, M. W.,Lehrner, A.,Daskalakis, N. P.,Koch, E.,Makotkine, I.,Flory, J. D.,Bierer, L. M.	Glucocorticoid-related predictors and correlates of post-traumatic stress disorder treatment response in combat veterans	Interface Focus	37	10.8	No	24
2014	Ziemba SJ, Bradley NS, Landry LA, Roth CH, Porter LS, Cuyler RN	Posttraumatic stress disorder treatment for Operation Enduring Freedom/Operation Iraqi Freedom combat veterans through a civilian community-based telemedicine network	Telemedicine and e-Health	18	10	No	25
2015	Folmer, R. L.,Theodoroff, S. M.,Casiana, L.,Shi, Y.,Griest, S.,Vachhani, J.	Repetitive Transcranial Magnetic Stimulation Treatment for Chronic Tinnitus: A Randomized Clinical Trial	JAMA Otolaryngol Head Neck Surg	70	20	No	26
2015	Watts, B. V.,Schnurr, P. P.,Zayed, M.,Young-Xu, Y.,Stender, P.,Llewellyn-Thomas, H.	A randomized controlled clinical trial of a patient decision aid for posttraumatic stress disorder	Psychiatr Serv	132	7.6	No	27
2015	Bair, M. J.,Ang, D.,Wu, J.,Outcalt, S. D.,Sargent, C.,Kempf, C.,Froman, A.,Schmid, A. A.,Damush, T. M.,Yu, Z.,Davis, L. W.,Kroenke, K.	Evaluation of Stepped Care for Chronic Pain (ESCAPE) in Veterans of the Iraq and Afghanistan Conflicts: A Randomized Clinical Trial	JAMA Internal Medicine	241	11.6	No	28
2016	Arain, S. R.,Freed, J. K.,Novalija, J.,Pagel, P. S.,Ebert, T. J.	Short-Term Angiotensin Subtype 1 Receptor Blockade Does Not Alter the Circulatory Responses to Sympathetic Nervous System Modulation in Healthy Volunteers Before and During Sevoflurane Anesthesia: Results of a Pilot Study	Journal of Cardiothoracic & Vascular Anesthesia	14	42.9	No	29
2016	Leung, A.,Shukla, S.,Fallah, A.,Song, D.,Lin, L.,Golshan, S.,Tsai, A.,Jak, A.,Polston, G.,Lee, R.	Repetitive Transcranial Magnetic Stimulation in Managing Mild Traumatic Brain Injury-Related Headaches	Neuromodulation	29	10	No	30
2017	Ebert, T. J.,Novalija, J.,Barney, J. A.,Uhrich, T. D.,Arain, S. R.,Freed, J. K.,Pagel, P. S.	Moderate, Short-Term, Local Hyperglycemia Attenuates Forearm Endothelium-Dependent Vasodilation After Transient Ischemia-Reperfusion in Human Volunteers	Journal of Cardiothoracic & Vascular Anesthesia	8	37.5	No	31
2017	Groessl, E. J.,Liu, L.,Chang, D. G.,Wetherell, J. L.,Bormann, J. E.,Atkinson, J. H.,Baxi, S.,Schmalzl, L.	Yoga for Military Veterans with Chronic Low Back Pain: A Randomized Clinical Trial	American Journal of Preventive Medicine	150	26	No	32
2017	Hauser, P.,Fuller, B.,Ho, S. B.,Thuras, P.,Kern, S.,Dieperink, E.	The safety and efficacy of baclofen to reduce alcohol use in veterans with chronic hepatitis C: a randomized controlled trial	Addiction	180	1.7	No	33
2017	Heapy, A. A.,Higgins, D. M.,Goulet, J. L.,LaChappelle, K. M.,Driscoll, M. A.,Czlapinski, R. A.,Buta, E.,Piette, J. D.,Krein, S. L.,Kerns, R. D.	Interactive Voice Response-Based Self-management for Chronic Back Pain: The COPES Noninferiority Randomized Trial	JAMA Internal Medicine	125	22.4	No	34
2017	Kilbourne, A. M.,Barbaresso, M. M.,Lai, Z.,Nord, K. M.,Bramlet, M.,Goodrich, D. E.,Post, E. P.,Almirall, D.,Bauer, M. S.	Improving Physical Health in Patients With Chronic Mental Disorders: Twelve-Month Results From a Randomized Controlled Collaborative Care Trial	Journal of Clinical Psychiatry	293	15.4	No	35
2017	Rasmusson, A. M.,Marx, C. E.,Jain, S.,Farfel, G. M.,Tsai, J.,Sun, X.,Geracioti, T. D.,Hamner, M. B.,Lohr, J.,Rosse, R.,Summerall, L.,Naylor, J. C.,Cusin, C.,Lang, A. J.,Raman, R.,Stein, M. B.	A randomized controlled trial of ganaxolone in posttraumatic stress disorder	Psychopharmacology	112	21.4	No	36
2018	Bradley, K. A.,Bobb, J. F.,Ludman, E. J.,Chavez, L. J.,Saxon, A. J.,Merrill, J. O.,Williams, E. C.,Hawkins, E. J.,Caldeiro, R. M.,Achtmeyer, C. E.,Greenberg, D. M.,Lapham, G. T.,Richards, J. E.,Lee, A. K.,Kivlahan, D. R.	Alcohol-Related Nurse Care Management in Primary Care: A Randomized Clinical Trial	JAMA Internal Medicine	304	9.5	No	37
2018	Davis, L. L.,Kyriakides, T. C.,Suris, A. M.,Ottomanelli, L. A.,Mueller, L.,Parker, P. E.,Resnick, S. G.,Toscano, R.,Scrymgeour, A. A.,Drake, R. E.,Va Csp Veterans Individual Placement,Support Toward Advancing Recovery, Investigators	Effect of Evidence-Based Supported Employment vs Transitional Work on Achieving Steady Work Among Veterans With Posttraumatic Stress Disorder: A Randomized Clinical Trial	JAMA Psychiatry	541	18.3	No	38
2018	Najavits, L. M.,Krinsley, K.,Waring, M. E.,Gallagher, M. W.,Skidmore, C.	A Randomized Controlled Trial for Veterans with PTSD and Substance Use Disorder: Creating Change versus Seeking Safety	Substance Use & Misuse	52	26.9	No	39
2018	Raskind, M. A.,Peskind, E. R.,Chow, B.,Harris, C.,Davis-Karim, A.,Holmes, H. A.,Hart, K. L.,McFall, M.,Mellman, T. A.,Reist, C.,Romesser, J.,Rosenheck, R.,Shih, M. C.,Stein, M. B.,Swift, R.,Gleason, T.,Lu, Y.,Huang, G. D.	Trial of Prazosin for Post-Traumatic Stress Disorder in Military Veterans	New England Journal of Medicine	304	2.3	No	40
2008	Vernon, S. W.,del Junco, D. J.,Tiro, J. A.,Coan, S. P.,Perz, C. A.,Bastian, L. A.,Rakowski, W.,Chan, W.,Lairson, D. R.,McQueen, A.,Fernandez, M. E.,Warrick, C.,Halder, A.,DiClemente, C.	Promoting regular mammography screening II. Results from a randomized controlled trial in US women veterans	J Natl Cancer Inst	5500	100	NA	NA
2014	Katz, Lori S,Douglas, Sarah,Zaleski, Kristen,Williams, Jenny,Huffman, Cristi,Cojucar, Geta	Comparing holographic reprocessing and prolonged exposure for women veterans with sexual trauma: A pilot randomized trial	Journal of Contemporary Psychotherapy	51	100	NA	NA
2015	Morland, L. A.,Mackintosh, M. A.,Rosen, C. S.,Willis, E.,Resick, P.,Chard, K.,Frueh, B. C.	Telemedicine Versus in-Person Delivery of Cognitive Processing Therapy for Women with Posttraumatic Stress Disorder: A Randomized Noninferiority Trial	Depression & Anxiety	126	100	NA	NA
2016	Castillo, D. T.,Chee, C. L.,Nason, E.,Keller, J.,C’De Baca, J.,Qualls, C.,Fallon, S. K.,Haaland, K. Y.,Miller, M. W.,Keane, T. M.	Group-delivered cognitive/exposure therapy for PTSD in women veterans: A randomized controlled trial	Psychological Trauma:Theory, Pesearch, Practice and Policy	86	100	NA	NA
2018	Najavits, L. M.,Enggasser, J.,Brief, D.,Federman, E.	A randomized controlled trial of a gender-focused addiction model versus 12-step facilitation for women veterans	American Journal on Addictions	66	100	NA	NA