Analysis and Reporting of Sex Differences in Phase III Medical Device Clinical Trials—How Are We Doing?

Abstract

Over the past decade, the scientific community has begun to recognize the importance of biological sex differences in disease pathology, diagnosis, prevention, and treatment; however, the practice of sex-specific analysis and reporting is not integrated as standard practice by either our federal health agencies or by major medical journals. Despite the reforms of 20 years ago and the general inclusion of women in drug clinical trials, we have yet to see data routinely analyzed and reported by sex. Major journals are not requiring it, and large, publicly available datasets, such as ClinicalTrials.gov, are not systematically collecting and pointing to it. However, federal health databases and medical journals have the potential to impact progress in sex-specific analysis and reporting. We conducted a search on ClinicalTrials.gov for phase III device clinical trials and assessed their practice of sex differences evaluation. Reporting of clinical trial results by sex will maximize scientific value of research investments, significantly impact clinical decision-making, and transform medical care.

Until the enactment by Congress of the National Institutes of Health (NIH) Revitalization Act of 1993,¹ women were not required to be included in clinical research. Scientists believed that women were biologically equivalent to men, except for their reproductive capabilities. Women's reproductive cycles, it was also argued, added to the cost and time of the studies, deterring researchers from including them. Further, the U.S. Food and Drug Administration (FDA) had issued a prohibition in 1977 excluding women in their reproductive years from clinical research.² In 1993, the Society for Women's Health Research (SWHR) successfully advocated for Congress to mandate that women and minorities be included in all clinical research involving human subjects funded by the NIH.¹ This policy particularly emphasized and aimed to promote the inclusion of women and minorities in phase III clinical trials to allow the evaluation of sex differences in treatment outcomes; however, phase III clinical trials are only a small percentage of the NIH research portfolio.³ In response, the FDA issued a formal guidance that same year repealing its 1977 prohibition and outlining its expectations for drug, biologic, and medical device developers to include both sexes and assess potential sex differences in their research studies.⁴ It specifically directed study sponsors to ensure “a representative sample of the entire population likely to receive the drug has been studied, including representatives of both genders” and stated that “analyses to detect the influence of gender should be carried out both for individual studies and in the overall integrated analyses of effectiveness and safety,” emphasizing the evaluation of potential sex differences in pharmacokinetics.⁴ Then in 1998, FDA issued a regulation requiring gender-based reporting of safety and effectiveness data in drug applications.⁵

The outcome of these legislative and regulatory changes was a vast increase in the body of knowledge of biological sex differences in disease pathology, diagnosis, prevention, and treatment. Even the Institute of Medicine (IOM) in its 2001 report, “Exploring the Biological Contributions to Human Health: Does Sex Matter?”⁶ concluded that the answer was a resounding yes. Despite this, the practice of sex-specific analysis and reporting is not required to be fully integrated into standard practice by either our federal agencies or by major medical journals; therefore, it does not usually occur.

Since 1993, several government reports have been issued by the Government Accountability Office (GAO) and IOM examining the issue of women's health research and sex-specific reporting. Each emphasized the need for our federal agencies to do a better job of enforcing their own regulations and guidelines with respect to reporting and analysis by sex, race, and ethnicity.^7–10 Even as recently as December 2011, FDA issued a draft guidance entitled “Evaluation of Sex Differences in Medical Device Clinical Studies,” which included nonbinding recommendations regarding adequate enrollment of women in trials and reporting and analysis of sex differences.¹¹ The FDA acknowledged in its guidance a concern that there is a continued lack of adequate representation of women and sex differences analysis in medical device clinical trials.

Intrigued by the attention to sex differences analyses by the FDA, we wondered if there was a systematic way to assess progress in sex-specific analysis and reporting. In seeking a database that could be queried to measure this, we decided to examine ClinicalTrials.gov, our federal database of all drug, biologic, and device clinical trials that are conducted in the United States and trials that are intending to seek approval to market its product in the United States. Although we recognize that there are limitations to using ClinicalTrials.gov, we chose it because it is one of the fundamental resources that members of the research, health care, and patient community are directed to use when seeking information about clinical trials and their study results.

We conducted a review of all phase III device clinical trials in the United States with adult and senior subjects that were registered and reported results on ClinicalTrials.gov. The search yielded 87 studies sponsored by industry, federal agencies, universities, and other private organizations. An overwhelming majority (97%) of studies reported the sex of their participants, and the average representation of women in device clinical trials was 50%, although it ranged from 18% to 100%. However, when average rates of female participation in the trials were examined by target disease category, the rate was notably lower (34%) in the area of cardiovascular disease than in all other categories. In a similar review of cardiovascular devices that obtained premarket approval from the FDA between 2000 and 2007, women also made up less than one third of the subject population.¹² Strikingly, a sizeable amount of the studies found in that review did not even indicate the sex of their participants.¹² This reveals not only a continued pattern of underrepresentation of women in cardiovascular device trials (as was noted by FDA in its 2011 guidance), but also FDA's gaps in enforcing its own directive regarding the reporting of data on participants' sex.

In addition, our review showed that very few studies reported or analyzed outcomes or adverse events by sex. Three of the 87 studies reported outcomes by sex and two reported adverse events by sex. None of the studies presented such data on ClinicalTrials.gov; rather, they were reported in journal articles that were published from the research.

It is noteworthy that while the primary purpose of ClinicalTrials.gov is to provide patients, health care providers, researchers, and the public with easy access to clinical trial information, registered trials are in fact not required to report sex-specific analyses on its site, and such information is not easily accessible even if it is available in articles that have been published from the research. This is presumably the result of two oversights: (1) ClinicalTrials.gov makes citations to publications optional in its reporting system and (2) PubMed does not report the ClinicalTrials.gov identifier for studies using data from the clinical trials unless the sole purpose of the publication was to report the trial results. These prevent access to critical sex-specific information; however, they can be easily changed with slight modifications to the reporting policies of these major databases.

Lack of sex differences reporting needs to be addressed. Even if sex differences do not exist based on the analyses, researchers should nonetheless report the absence of sex differences because that is a critical piece of information with scientific value. While SWHR only examined device clinical trials and not drugs and biologics following the focus of FDA's 2011 draft guidance, these findings may represent persisting challenges in implementing federal guidelines and regulations in the broader clinical research enterprise.

In reflecting on key milestones in sex differences research, mandatory inclusion of women in all federally funded clinical trials represented an essential step towards gaining a better understanding of how sex-based differences might affect the way men and women respond differently to drugs, biologics, and medical devices. Indeed, great strides have been made toward reducing sex discrimination in clinical trials since the 1990s. Its success was clearly demonstrated by our assessment of medical device clinical trials, in which nearly all of the studies that registered and reported results to ClinicalTrials.gov included women as research subjects. However, even after 20 years and despite the growing recognition of sex as a clinically relevant variable in medical research, the practice of conducting sex-specific analyses and reporting sex differences remains sparse in medical device clinical trials. In fact, most if not a vast majority of all studies, from basic science and animal research to early phases of human subject research lack the inclusion of female lab cells and animals,¹³ let alone women. As a result, they do not address potential sex differences in dosing, safety, and effectiveness, some of which could have been detected in earlier stages of research had females been included and sex differences examined from the start.

FDA's efforts to clarify its expectations and promote adherence to its recommendations through guidance documents are useful and commendable but have had minimal success in achieving the end goal of improving sex differences evaluation. Ultimately, in an environment in which best practices are encouraged but not mandated and well enforced, researchers have lagged in fully utilizing research data to examine sex differences that might exist in research outcomes and the incidence of adverse events.

FDA issued a draft guidance on sex differences evaluation;¹¹ the International Committee of Medical Journal Editors established Uniform Requirements for Manuscripts Submitted to Biomedical Journals, one of which stipulated that “where scientifically appropriate, analyses by such variables as age and sex should be included”;¹⁴ and many high-impact journals began instituting policies regarding the reporting of results by sex.¹⁵ Yet, these actions have not sufficiently driven changes in the attitudes and behaviors of key stakeholders. In a 2012 IOM Workshop Summary “Sex-Specific Reporting of Scientific Research,” it was pointed out that if medical journals and research funders do not jointly implement change in their policies and follow through with enforcement, issues of feasibility will arise.¹⁰ For example, if journals require sex-specific analyses, but funders do not prioritize the issue and allow for the higher costs associated with developing a study large enough to power subgroup analyses, then researchers will still be unable to incorporate such analyses in their study design and draw meaningful conclusions from them.¹⁰ Thus, SWHR has urged and continues to urge journals to mandate sex differences analysis and reporting as requisites for publishing; FDA to require such reporting before granting approval to any device, drug, or biologic product; and NIH to stipulate this of their grant recipients, as well as be willing to accommodate for investigators' additional funding needs.

The continued lack of sex differences analysis and reporting poses a significant barrier to advancing our understanding of sex differences in the experience of diseases and treatments, as well as their underlying mechanisms. Clinical applications derived from knowledge of sex differences have a great potential of transforming medical care by enabling personalized medicine. In order to maximize the scientific value of research investments, our federal agencies, industry, investigators, medical journals, and associations of editors alike need to make a concerted effort to ensure that sex-specific analyses and reporting in clinical trials are being conducted. Only with the commitment from all sides will it be possible to achieve routine sex-specific analysis and reporting in biomedical research. The knowledge gained from this would in turn impact clinical decision-making and allow evidence-based medicine for both men and women, ultimately transforming biomedical research and the quality of medical care.