Sex-specific Analysis of Data in High-impact Orthopaedic Journals: How Are We Doing?

Abstract

Background

In 2001, the Institute of Medicine released a report stating that sex must be considered in all aspects and at all levels of biomedical research. Knowledge of differences between males and females in responses to treatment serves to improve our ability to care for our patients.

Questions/purposes

The purpose of our study was to determine (1) if there is an increase in the proportion of sex-specific reporting from 2000 to 2005 and to 2010; and (2) whether there is a proportional difference in such reporting based on journal type: subspecialty versus general orthopaedics. We hypothesize that assessment of the role of sex in outcomes has improved during the past 15 years and that the proportion of studies with of sex-specific analyses has increased with awareness of the role of sex in clinical outcomes and disease states. We additionally hypothesized that the reporting of sex would be similar between subspecialty and general orthopaedic journals.

Methods

Five high-impact orthopaedic journals, consisting of two general and three subspecialty journals, were chosen for review. Issues from even-numbered months during three calendar years (2000, 2005, 2010) were critically assessed for the presence of sex-specific analyses and reporting by two separate reviewers. Retrospective and prospective clinical studies, with a minimum of 20 patients, were included for analysis. Cadaveric, biomechanical, and in vitro studies were excluded. Review articles and clinical studies with less than 20 patients were excluded. A total of 821 studies that met inclusion criteria were analyzed: 206 in 2000, 277 in 2005, and 338 in 2010.

Results

Overall, the proportion of sex-specific analyses increased during the three times studied (19%, 40/206, [95% CI, 0.14–0.25] of the studies in 2000; 27%, 77/277, [95% CI, 0.23–0.33] in 2005; and 30%, 102/338, [95% CI, 0.25–0.35] in 2010). The increase in the proportion of sex-specific analysis was significant between 2000 and 2005 (p = 0.033), but was not significant between 2005 and 2010 (p = 0.518). During each of the three specific years studied, general and subspecialty journals increased in the proportions that reported sex-based analyses, but specialty journals had significantly higher reporting rates only in 2000 (2000: 11.9%, 13/109, [95% CI, 0.06–0.18] and 27.8%, 27/97, [95% CI, 0.19–0.37], p = 0.004; 2005: 22.9%, 33/144, [95% CI, 0.16–0.30], and 33.1%, 44/133, [95% CI, 0.25–0.41], p = 0.059; 2010: 28.2%, 51/181, [95% CI, 0.22–0.35] and 32.5%, 51/157, [95% CI, 0.25–0.40], p = 0.390).

Conclusions

Our findings indicate that inclusion of sex-specific analysis and reporting in the orthopaedic literature improved during our study period, but are present in less than 1/3 of the studies. Although subgroup analysis and reporting are required by NIH guidelines, it is important that such analyses be published in non-NIH-funded studies to generate hypotheses regarding sex differences for subsequent research. These data also are important as they can be used in systematic reviews where large independent studies may not be available in the literature.

Clinical Relevance

Where evaluating conditions that affect males and females, studies should be designed with sufficient sample size to allow for subgroup analysis by sex to be performed, and they should include sex-specific differences among the a priori research questions.

Introduction

Significant differences in the prevalence of medical and orthopaedic conditions and treatment outcomes between males and females have been identified and reported (Table 1) [2, 3, 6–8, 10, 13, 16, 19, 20, 22, 24]. Recognizing the importance of research on sexual dimorphism, the NIH instituted a policy in 1986 urging the inclusion of women in clinical trials. This policy became law in 1994 with the NIH Revitalization Act, which requires that NIH-supported clinical research include women as subjects [21]. Soon after the NIH Revitalization Act took effect, the Agency for Health Research and Quality and the CDC developed similar guidelines [1, 5]. In 2001, the Institute of Medicine released a report stating that sex must be considered in all aspects and at all levels of biomedical research [14].

Table 1.

Known musculoskeletal differences between males and females

Musculoskeletal parameter	Sex differences
ACL tear	Female athletes are up to six times more likely to rupture their ACL compared with their male counterparts participating in the same sport [7].
Cartilage	Men have substantially larger knee cartilage volume than women, even after adjusting for age, height, weight, and bone volume [6].
Muscle	Men have more muscle mass than women, and hence have greater losses in total muscle mass; however, women experience earlier strength losses and have greater declines in muscle quality. As a result, women are more likely to have health problems from this owing to longer lifespan [8].
Musculoskeletal pain	Musculoskeletal pain is more prevalent in older women than older men, and men and women differ in the factors associated with musculoskeletal pain [19].
Osteoarthritis	Worldwide estimates are that 9.6% of men and 18.0% of women 60 years old or older have symptomatic osteoarthritis [2, 20]. Before 50 years of age, the prevalence of osteoarthritis in most joints is greater in men than women, but after this age, more women than men are affected by osteoarthritis of the hand, foot, and knee [3, 13, 24].
Osteoporosis	Osteoporosis, which is characterized by loss of bone mass leading to increased risk of fracture, affects eight million women and two million men [22].
Osteoporotic fractures	One in two women and one in four men older than 55 years will have an osteoporosis-related fracture [10, 22]. The rate of hip fracture is two to three times higher in women than men; however, after a hip fracture, men are nearly twice as likely as women to die [10, 22]. Women had a 23% greater mortality rate than men after one or more vertebral fractures [16].

With the continued emphasis on including women in clinical trials, and reporting outcomes by sex [18], we were interested in evaluating whether clinical research published in the orthopaedic literature included sex-specific data stratification. Specifically, we aimed to determine (1) if there is an increase in the proportion of sex-specific reporting from 2000 to 2005 and to 2010; and (2) whether there is a proportional difference in such reporting based on journal type: subspecialty versus general orthopaedics.

Materials and Methods

Five of the highest-impact orthopaedic journals were chosen for review. Two general orthopaedic journals were chosen (Journal of Bone and Joint Surgery American Volume, and Clinical Orthopaedics and Related Research^®) along with three subspecialty journals (Journal of Shoulder and Elbow Surgery, Spine, and American Journal of Sports Medicine). Issues published during even-numbered months of 2000, 2005, and 2010 were critically assessed for the presence of sex-specific analyses; a total of 126 journal issues were reviewed. Two independent reviewers (SH and LL) performed the assessments. Articles were screened first via the abstract for the inclusion of males and females in the study. If the article qualified, the Materials and Methods and Results were reviewed to determine if sex was included as a variable in the statistical analysis or simply as a demographic. To be marked as having sex-specific analysis an article had to include sex in a multifactorial statistical model. There was third-party adjudication for any discrepancies in interpretation of the analysis of sex by a senior author (CH).

Prospective and retrospective clinical studies with a minimum of 20 patients were included for analysis. Studies were excluded if they were nonhuman, cadaveric, biomechanical, in vitro, or review articles. Clinical studies with less than 20 patients were excluded from the analysis owing to lack of sufficient sample size to perform basic sex-specific analyses. Studies on teams or groups that did not include women also were excluded. In total, 206 articles met our inclusion criteria for 2000, 277 for 2005, and 338 for 2010.

To assess the proportion of sex-specific analyses reported in the three years, we generated a 2 × 3 table and performed chi-square analysis. We then performed subgroup analysis for each year, comparing general versus specialty on the proportion of sex-specific reporting. All analyses were performed by a statistician (YG).

Results

Overall, the proportion of sex-specific analyses increased during the three times studied (19%, 40/206, [95% CI, 0.14–0.25] of the studies in 2000; 27%, 77/277, [95% CI, 0.23–0.33] in 2005; and 30%, 102/338, [95% CI, 0.25–0.35] in 2010) (Fig. 1). The increase in the proportion of sex-specific analysis was significant between 2000 and 2005 (p = 0.033), but was not significant between 2005 and 2010 (p = 0.518).

Fig. 1.

The percentages of articles that included sex-specific analyses in all journals combined for 2000, 2005, and 2010 are shown.

During the three years studied, general and subspecialty journals increased in the proportions that reported sex-based analyses (Table 2). Specialty journals had significantly higher reporting rates only in 2000 (2000: 11.9%, 13/109, [95% CI, 0.06–0.18] and 27.8%, 27/97, [95% CI, 0.19, 0.37], p = 0.004; 2005: 22.9%, 33/144, [95% CI, 0.16–0.30], and 33.1%, 44/133, [95% CI, 0.25–0.41], p = 0.059; 2010: 28.2%, 51/181, [95% CI, 0.22–0.35] and 32.5%, 51/157, [95% CI, 0.25–0.40], p = 0.390) (Fig. 2).

Table 2.

Percentage of studies with sex-specific analyses by journal

Journal/year	Total	Women	Percent
2000
Journal of Bone & Joint Surgery, American Volume	42	2	4.76%
American Journal of Sports Medicine	40	13	32.50%
Clinical Orthopaedics and Related Research ®	67	11	16.42%
Spine	45	11	24.44%
Journal of Shoulder and Elbow Surgery	12	3	25.00%
Total	206	40	19.42%
2005
Journal of Bone & Joint Surgery, American Volume	77	22	28.57%
American Journal of Sports Medicine	41	13	31.71%
Clinical Orthopaedics and Related Research ®	67	11	16.42%
Spine	55	16	29.09%
Journal of Shoulder and Elbow Surgery	37	15	40.54%
Total	277	77	27.80%
2010
Journal of Bone & Joint Surgery, American Volume	64	25	39.06%
American Journal of Sports Medicine	64	22	34.38%
Clinical Orthopaedics and Related Research ®	117	26	22.22%
Spine	52	21	40.38%
Journal of Shoulder and Elbow Surgery	41	8	19.51%
Total	338	102	30.18%

Fig. 2.

The percentages of articles that included sex-specific analyses in general versus specialty journals in 2000, 2005, and 2010 are shown.

Discussion

In 2001, the National Academy of Sciences Institute of Medicine published its report entitled Exploring the Biological Contributions to Human Health: Does Sex Matter? [25]. Their findings supported the importance of sex as an important variable in design and analysis of all health research [25]. Furthermore, the report emphasized that “understanding the bases of these sex-based differences [is] important to developing new approaches to prevention, diagnosis, and treatment [25]. Similarly the NIH recognizes an ethical principal of justice and importance of balancing research burdens and benefits across the population. This focus is evident in the NIH Revitalization Act of 1993, PL 103-43, which resulted in guidelines for the inclusion of women and minorities in clinical research [21]. Geller et al. [11, 12] looked at such inclusion and found that in federally funded randomized controlled trials on average, 37% of the study participants were women, showing their underrepresentation [11, 12]. Seventy-five percent of these studies did not report any outcomes by sex [12]. Rates of inclusion and data stratification did not improve during the 5-year period reviewed [11, 12]. Moreover, they found no differences in the rates of sex-specific data stratification in reporting or analysis when comparing 25 nonfederally funded studies with the federally funded studies, which were bound by the guidelines of the NIH and other federal funding agencies [12].

Recently, Clinical Orthopaedics and Related Research^® published an editorial calling for increased inclusion, reporting, and analysis of sex and gender in all submitted studies [18]. The editorial emphasized the important differences between sex and gender and the implications in orthopaedics [18]. Because of this emphasis on sex-specific analysis and reporting, and because little is known regarding the current baseline of reporting in orthopaedics, we aimed to determine whether sex-specific reporting increased with time and if there were differences between general and subspecialty journals.

The most important limitation of our study is lack of knowledge of the percentage of studies that should include a sex-specific analysis, as this is not practical for all studies. For certain studies, and certain diseases, it may not be practical to obtain sufficient patient numbers to perform a multivariable analysis. Increasing numbers of patients to levels to adequately perform these analyses potentially would increase costs and make analyses more complex, requiring a statistician. Data were taken from 2000, 2005, and 2010, and although it would be more illustrative to include every year, we thought this was a representative sample, with years spaced enough to show changes with time.

We found an increase in sex-specific analyses with time, most significantly between 2000 and 2005 in general and subspecialty journals. In 2010, 70% of orthopaedic studies did not include a sex-specific analysis, similar to the 75% cited by Geller [12]. Similar concerns regarding inclusion and appropriate sex-specific result reporting are found across multiple medical specialties and are particularly prevalent in the cardiovascular literature [4, 15, 17, 25]. Circulation, a journal of the American Heart Association, has responded favorably by instructing authors to provide sex-specific data when appropriate in describing outcomes of epidemiologic analysis or clinical trials. Moreover, the journal requires authors to specifically state that no sex-based differences were present when none are found. Perhaps orthopaedic journals should institute similar policies requiring authors to clearly label single-sex studies and include a statement justifying the inclusion of only one sex in their research design or justifying why there was no sex-specific data stratification in their analysis. Researchers continue to acknowledge these differences and focus groups to better analyze them, such as the Gendered Innovations project that develops methods of sex analysis [23]. The American Academy of Orthopaedic Surgeons has emphasized the importance of sex-specific information for medical devices through the Health of Women program [9].

Inclusion of sex-specific analyses increased in the orthopaedic literature during the years we studied, however, 70% of the clinical studies included did not include a sex-specific analysis. Although it is not practical to include a sex-specific analysis in all studies, it is imperative for educators and researchers to promote and support data stratification by sex in clinical and basic science research. Understanding how males and females may present differently with similar diagnoses and/or respond differently to various treatments can translate into orthopaedic clinical practice and ultimately improve patient care and outcomes.