Gendered Innovations in Orthopaedic Science: Sex, Lies, and Stereotype: In Praise of the Systematic Review

Sir William Osler is credited with the phrase, “The greater the ignorance, the greater the dogmatism” [1]. The role of clinician scientists in orthopaedics is to fill in the gaps—decrease the ignorance—in order to replace dogmatism with evidence-driven practices. Researchers dedicated to the study of medical ignorance emphasize that scientific facts, like dogma, are neither solid nor immutable, but should be vigorously challenged and revised by successive generations [14]. The systematic review represents a tool that can help us answer clear, important questions premised on currently available evidence. Research about sex and gender—the purview of this column—will only improve with thoughtfully performed systematic reviews.

At a basic level, differentiation between studies examining sex (biology), gender (society’s lens of sex), or both is critical, but often confused in the literature [15, 17]. Recent attention to examining sex as a variable in scientific research will help clarify and guide analysis from retrospective reviews to prospective analyses. A January 2014 editorial in Nature [9] indicated that a failure to account for sex differences and poor experimental design may contribute to poor reproducibility and generalizability in biomedical research. In June 2015, the National Institutes of Health (NIH) announced the inclusion of sex as a biological variable in grant applications, citing the Nature article in addition to other pivotal publications [19]. The NIH provided supplemental resources to identify accurate terms and variables for both sex and gender in submitting grant applications [20]. The important NIH 2015 language includes: “Accounting for sex as a biological variable begins with the development of research questions and study design. It also includes data collection and analysis of results, as well as reporting of findings. Consideration of sex may be critical to the interpretation, validation, and generalizability of research findings. Adequate consideration of both sexes in experiments and disaggregation of data by sex allows for sex-based comparisons and may inform clinical interventions. Appropriate analysis and transparent reporting of data by sex may therefore enhance the rigor and applicability of preclinical biomedical research” [19].

Scientific rigor, as this document indicates, requires developing meaningful a priori research questions and a study design that contributes to the body of knowledge as we know it. Here, I use “meaningful” to provide contributions that beget change—with statements, facts, and findings that are valid in terms of both logic and statistics in that they withstand the test of time. Recall Gertrude Stein’s observation on the topic: “A difference, to be a difference, must make a difference”. Inferring differences and similarities between populations and subgroups based on things like sex, gender, or ethnicity require carefully framed and thoughtfully tested questions. Whereas American pundit Mark Twain joked about the three types of lies—“lies, damn lies, and statistics” [25]—this scientific rigor challenges the manipulation of numbers to suit a weak argument.

How do we apply sex and gender in orthopaedic science to the new directives of the 2015 NIH announcement and future research proposals [19]? A recent analysis published in CORR ^® reported on the increase in sex-specific analysis in high impact orthopaedic journals between 2000 and 2010 [13]. Although an increasing number of studies examine the effects of sex or gender on observed findings, such analyses are performed in less than one-third of studies. We need better guidelines so researchers have a clearer understanding of when and how these analyses should be performed [16].

A recent Canadian collaborative published a series of preliminary briefing notes to identify checklists for performing systematic reviews related to sex and gender [11]. Importantly, the musculoskeletal section of the Cochrane Collaboration, a consortium promoting the best methodology for examining randomized controlled trials and systematic reviews, was one of the three participating section contributors for this document. This, along with other guideline documents [3, 8, 12], the 2015 NIH supportive documents [20], and the PRISMA checklist are excellent resources [18].

At the risk of studying and stating the obvious, when choosing meaningful research questions for systemic reviews and iterative studies, the stereotype is a good starting point. We use the concept of a stereotype as oversimplifying or typecasting—derived from the relief casting of a printing plate used in typography—to identify a person, an attribute, or habit. Take, for example, the study of osteoporosis. How osteoporosis has changed over time, particularly with regards to the influence of sex on the observed findings, is especially instructive. Preliminary studies characterized the common problem of osteoporosis in postmenopausal women—a logical stereotype. Early review articles about men found only limited, fragmentary information [21]. This invited better hypothesis-driven questions such as, “How do we measure normal bone density in men?” Dedicated researchers launched the Osteoporotic Fractures in Men (MrOS) Study in 2000, a prospective cohort analysis [10]. Many research questions have been examined and refined in examining this cohort, providing not only fracture information associated with low bone density and low muscle mass [5], but also information about physical performance associated with higher fracture risk [23]. Applying similar scrutiny to yet a different cohort (children) yields other opportunities for understanding bone health across the lifespan. A recent systematic review (rather unsurprisingly) found that exercise in children can help prevent osteoporosis [24]. Although no difference has yet been discerned between boys and girls, even a review that offers somewhat intuitive conclusions can help us ask better questions and design better studies. Does duration of exercise, the type of sport, or the number of years of sport participation affect or influence adult bone density? A series of longitudinal studies examining such variables, including demographics of the population related to sex, ethnicity, and gender-specific activities would reveal such answers. A daunting and expensive endeavor no doubt, but one that cannot begin until we find the right questions to ask. Systematic reviews help point to the gaps that need to be filled.

Good systematic reviews provide the opportunity to create guidelines, checklists, and working groups, with a common goal in mind. Designing meaningful research questions in specific areas of interest includes creating consortia of “stakeholders.” These members typically represent interested and identified leaders in the field, health policymakers, support from industry, and others (including patients) who have something to gain from improvements and progress in the field. Like the MrOS study, the Cochrane Collaboration is a consortium of stakeholders who promote the best methodology for examining randomized controlled trials and systematic reviews to inform and facilitate healthcare decisions. Honoring the colorful epidemiologist Archie Cochrane who pointedly challenged the efficiency and efficacy of the British healthcare system [6], they are comprised of a global independent network of more than 37,000 volunteers of “patients and other healthcare consumers, health practitioners, policymakers, guidelines developers and research funders” [7].

Consortia who design the systematic review process, and develop trials and longitudinal studies often operate using consensus methodology. Decisions are made and solutions proffered through a combination of the iterative Delphi process known as “group response” enumerated by the RAND Corporation [22] and the nominal group technique [4]. The consensus process is messy, inefficient, and, given the voluntary status of busy participants, the process can take years [12]. The study designs, however, are refined, vetted, and validated. The end goal emanates from robust systematic reviews.

There is so much to do. Reexamining common and befuddling orthopaedic problems, disorders, and disease with the lens of sex and gender analysis through robust systematic reviews will yield robust prospective investigation. The NIH guidelines serve as a catalyst. Like all of science, the trajectory of discovery is filled with dead ends and heartbreaks, but also wonder and breakthroughs. To quote the great neuroscientist and histologist Santiago Ramon y Cajal, “Everything … discovered in a given domain is almost nothing in comparison with what is left to be discovered” [2].