Socioeconomic Status and Race Are Rarely Reported in Randomized Controlled Trials for Achilles Tendon Pathology in the Top 10 Orthopaedic Journals: A Systematic Review

Abstract

Background:

Randomized controlled trials (RCTs) are crucial in comparative research, and a careful approach to randomization methodology helps minimize bias. However, confounding variables like socioeconomic status (SES) and race are often underreported in orthopaedic RCTs, potentially affecting the generalizability of results. This study aimed to analyze the reporting trends of SES and race in RCTs pertaining to Achilles tendon pathology, considering 4 decades of data from top-tier orthopaedic journals.

Methods:

This review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and used PubMed to search 10 high–impact factor orthopaedic journals for RCTs related to the management of Achilles tendon pathology. The search encompassed all articles from the inception of each journal until July 11, 2023. Data extraction included year of publication, study type, reporting of SES and race, primary study location, and intervention details.

Results:

Of the 88 RCTs identified, 68 met the inclusion criteria. Based on decade of publication, 6 articles (8.8%) reported on SES, whereas only 2 articles (2.9%) reported on race. No RCTs reported SES in the pre-1999 period, but the frequency of reporting increased in subsequent decades. Meanwhile, all RCTs reporting race were published in the current decade (2020-2030), with a frequency of 20%. When considering the study location, RCTs conducted outside the United States were more likely to report SES compared with those within the USA.

Conclusion:

This review revealed a concerning underreporting of SES and race in Achilles tendon pathology RCTs. The reporting percentage remains low for both SES and race, indicating a need for comprehensive reporting practices in orthopaedic research. Understanding the impact of SES and race on treatment outcomes is critical for informed clinical decision making and ensuring equitable patient care. Future studies should prioritize the inclusion of these variables to enhance the generalizability and validity of RCT results.

Introduction

Randomized controlled trials (RCTs) have long been the gold standard for comparative research. ⁶⁵ RCTs help minimize bias by randomly assigning patient characteristics, theoretically allowing for the possibility to attribute all study outcomes to a specific factor. ³¹ Although conducting an RCT involves detailed procedures geared toward effective randomization of the cohort through methods such as simple, blocked, or stratified randomization, ⁵⁴ studies still fail to include all possible confounding variables when assigning groups. ⁶⁰ For example, socioeconomic status (SES) is rarely reported across all medical disciplines and may have a large effect on the generalizability of the results of a particular RCT.^27,60 Although certain fields, such as neuroscience, have consistently included SES metrics within the randomization process, ⁸⁹ orthopaedic surgery has consistently underreported SES.^29,40,56 Additionally, patient demographics, such as race, are rarely reported in orthopaedic RCTs.^83,101 Race and SES have both demonstrated strong effects on outcomes after traumatic events and various orthopaedic procedures, and could potentially be large confounding variables in any orthopaedic RCT.^75,106 As race and SES have shown a clear link in the quality of health care received, studying both metrics simultaneously was of great interest. ¹¹²

Across orthopaedics, there is evidence that SES and race can have an impact on outcomes of spine surgery, foot and ankle surgery, total joint arthroplasty, and follow-up metrics.^{29,40,44,45,64} However, the rate of reporting of these metrics is low in top orthopaedic journals. ²² Within foot and ankle orthopaedics, one previous systematic review organized SES and race reporting for RCTs encompassing all foot and ankle surgery. ⁶⁴ Although this review found evidence demonstrating consistent underreporting of SEC and race within foot and ankle surgery, it was limited to only a 6-year time frame (2016-2021) and only described studies within 4 specific journals. ⁶⁴ To follow up on this review, we extended the time frame to include RCTs from the last 4 decades, while also differentiating by the top 10 orthopaedic journals to better understand trends over time for each journal. Furthermore, the broad nature of this review prevented analysis of individual foot and ankle pathologies that may be more affected by SES and race. Therefore, our study only investigates SES and race reporting for Achilles tendon pathology—a condition well described in foot and ankle RCTs. Although this condition was purely chosen because of its high volume of RCTs and to address the possible confounding factors of previous reviews, ⁶⁴ studies have found that race has an impact on Achilles tendon rupture in elite African American gymnasts.^14,20 Although there is no sound evidence that SES and race are significantly related, one could argue that it is important to study SES and race within minority populations as a slight impact has been shown. Although no study has reported on SES directly for Achilles tendon pathology, SES is indirectly related to race and would likely have a significant impact as well. Therefore, the primary aim of this review was to evaluate SES and race reporting for foot and ankle surgery by evaluating Achilles tendon pathology RCTs over the last 4 decades to better understand reporting trends for these factors in major orthopaedic journals.

Methods

Study Creation

This systematic review was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) most recent guidelines. This study utilized PubMed to search 10 top-tier orthopaedic journals for randomized controlled trials (RCTs) related to management of Achilles tendon pathology. Journals were found by searching the Observatory of International Research (OOIR) website (https://ooir.org/journals.php?field=Clinical+Medicine&category=Orthopedics&metric=jif) for the top 5 general orthopaedic journals as well as the top 5 foot and ankle orthopaedic journals by impact factor. PubMed then was searched from database inception for each of the included journals until July 11, 2023. Search algorithm used in PubMed with the 10 top-tier orthopaedic journals with a filter of “Randomized Controlled Trials” was (“J Physiother”[journal]) OR (“Osteoarthritis Cartilage”[journal]) OR (“Am J Sports Med”[journal]) OR (“J Bone Joint Surg Am”[journal]) OR (“J Orthop Sports Phys Ther”[journal]) OR (“Foot Ankle Int.”[journal]) OR (“J Foot Ankle Res.”[journal]) OR (“Foot Ankle Surg”[journal]) OR (“Foot Ankle Clin.” [journal]) OR (“J Foot Ankle Surg”[journal]). All RCTs in the 10 top-tier orthopaedic articles by impact factor were initially retrieved for evaluation.

Inclusion and Exclusion Criteria

RCTs were included if the articles dealt primarily with Achilles tendon pathology, used human subjects, and were found in any of the 10 top-tier orthopaedic journals used for this study. Exclusion criteria were articles not primarily dealing with Achilles tendon pathology (eg, plantar fasciitis), non-English articles, non-full-text articles, and those that used nonhuman subjects (eg, cadavers or animals).

Study Definitions and Groups

For the purposes of this study, SES was broadly defined to give the most comprehensive reporting of SES in the high-level evidence in the literature. SES includes information on patient education levels, income levels, wealth, and occupation as well as any related terminology. RCTs were grouped by decade (1999 and prior, 2000-2009, 2010-2019, and 2020 and later), intervention (surgical, nonsurgical, or both), and primary study location (within the United States or outside of the United States).

Article Screening Process

After the initial search, articles were screened by abstract and title followed by full-text screening. Article screening was performed by multiple authors, with any discrepancies solved by a single author. Hand-screening of included articles’ references was also performed.

Data Extraction

Data extraction was performed by a single author. Data extracted included year of publication, type of study, reporting of SES, reporting of race, primary study location, and intervention as well as specifics about SES for further reporting.

Statistical Analysis

The Statistical Package for the Social Sciences (SPSS) version 29.0 (IBM Corp, Armonk, NY) was used for the statistical analysis of this study. As this study focused on qualitative data, only frequency reporting and descriptive statistics were needed.

Results

Initial Search Results

A total of 68 met inclusion criteria from a total of 88 RCTs found from the top-tier orthopaedic journals.^{1 -3,6 -8,10,12,13,16,17,19,23 -26,30,32 -34,36 -38,41 -43,46 -49,51,52,57,59,61 -63,66 -73,76 -78,80,81,84,85,87,88,90 -95,97,103 -105,107,110,113,114} Refer to Figure 1 for the PRISMA diagram for this systematic review. The RCTs in this study were taken from the top 10 orthopaedic journals by impact factor (Table 1).

Figure 1.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram used for this systematic review. This diagram outlines the initial search to final article inclusion.

Table 1.

Description of the 10 orthopaedic articles searched on PubMed for randomized controlled trials (RCTs) included in this systematic review. ^a

Impact Factor Ranking	Journal Name	Impact Factor	PubMed Abbreviation	PubMed Search Format
1	Journal of Physiotherapy	ca. 10.71	J Physiother	("J Physiother"[journal])
2	Osteoarthritis and Cartilage	ca. 7.51	Osteoarthritis Cartilage	("Osteoarthritis Cartilage"[journal])
3	American Journal of Sports Medicine	ca. 7.01	Am J Sports Med	("Am J Sports Med"[journal])
4	Journal of Bone and Joint Surgery-American Volume	ca. 6.56	J Bone Joint Surg Am	("J Bone Joint Surg Am"[journal])
5	Journal of Orthopaedic & Sports Physical Therapy	ca. 6.28	J Orthop Sports Phys Ther	("J Orthop Sports Phys Ther"[journal])
6	Foot & Ankle International	ca. 3.57	Foot Ankle Int	("Foot Ankle Int."[journal])
7	Journal of Foot and Ankle Research	ca. 3.05	J Foot Ankle Res	("J Foot Ankle Res."[journal])
8	Foot and Ankle Surgery	ca. 2.84	Foot Ankle Surg	("Foot Ankle Surg"[journal])
9	Foot and Ankle Clinics	ca. 2.03	Foot Ankle Clin	("Foot Ankle Clin. "[journal])
10	Journal of Foot & Ankle Surgery	ca. 1.35	J Foot Ankle Surg	("J Foot Ankle Surg"[journal])

^aData recorded included rank by Impact Factor (IF), name of the journal, IF, PubMed abbreviation, and PubMed search term by abbreviation.

Reporting by Decade

Based on decade, there were 4 articles published in 1999 or prior, 25 articles published between 2000 and 2009, 29 articles published between 2010 and 2019, and 10 articles published in 2020 and later. Overall, there was a total of 6 articles (8.8%) that reported SES and 2 articles (2.9%) that reported race. Based on decade, no RCTs (0.0% of RCTs within the decade) reported on SES in 1999 or prior years, 2 RCTs (8.0% of RCTs within the decade) reported on SES in 2000-2009, 3 RCTs (10.3% of RCTs within the decade) reported on SES in 2010-2019, and only 1 of the RCTs (10.0% of RCTs within the decade) reported on SES in 2020 or later years. The only 2 RCTs to report on race were in 2020 or later years as no RCTs reported on race prior to 2020. Refer to Figure 2 for a visual summary of reporting by decade.

Figure 2.

Reporting by decade for socioeconomic status (SES) and race. Decades include 1999 and prior, 2000-2009, 2010-2019, and 2020 and later. Abbreviations: RCTs, randomized controlled trials; SES, socioeconomic status.

Reporting by Surgical or Nonsurgical Intervention

From the 68 included RCTs, 45 RCTs (66.2% of RCTs) reported on nonsurgical treatment of Achilles tendon pathology, 11 RCTs (16.2% of RCTs) reported on surgical treatment of Achilles tendon pathology, and 12 RCTs (17.6% of RCTs) reported on both surgical and nonsurgical treatment of Achilles tendon pathology. Four of the RCTs (66.6% of RCTs reporting on SES) focused on nonsurgical interventions, 1 of the RCTs (16.7% of RCTs reporting on SES) focused on surgical interventions, and the remaining RCTs (16.7% of RCTs reporting on SES) focused on both surgical and nonsurgical interventions. One of the RCTs (50% of RCTs reporting on race) that reported on race focused on nonsurgical interventions whereas the other one of the RCTs (50% of RCTs reporting on race) that reported on race focused on surgical interventions. Refer to Figure 3 for a visual summary of reporting by type of intervention.

Figure 3.

Reporting by intervention for socioeconomic status (SES) and race. Interventions included nonsurgical, surgical, or both. Abbreviations: RCTs, randomized controlled trials; SES, socioeconomic status.

Reporting by Primary Study Location

From the 68 included RCTs, 6 RCTs (8.8% of RCTs) had a primary study location within the United States, 61 RCTs (89.7% of RCTs) had a primary study location outside of the United States, and one of the RCTs (1.5% of RCTs) had an unknown primary study location. Five of the 6 RCTs (83.3% of RCTs reporting on SES) that reported on SES had a primary study location outside of the United States, with 1 of the RCTs (16.7% of RCTs reporting SES) having an unknown primary study location. However, both RCTs (100% of RCTs reporting race) had a primary study location within the United States. Refer to Figure 4 for a visual summary of reporting by study location.

Figure 4.

Reporting by primary study location for socioeconomic status (SES) and race. Primary study location included within the United States (USA) or outside of the United States (Non-USA). Abbreviations: RCTs, randomized controlled trials; SES, socioeconomic status.

Specifics on Socioeconomic Reporting

Of the 6 RCTs that reported on SES, all 6 RCTs (100%) reported SES as occupation level in terms of difficulty: heavy, light, sedentary, or mobile. No RCT reported on SES in terms of patient education level, wealth level, or income level.

Discussion

Given criticism within foot and ankle RCTs for lack of appropriate confounding variable reporting, RCTs pertaining to Achilles tendon pathology were studied to understand the frequency and trends of SES and race reporting by decade and by orthopaedic journal. As race and SES have been shown to be directly related, ¹¹¹ this review was of high relevance. The aim of this review was to follow up on previous reviews by extending the time frame of interest while also including a broader scope of journals.

In previous reviews, such as the one by Martinazzi et al, ⁶⁴ SES was studied on RCTs published from 2016 through 2021. This review found that RCTs rarely report SES, and more important, found that the frequency of reporting did not increase over time, that studied journals had no difference in frequency of reporting, and no difference in reporting for various outcome studies. ⁶⁴ As this previous review focused on all RCTs encompassing foot and ankle surgery, we aimed to study a particular well-studied pathology in order to better understand reporting trends. Although studying all foot and ankle conditions may better illuminate the issue of not reporting SES within foot and ankle orthopaedics, SES reporting may have a greater impact on pathologies that frequently require surgical intervention, pathologies that require longer follow-up periods, or pathologies that require additional financial stability.^11,21,82 Thus, although the previous review found no difference in reporting for various outcome studies, it may be hard to compare pain management and operative procedures of different pathologies without introducing additional confounding factors. ¹⁰⁰ Therefore, the aim of this study was to study a particular pathology, such as Achilles tendon pathology. As this condition is widely published across the foot and ankle literature, RCTs focused on all forms of outcome studies (nonoperative, operative, or both) allowing for comparison within the same pathology. Furthermore, the studied time frame of the previous review was limited to 6 years. Although trends in reporting were assessed and found to be nonsignificant over the studied time frame, ⁶⁴ expanding the time frame may better demonstrate the trends in SES reporting. Finally, in the previous review, the authors only studied reporting by 4 orthopaedic journals. Including additional top journals was of high priority within this review, as excluding journals may not fully represent the true trends in SES reporting, or more importantly, the trends across different journals.

Although Achilles tendon pathology was studied because of its high prevalence in the literature, this condition was also chosen because of the possibility that SES may have a large impact on treatment and outcomes. In a previous review that studied tendinopathy, ³⁵ it was found that certain tendinopathic conditions led to patients taking additional sick leave to heal, decreased productivity, and most importantly, a decrease in function while performing everyday tasks such as driving, sleeping, carrying objects, and dressing.^{5,15,35,96,108,109} Therefore, including this condition was of great interest as foot and ankle tendinopathy was not addressed in detail in previous reviews. As SES has demonstrated impact on orthopaedic patient outcomes, clinically and subjectively, it is of great interest to understand SES reporting frequency in Achilles tendon pathology to better interpret the generality of an RCT’s results.^{18,39,50,53,55,58,74,79,86,98,99,102,115}

In this review, we found upward trends in SES reporting for Achilles tendon pathology. Comparing decades, RCTs included from 1990 through 2000 failed to report SES, whereas RCTs from 2000 to 2010 and 2010 to 2020 included SES in 2 and 3 RCTs, respectively. Although the overall percentage of reporting was low for both decades (8% for 2000-2010, 10.3% for 2010-2020), the frequency is increasing. However, small sample sizes were included from each decade, so direct statistical comparison was difficult. But on an absolute scale, the frequency of SES reporting is increasing. Interestingly, the rate of SES reporting for the included articles from 2020 to 2023 have roughly the same frequency as the previous decade (10.3% for 2010-2020, 10% for 2020-2023). Future investigation into SES reporting by decade would be necessary to understand the trends of the current decade (2020-2030). Without grouping by decade, comparison of RCT reporting of SES by location revealed interesting results. Although the majority of our included RCTs came from outside the USA, RCTs from non-US countries were the only included RCTs to report SES. Future investigation into SES on Achilles tendon pathology RCTs from the USA would allow for statistical comparison based on location.

Another crucial aspect of this review, beyond the reporting of SES, was the frequency of reporting race in RCTs. As many studies have shown that race has an impact on intraoperative complications, postoperative success, and even perceived pain and patient satisfaction.^9,53,86,98 Therefore, it was of great interest to include race as part of our analysis, particularly as race and SES are indirectly related. Our results demonstrate that until the current decade (2020-2030), race has not been reported for Achilles tendon pathology RCTs. However, in the current decade, the frequency of race reporting is higher than the frequency of SES reporting (10% for SES, 20% for race). Nevertheless, no statistical analysis was able to be performed to determine if this increase is significant compared to previous decades. This slight increase in reporting may be linked to the increase in race and SES political discussion in the last decade. Although reporting solely SES may encompass a variety of confounders, direct reporting of both race and SES is of paramount importance based on the impact race has on orthopaedic outcomes.^9,53,86,98 Furthermore, as the NIH has emphasized the requirement to include women and minorities in future research, SES and race reporting should increase as a result. However, as methods such as Area Deprivation Index can account for race when SES is controlled for, reporting solely SES may be sufficient for RCTs. It is important to establish the standard for reporting race and SES in research to understand where orthopaedic ranks with respect to other specialties. To no surprise, orthopaedics is not the only specialty underreporting these metrics, with RCTs pertaining to general and internal medicine, oncology, cardiac and cardiology, obstetrics and gynecology, and infectious disease all underreporting race and SES. ²⁸ Even in times where race and SES have been of great political discussion across the world, general medical journals consistently underreport these metrics regardless of specialty. ⁴

The primary limitation of our study is being unable to directly compare journals for SES and race reporting frequency. As only 8 of 68 RCTs included SES and/or race reporting, statistical comparison was not feasible. As more RCTs begin including SES and race, investigation into the frequency of reporting these factors by journal would be insightful in the future. Next, no study to date has examined why exactly race and SES are underreported in orthopaedic RCTs. Future research investigating this trend and its possible origin could be extremely impactful. As race and SES have both demonstrated strong effects on orthopaedic outcomes, their inclusion in RCTs would be completely justified.^75,106 Furthermore, race and SES have both shown negative impact on the level of involvement in health-related research. Although our review found consistent underreporting on race and SES, future research should also attempt to include those of all races and SES, at least proportionally to the area of study, to potentially limit the bias that comes from recruiting for medical research.

Conclusion

In conclusion, this review focused on the reporting of SES and race in RCTs pertaining to Achilles tendon pathology. The aim was to extend previous reviews by analyzing a broader scope of journals and a longer time frame. Findings revealed an upward trend in SES reporting over the decades studied, but no statistical analysis prevented conclusive statements from being made regarding the significance of this upward trend. Interestingly, RCTs from non-USA countries were the only ones to report SES, highlighting potential regional differences in reporting practices. Moreover, race reporting has been absent until the current decade, further demonstrating the need for more comprehensive reporting practices in orthopaedic research to better understand the impact of race and SES on treatment outcomes.