How Well Do Studies of Mesenchymal Stem Cell Intervention and Knee Osteoarthritis Adhere to the Minimum Information for Studies Evaluating Biologics Guidelines? A Systematic Review of Randomized Controlled Trials

Abstract

Background:

The Minimum Information for Studies Evaluating Biologics in Orthopaedics (MIBO) guidelines for mesenchymal stem cell (MSC) research, published in 2017, contain a suggested checklist for reporting items in manuscripts involving MSCs.

Purpose:

We sought to determine how well randomized controlled trials (RCTs) on MSC intervention for knee osteoarthritis (OA) adhered to the MIBO guidelines.

Methods:

A comprehensive literature search was performed in the PubMed/MEDLINE and Web of Science databases. Inclusion criteria included English-only RCTs that assessed MSC intervention for knee OA published between 2018 and 2022. Metrics were extracted, including year of publication, study design, first author name, journal name, patient demographics, and MIBO checklist criteria.

Results:

In 27 RCTs analyzed, 1006 patients were included, with a weighted male percentage of 41.8% and weighted mean age of 60.5 ± 7.2 years. On average, 70.5% (range, 30.2%–90.6%) of the modified 53-point MIBO checklist elements were reported per article. Seven (25.9%) articles had adherence rates of 80% or more, 13 (48.1%) had rates between 60% and 79.9%, and 7 (25.9%) had rates of 59.9% or less. The MIBO “intervention” category had the greatest adherence (100%), while the other categories had more variability. Six (50.0%) categories had an adherence level of 80% or more, 3 (25.0%) had adherence levels of 60% to 79.9%, and 3 (25.0%) had an adherence level of 59.9% or less.

Conclusion:

The overall mean adherence to MIBO guidelines of RCTs on MSC intervention for knee OA was 70.5%. Authors should better integrate the MIBO guidelines into their methodology to improve transparency, reproducibility, and reporting.

Introduction

Osteoarthritis (OA) is a common cause of knee pain, affecting nearly 1 in 4 individuals older than 40 years [3]. Many therapeutic modalities exist for this condition, such as anti-inflammatory medications and intra-articular steroid injections [14]. These interventions, however, are merely used for pain control. A new treatment option for OA has been proposed, which involves mesenchymal stem cells (MSCs). Stem cells have the potential to treat degenerative diseases, such as knee OA, but studies on their effectiveness, safety, and potential are limited [11]. The authors of a 2020 systematic review of 10 randomized controlled trials (RCTs) found that intra-articular injection of MSCs relieved pain and improved function in patients with knee OA, but results varied among the trials [8]. To date, the potential efficacy of MSC treatment remains unclear due in part to a lack of consistency and transparency in the methodology of MSC preparation and study.

In 2017, the American Academy of Orthopaedic Surgeons developed and published the Minimum Information for Studies Evaluating Biologics in Orthopaedics (MIBO) guidelines for platelet-rich plasma (PRP) and MSC research [4,9]. These guidelines contain a suggested checklist of reporting items for manuscripts involving PRP or MSCs, which can be used to improve the transparency, efficacy, reproducibility, and clinical evaluation of studies [9]. There are 12 categories comprising the 25-statement checklist for MSC research.

The purpose of this systematic review was to determine how well RCTs on MSC intervention for knee OA adhered to the MIBO guidelines after the guidelines were published. We hypothesized that adherence to MIBO guidelines would be low.

Methods

In December 2022, a comprehensive literature search of the PubMed/MEDLINE and Web of Science databases was performed by 2 authors following preferred reporting items for systematic reviews and meta-analyses, (PRISMA) guidelines. The keywords used in the search statements were (“mesenchymal stem cell” OR “stem cell” OR “mesenchymal stromal cell” OR “stromal cell”) AND (“osteoarthritis”). Inclusion criteria included English-only RCTs that assessed MSC intervention for knee OA from 2018 to 2022. Exclusion criteria (Fig. 1) included articles that were not in English, did not involve human subjects, non-RCTs, were not related to knee OA, or analyzed bone marrow aspirate injections. Reviews, protocol-only articles, and abstract-only articles were also excluded. Because MIBO guidelines were not published until May 2017, publications before January 1, 2018 were excluded, allowing authors enough time to recognize and incorporate the MIBO guidelines.

Fig. 1.

Preferred reporting items for systematic reviews and meta-analyses (PRISMA) flowchart. OA osteoarthritis.

All data were extracted and analyzed on the Microsoft Excel (Microsoft, Redmond, Washington, USA) software. Two authors extracted data from the articles that met inclusion criteria. Any disagreements by the 2 authors were resolved by consultation with a third (senior) author. Multiple metrics were extracted from each article, including year of publication, study design, first author name, journal of publication, patient demographics, and MIBO checklist criteria.

The original 25 MIBO checklist items were separated and modified into a 53-point checklist (Supplemental Table 1). Adherence was measured by calculating the percentage of checklist items each article adequately and clearly reported on the 53-point item list. Each of the article’s adherence percentages were averaged together for a final score. Good adherence was defined as 80% compliance or greater; partial, as 60% to 79.9%; and poor, as 59.9% or less.

Total percentage of adherence for each of the 12 MIBO categories was determined by assessing whether articles adequately and clearly reported on each of the subsections of the 12 categories in the modified 53-point item list. The average of all articles was used for the final category adherence percentage. For the “donors” category, studies that were assessing autologous MSC were recorded as adequate reporting. Weighted averages and standard deviations were used when appropriate to account for varying numbers of patients in each study. Only studies with complete data points for the values being measured were included in the statistical analysis.

This systematic review included only RCTs. The modified Coleman methodology score used to assess all studies utilizes a scoring system between 0 and 100, where 100 indicates a high-quality study without bias or other confounding factors [1]. Scores of 85 to 100 were considered excellent; 70 to 84, good; 55 to 69, fair; and 54 or less, poor [4].

Results

A total of 27 RCTs met inclusion criteria and were included in this systematic review. A total of 1006 patients were included, with a weighted male percentage of 41.8% ± 16.5%. The weighted mean age of all patients was 60.5 ± 7.2, and the body mass index (BMI) was 27.2 ± 1.7 (Supplemental Table 2). The average modified Coleman score was 72.9 ± 5.3 (range, 63–87), which indicates good overall quality.

On average, 70.5% ± 15.6% (range, 30.2%–90.6%) of the 53-point MIBO checklist was reported per article. Seven (25.9%) articles had adherence rates of 80% or more, 13 (48.1%) had rates between 60% and 79.9%, and 7 (25.9%) had rates of 59.9% or less (Fig. 2). The lowest mean adherence percentage (59.9%) was in 2021, and the highest (85.8%) was in 2022 (Fig. 3).

Fig. 2.

Adherence heatmap of the 53 MIBO checklist items. Green indicates adequate and clear reporting of an item, and red indicates unreported variables. MIBO Minimum Information for Studies Evaluating Biologics, CONSORT Consolidated Standards of Reporting Trials, MSC mesenchymal stem cell.

Fig. 3.

Breakdown of adherence to required items for each of the 12 categories is shown in MIBO adherence by year. MIBO Minimum Information for Studies Evaluating Biologics.

Adherence to required items for each of the 12 categories is shown in Fig. 4. Categories fluctuated in adherence, with intervention having the greatest adherence (100%) and cell culture having the lowest adherence (46.9%). Six (50.0%) categories had an adherence level of 80% or more, 3 (25.0%) had adherence levels of 60% to 79.9%, and 3 (25.0%) had an adherence level of 59.9% or less.

Fig. 4.

Percentage of adherence by MIBO category. MIBO Minimum Information for Studies Evaluating Biologics, MSC mesenchymal stem cell.

Discussion

On average, RCTs assessing the intervention of MSCs for knee OA partially (70.5%) adhered to the MIBO guidelines. There were 7 (25.9%) articles that adhered to 80% or more of the checklist statements, and there were 6 (50.0%) categories in which guidelines were met on average 80% or more of the time. There was high variability in compliance rates between 2018 and 2022, and definitive conclusions could not be made due to the small sample sizes per year. Based upon our analysis, reporting adherence improved in 2022.

DeClercq et al [4] published a similar systematic review assessing MIBO guideline compliance for PRP intervention following rotator cuff tears. An overall compliance rate of 58.5% was found among 19 studies published between 2008 and 2019. The 2 categories that were inconsistently reported were whole-blood and PRP processing and post-processing characteristics [4].

While we focused our search on MSCs instead of PRP, the MIBO guidelines for both MSCs and PRP were originally published in the same article by Murray et al [9] in May 2017. Our study builds upon this literature. Our goal was to determine whether authors of MSC studies are complying with MIBO guidelines. The compliance rate in our review was higher than that reported by DeClercq et al [4] (70.5% versus 58.5%, respectively), but this was likely due to the methodology used. First, we excluded all articles prior to 2018 to allow investigators time to recognize and follow the recommended MIBO guidelines. We also excluded all non-RCTs while DeClercq et al [4] did not. Our reasoning for only including RCTs was because they afford the highest level of evidence available [12].

This study is not without limitations. First, we modified the original MIBO 25-point item list into a 53-point list because many of the original MIBO guidelines grouped multiple items together. One such example is item 22, which reads, “MSC delivery described sufficiently to enable replication (including point of delivery, volume of suspension, and media used as vehicle.” Similar to the methodology of DeClercq et al [4], we broke down this category into individual items to better determine adherence. Second, even with an expanded checklist, determining whether an article adequately and clearly reported on an item remains subjective. We attempted to account for this by having 2 reviewers assess each article, with any disagreements resulting in discussions until a final decision was made. Third, although the articles included in this systematic review were published after the MIBO guidelines were released, data collection may have started before this release. Similarly, some articles may have been submitted to journals prior to the MIBO guideline’s release, and they were subsequently published after 2018.

While conclusions can be drawn from retrospective or even prospective cohort studies, RCTs are considered the gold standard for effective research [6]. Authors of RCTs should make a special effort to follow MIBO guidelines to improve both the transparency and reproducibility of their studies. Similar to PRISMA guidelines for systematic reviews and meta-analysis, it would be of benefit for journals to require adherence to MIBO guidelines for studies using either PRP or MSCs. However, similar to DeClercq et al [4], we recognize that the MIBO guidelines checklists will evolve as research and development advances. For example, DeClercq et al [4] recommended the addition of a BMI item into the checklists. In the context of knee OA, BMI has been linked as an important risk factor and predictor of knee OA treatment efficacy [15].

Smoking status was regularly missing in the patient demographics category; only 8 (29.6%) of the 27 articles reported on smoking status. Unlike BMI, which is a well-known predictor of knee OA and outcomes [10,13,15], the available literature on the relationship between smoking status and knee OA is controversial. For example, in a retrospective cross-sectional study of the National Health and Nutrition Examination Survey, Charen et al [2] found that there was an association between knee pain and smoking status (odds ratio = 1.15); however, Johnsen et al [7] and Dube et al [5] did not find an association.

There was poor compliance in 3 of the 12 MIBO categories: processing, cell culture, and post-treatment care. Processing and cell culture categories are important for future researchers to replicate the studies. Digestion solution concentration, volume, processing, and temperature are all required to adequately reproduce MSCs preparations; however, this information was often missing. Poor compliance in post-treatment care will make it difficult to know how patients should continue rehabilitation and is critical for achieving good outcomes.

The heterogeneity in reporting present in the included articles of this systematic review made it difficult to assess the quality and reproducibility of data; thus, the conclusions about therapy stated in the articles may not be highly reliable. With RCTs being the gold standard of medical decision-making, it is especially important for these studies to be as standardized as possible. Of note, none of the articles included in this systematic review had a 100% compliance rate. Similar to the conclusions reported by DeClercq et al [4], if journals were to adhere strictly to the MIBO guidelines, none of the studies would be published. Overall, the current MIBO guidelines were developed by expert consensus to provide the orthopedic community with a framework for how to best report studies involving PRPs or MSCs [4]. Adherence to the guidelines can improve study design, reproducibility, and reporting transparency in MSC intervention for knee OA.

In conclusion, the overall mean adherence to MIBO guidelines of RCTs on MSC intervention for knee OA was 70.5%. The 3 categories with the lowest reporting percentages were processing, cell culture, and post-treatment care. With RCTs being the highest level of evidence, it is important that authors better integrate the MIBO guidelines into their methodology to improve transparency, reproducibility, and reporting.