Reporting Bias is Highly Prevalent in Systematic Reviews and Meta-Analyses of Platelet Rich Plasma Injections for Hip Osteoarthritis

Abstract

Purpose

To describe the incidence and types of spin in systematic reviews of platelet-rich plasma (PRP) injections for hip osteoarthritis (OA) and to determine whether patterns in study characteristics could be identified among studies with identifiable spin.

Methods

The PubMed, Scopus, and SPORTDiscus databases were queried. Inclusion criteria were systematic reviews or meta-analyses that included an assessment of intra-articular PRP injections as a stand-alone treatment for hip OA. Two authors independently assessed the presence of spin in the included studies and recorded general study characteristics. The prevalence of the 15 different categories of spin was quantified using descriptive statistics.

Results

Fifteen studies met inclusion criteria for this study. All studies contained at least two types of spin (range 2-9), with a median of 2. The most common type of spin was type 14 (“Failure to report a wide confidence interval of estimates”), which was observed in 10 studies. The second most common type of spin was type 13 (“Failure to specify the direction of the effect when it favors the control intervention”), found in 6 studies.

Conclusions

Spin is highly prevalent in abstracts of systematic reviews of PRP in the treatment of hip OA. Several associations were found between spin types and the study characteristics of AMSTAR 2 rating, Scopus CiteScore, journal impact factor, and PROSPERO preregistration. When present, spin in the abstracts of reviewed studies tended to favor the use of PRP in hip osteoarthritis.

Clinical Relevance

It is important to understand the prevalence of spin in published abstracts, especially in areas of great impact or interest, so authors and readers can have a greater awareness of this potential form of bias.

Intra-articular injection with platelet-rich plasma (PRP) is an emerging non-operative treatment option for hip osteoarthritis (OA), which is estimated to cause symptomatic disease in one in four people by age 85 years.¹ While there have been multiple studies examining the potential of PRP to improve hip pain and function,2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 it is difficult to make conclusions about its efficacy with the available evidence for several important reasons. Although comparisons of placebo (saline solution) with PRP have been made indirectly via network meta-analysis,⁹ there are limitations to this type of study.¹⁷ Moreover, existing trials typically compare PRP with hyaluronic acid (HA), itself an emerging injectable.¹¹ Studies additionally suffer from bias and heterogeneity, varying in PRP preparation (volume, leukocyte-rich vs -poor, etc.), follow-up timeline, radiographic grade of osteoarthritis, and outcome measures.⁸ Although the use of PRP for OA is better studied in the knee, such findings cannot be extrapolated to the hip given specific differences in joint biomechanics and morphology that may affect which treatments may be appropriate.¹⁸

Systematic reviews and meta-analyses aggregate the limited number of trials that have been conducted and attempt to evaluate the current state of the evidence. However, the limitations in these trials predispose them, and the systematic reviews that collate their data, to bias, particularly spin, which has been described as the reporting of findings that emphasizes benefits or downplays negative effects despite a lack of evidence to support those conclusions.^19,20 Spin has been categorized as (1) misleading representation, (2) misleading reporting, and (3) inappropriate extrapolation, and commonly appears in study abstracts.²¹ Biases such as spin are often exaggerated in abstracts, where strict word count limits and availability biases encourage authors to emphasize the most meaningful findings while neglecting others. This practice is particularly problematic given that many physicians often rely heavily on abstracts for clinical decision-making.^22,23

The purposes of this study were to describe the incidence and types of spin in systematic reviews of PRP injections for hip OA and to determine whether patterns in study characteristics could be identified among studies with identifiable spin. We hypothesized that spin is widely present in the abstracts of systematic reviews concerning PRP injections for hip OA.

Methods

Eligibility

Inclusion criteria were systematic reviews or meta-analyses that included an assessment of intra-articular PRP injections as a stand-alone treatment for hip osteoarthritis (i.e., not given as an adjunct to arthroscopy). Included studies were those specifically investigating hip osteoarthritis or those that included other pathologies but had a subgroup analysis for PRP use in hip osteoarthritis. Databases were queried from inception to August 28, 2022, on which the searches were performed. Exclusion criteria were studies that utilized adjuvant therapy in addition to PRP, were not published in English, studies that were not peer reviewed, studies that were retracted or withdrawn, studies published without an abstract, and studies without full text available. This study was performed according to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines using a protocol developed a priori.²⁴ A single author (D.R.K.) searched for publications in the PubMed, Scopus, and SPORTDiscus databases using the keywords “[platelet rich plasma] AND [hip osteoarthritis] AND ([meta-analysis] OR [systematic review]).” Results were compiled and deduplicated in EndNote 20 (Clarivate, Philadelphia, PA). Two authors (D.R.K. and B.S.B.) independently screened the identified studies for inclusion. Disagreements were resolved via consensus or third-party guidance.

Training

Two authors (D.R.K. and B.B.) were trained on how to identify the 15 most common types of spin in the abstracts of included studies according to the method proposed by Yavchitz et al.,²¹ which are summarized in Table 1, and also to assess study quality using version 2 of A Measurement Tool to Assess Systematic Reviews (AMSTAR 2).²⁵ AMSTAR 2 is a comprehensive critical instrument for systematic reviews of randomized trials that appraises qualities such as a priori protocol development and adherence, rationale for inclusion criteria, duplicate study selection and data extraction, risk of bias assessment, follow-up and conflict of interest characterization, and best practices for meta-analysis.²⁵ AMSTAR 2 has been validated for assessment of systematic reviews with a moderate strength of interrater reliability.²⁶

Table 1.

Overview of included studies

Study	Author	Journal	Year	Level of Evidence	Study Type
The use of ultrasound-guided platelet-rich plasma injections in the treatment of hip osteoarthritis: a systematic review of the literature2	Ali et al.	Journal of Ultrasound	2018	I	Systematic review
Platelet-Rich Plasma Versus Hyaluronic Acid for Hip Osteoarthritis Yields Similarly Beneficial Short-Term Clinical Outcomes: A Systematic Review and Meta-analysis of Level I and II Randomized Controlled Trials3	Belk et al.	Arthroscopy	2022	II	Systematic review
Platelet-rich plasma injections for hip osteoarthritis: A review of the evidence4	Berney et al.	Irish Journal of Medical Science	2021	IV	Systematic review
Platelet-rich plasma in the management of articular cartilage pathology: A systematic review5	Dold et al.	Clinical Journal of Sports Medicine	2014	IV	Systematic review
The effects of platelet-rich plasma injection in knee and hip osteoarthritis: A meta-analysis of randomized controlled trials6	Dong et al.	Clinical Rheumatology	2021	I	Systematic review
State of art in intra-articular hip injections of different medications for osteoarthritis: A systematic review7	Ferrara et al.	BMC Musculoskeletal Disorders	2021	I	Systematic review
Preparation methods and clinical outcomes of platelet-rich plasma for intra-articular hip disorders: A systematic review and meta-analysis of randomized clinical trials8	Garcia et al.8	Orthopedic Journal of Sports Medicine	2020	II	Systematic review
Intra-articular saline injection is as effective as corticosteroids, platelet-rich plasma and hyaluronic acid for hip osteoarthritis pain: A systematic review and network meta-analysis of randomized controlled trials9	Gazendam et al.	British Journal of Sports Medicine	2021	I	Systematic review
PRP for degenerative cartilage disease: A systematic review of clinical studies10	Laver et al.	Cartilage	2017	IV	Systematic review
The use of intra-articular platelet-rich plasma as a therapeutic intervention for hip osteoarthritis: A systematic review and meta-analysis11	Lim et al.	American Journal of Sports Medicine	2022	IV	Systematic review
Effectiveness of platelet-rich plasma in the management of hip osteoarthritis: A systematic review and meta-analysis12	Medina-Porqueres et al.	Clinical Rheumatology	2021	I	Systematic review
Efficacy and safety of intra-articular therapies in rheumatic and musculoskeletal diseases: an overview of systematic reviews13	Rodriguez-García et al.	RMD Open	2021	I	Systematic review
The effects of platelet-rich plasma in the treatment of large-joint osteoarthritis: a systematic review14	Tietze et al.	The Physician and Sportsmedicine	2014	IV	Systematic review
Platelet rich plasma versus hyaluronic acid in patients with hip osteoarthritis: A meta-analysis of randomized controlled trials15	Ye et al.	International Journal of Surgery	2018	I	Systematic review
Different intra-articular injections as therapy for hip osteoarthritis: A systematic review and network meta-analysis16	Zhao et al.	Arthroscopy	2020	II	Systematic review

Data Extraction

Two authors (D.R.K. and B.B.) independently extracted characteristics of the included studies, including title, authors, journal, year of publication, funding source, adherence to PRISMA, PROSPERO preregistration of the study protocol (University of York, York, UK), Clarivate 2021 Journal Impact Factor (Clarivate Analytics, Philadelphia, PA), Scopus CiteScore 2021 (Elsevier, Amsterdam, the Netherlands), primary and secondary outcome measures, and level of evidence. Level of evidence, if not explicitly stated in the study, was determined according to Wright et al.²⁷ Abstracts were reviewed after reading the full texts to assess the presence of the 15 most common types of spin. Full texts were reviewed to assess study quality according to AMSTAR 2 criteria. Study quality was further categorized into high, moderate, low, and critically low according to a scheme suggested by the authors of AMSTAR 2.²⁵ All disagreements were resolved via consensus and reference to full text for clarification.

Statistical Analysis

The prevalence of the 15 different categories of spin was quantified using descriptive statistics. Given the small number of included studies, this analysis was underpowered for a multivariable logistic regression. Associations between categorical study characteristics and spin types were examined using Fisher’s exact test rather than the χ² test given the small sample size and possibility of unbalanced contingency tables. Statistical analysis was performed using the R programming language (version 4.2.1; R Foundation for Statistical Computing, Vienna, Austria). Statistical significance was set at P < .05.

Results

Database searches identified 44 studies of which 13 were duplicates and removed. Following title, abstract, and full-text screening, a further 16 studies were removed for not meeting inclusion and exclusion criteria. Ultimately, 15 studies in 13 individual journals remained for inclusion in this study. This screening process is illustrated in Figure 1. Basic study characteristics can be found in Table 1. Eight studies included a meta-analysis (8 of 15 [53%]). As for funding source, 4 studies did not disclose whether they received external funding (4 of 15 [27%]), 7 stated that they did not receive external funding (7 of 15 [47%]), and 4 disclosed that they received external funding (4 of 15 [27%]). Eleven studies reported adherence to PRISMA guidelines (11 of 15 [73%]). Only four studies (4 of 15 [27%]) stated that their protocols were preregistered with a public register such as PROSPERO. The average 2021 Clarivate Journal Impact Factor was 4.82 (range 0.23-18.49). The average 2021 Scopus CiteScore was 6.53 (range 1-21.3).

Fig 1.

Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram.

Frequency of Spin and Analysis

All studies contained at least 2 types of spin (range 2-9), with a median of 2. Frequency of each spin type can be found in Table 2. The most common type of spin was type 14 (“Failure to report a wide confidence interval of estimates,” 10 of 15 [67%]). Each type of spin was observed in at least one abstract except type 6 and type 7. The spin category that was identified with the highest frequency was misleading reporting, which was observed in all 15 studies (15 of 15 [100%]).

Table 2.

Frequency of Each Spin Category and Type in Reviewed Studies

Type	Spin Description	Abstracts w/ Spin	%
1	The conclusion formulates recommendations for clinical practice not supported by the findings	2/15	13.3%
2	The title claims or suggests a beneficial effect of the experimental intervention not supported by the findings	1/15	6.7%
3	Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention	5/15	33.3%
4	The conclusion claims safety based on nonstatistically significant results with a wide confidence interval	2/15	13.3%
5	The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies	5/15	33.3%
6	Selective reporting of or overemphasis on harm outcomes or analysis favoring the safety of the experimental intervention	0/15	0.0%
7	The conclusion extrapolates the review findings to a different intervention (e.g., claiming efficacy of one specific intervention although the review covered a class of several interventions).	0/15	0.0%
8	Conclusion extrapolates the review's findings from a surrogate marker or a specific outcome to the global improvement of the disease	1/15	6.7%
9	Conclusion claims the beneficial effect of the experimental treatment despite reporting bias	3/15	20.0%
10	Authors hide or do not present any conflict of interest	5/15	33.3%
11	Conclusion focuses selectively on statistically significant efficacy outcome	5/15	33.3%
12	Conclusion claims equivalence or comparable effectiveness for nonstatistically significant results with a wide confidence interval	3/15	20.0%
13	Failure to specify the direction of the effect when it favors the control intervention	6/15	40.0%
14	Failure to report a wide confidence interval of estimates	10/15	66.7%
15	Conclusion extrapolates the review's findings to a different population or setting	3/15	20.0%

AMSTAR 2 Rating

The authors of AMSTAR 2 have proposed a scheme for appraising the overall confidence in the results of systematic reviews into high, moderate, low, or critically low, based on weaknesses in critical and noncritical items.²⁵ According to this scheme, all but one study was found to have a rating of critically low (14 of 15 [93%]), the exception being the study by Rodriguez-Garcia et al.,¹³ which had a rating of low. Complete AMSTAR 2 assessments are detailed in Table 3.

Table 3.

AMSTAR 2 Assessment Of Reviewed Studies

Item	AMSTAR Adherence	AMSTAR Items in Full Text	%
1	Did the research questions and inclusion criteria for the review include the components of PICO?	12/15	80.0%
2	Did the report of the review contain an explicit statement that the review methods were established prior to conducting the review, and did the report justify any significant deviations from the protocol?	5/15	33.3%
3	Did the review authors explain their selection of the study designs for the inclusion in the review?	2/15	13.3%
4	Did the review authors use a comprehensive literature search strategy?	14/15	93.3%
5	Did the review authors perform study selection in duplicate?	12/15	80.0%
6	Did the review authors perform data extraction in duplicate?	7/15	46.7%
7	Did the review authors provide a list of excluded studies and justify the exclusions?	0/15	0.0%
8	Did the review authors describe the included studies in adequate detail?	15/15	100%
9	Did the review authors use a satisfactory technique for assessing the RoB in individual studies that were included in the review?	12/15	80.0%
10	Did the review authors report on the sources of funding for the studies included in the review?	1/15	6.7%
11	If meta-analysis was justified did the review authors use appropriate methods for statistical combination of results?	7/15	46.7%
12	If meta-analysis was performed did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis	5/15	33.3%
13	Did the review authors account for RoB in individual studies when interpreting/discussing the results of the review?	10/15	66.7%
14	Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review?	14/15	93.3%
15	If they performed quantitative synthesis did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?	2/15	13.3%
16	Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review.	5/15	33.3%

RoB, risk of bias.

Discussion

Our analysis of 15 systematic reviews investigating the use of PRP for hip OA identified the most common types of spin in their abstracts as described by Yavchitz et al.²¹ and found that all studies contained at least 2 types of spin. The most common type of spin was type 14 (“Failure to report a wide confidence interval of estimates”), which was observed in 10 studies. The second most common type of spin was type 13 (“Failure to specify the direction of the effect when it favors the control intervention”), found in 6 studies. The most common category of spin identified was misleading reporting, which was observed in all 15 studies. Although 73% of studies reported adhering to PRISMA guidelines, only 27% submitted their protocols to a public register such as PROSPERO, and 27% of studies did not disclose whether they received external funding.

Spin types 14 and 13 were the most identified types in our study. Regarding type 14 (“Failure to report a wide confidence interval of estimates”), two thirds of the included systematic reviews did not report wide confidence intervals in the abstract. Although the decision to include confidence intervals specifically in the abstract may be influenced by the style of a given journal, we found that leaving out these values can predispose studies to misinterpretation. For example, Garcia et al.⁸ performed a meta-analysis on 7 randomized controlled trials and made 2 claims in the abstract: (1) treatment of hip OA with PRP demonstrated reductions in pain and improved patient-reported outcomes for up to 1 year, and (2) that there was no statistically significant difference between PRP and HA in pain reduction. No confidence intervals are provided for either claim. For example, the abstract reads “pooled effect sizes found no statistically significant difference between PRP and HA regarding pain visual analog scale scores at short-term (≤2 months; P = .27), midterm (4-6 months; P = .85), or long-term (1 year; P = .42) follow-up.”⁸ The full text explains that the first claim is supported by data from primary studies; however, upon review of the full text, we discovered that not all of this data was statistically significant. Garcia et al. made no attempt to conduct a meta-analysis or otherwise support the pooled data. In contrast, a meta-analysis was performed for the second claim to determine whether there were statistically significant differences between PRP and HA in pain reduction. Readers may be unaware that the first claim is not supported by a meta-analysis if only the abstract is reviewed. If confidence intervals were reported in the abstract (from the primary studies if a meta-analysis was not attempted), readers would be more informed about the statistical validity of the claim in question. In this example, the abstract is spun toward the efficacy of PRP for pain. The reader may incorrectly interpret the claim as having stronger support than actually exists because the lack of statistical validity is concealed in the abstract. The inclusion of confidence intervals in the abstract, regardless of the presence of statistical significance, clarifies to the reader which claims are supported by a quantitative meta-analysis and how significant the claim is.

Regarding spin type 13 (“Failure to specify the direction of the effect when it favors the control intervention”), 40% of the included studies did not specify the direction of the effect when it favored the control intervention. For example, Dong et al.⁶ conducted a meta-analysis of 24 RCTs (21 for knee OA, and 3 for hip OA) and concluded that PRP provided better effects than other injections for OA patients, especially in knee OA patients, in terms of pain reduction and function improvement at short-term follow-up. However, a subgroup analysis in the full text that examines the 3 hip OA studies alone finds that total Western Ontario and McMaster (WOMAC) score, WOMAC pain score, and WOMAC stiffness score are actually worse after PRP for hip OA compared to knee OA.⁶ In this case, the effect of the intervention runs in the opposite direction for the hip compared to the knee, but the difference is not only neglected in the conclusion but also spun to suggest that OA outcomes were also superior with PRP at the hip (and especially so in knee OA). For this reason, this abstract also contains spin type 3 (“Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention”) and type 8 (“Conclusion extrapolates the review's findings from a surrogate marker or a specific outcome to the global improvement of the disease”). As seen in the previous example, the abstract by Dong et al.⁶ is spun toward the efficacy of PRP. It is worth noting that confidence intervals were not included in the abstract (thereby containing spin type 14), and for the reasons described earlier, the conclusion would have had lower risk of these other types of spin had the confidence intervals been included.

The current state of the quality of PRP studies for hip OA is exemplified by our finding that 14 of the 15 included studies would be rated in the “critically low” category according to the AMSTAR 2 assessment of confidence in the results of systematic reviews.²⁵ Common missing criteria included explicit statement of an a priori establishment of the review methods, provision of a list of excluded studies with justifications, use of a satisfactory technique for assessing risk of bias in primary studies, assessment of the impact of risk of bias on the results if meta-analysis was performed (e.g., with meta-regression), and an adequate investigation of publication bias. Our findings highlight that the existing problems with bias and heterogeneity in the primary literature (e.g., lack of consistency in comparators, PRP preparation, follow-up timeline, radiological grade of osteoarthritis) are compounded by bias and spin in the systematic reviews that collate those data.

It is important to note that the presence of spin in abstracts is influenced by many factors under varying levels of control by authors. The systematic reviews in this study were limited by a small number of trials or study populations, which renders definitive conclusions difficult and increases the risk of bias and spin. Importantly, spin can be unintentional. Abstracts are constrained by word counts, leading authors to make choices about which findings are most important to relay, and by nature of compressing complicated analyses some amount of nuance can be lost. Journals may enforce style guidelines that leave out numerical details for the sake of brevity and readability. The above decisions are often made with the expectation that readers will glean important details from the full text which may be obscured to those who only read the abstract. Importantly, the prevalence of spin in the literature poses risks for clinical decision-making. For example, Boutron et al.¹⁹ performed a randomized trial to assess the impact of spin on the interpretation of results of abstracts in the field of cancer and found that clinicians who read abstracts with spin rated experimental treatments as more beneficial than those who did not.

Given the lack of high-quality trials for PRP in the treatment of hip OA, it is especially important for the systematic reviews that evaluate the available evidence to convey findings with as little spin as possible. We propose several strategies to reduce spin. We recommend adhering to PRISMA guidelines and preregistering study protocols with public registers like PROSPERO to ensure high quality study design. As our findings indicate that AMSTAR 2 rating can be associated with spin, we recommend that investigators assess the methodological quality of incorporated studies using critical appraisal tools. Furthermore, reporting wide confidence intervals in the abstract not only accurately conveys the strength of a study’s findings but can also prevent other types of spin by clarifying the quantitative basis of its claims. Improving the awareness of spin types via education can also help investigators reduce the incidence of spin. Further studies should continue to characterize spin in the literature of other fields, investigate ways in which it is influenced by various study characteristics, and propose methods to reduce its prevalence.

Limitations

This study was limited by multiple factors. Given that the use of PRP in hip OA is relatively understudied, only a small number of studies met our inclusion criteria, leaving our study underpowered for a regression and generally limiting the extent of the multivariate analysis. Multiple association tests were performed to test independence between study characteristics and spin types, which increases the risk of type I errors. Furthermore, the evaluation of spin and bias is to some extent subjective. We attempted to make the process as rigorous as possible via independent assessments by two different authors and the use of standardized training materials.

Conclusion

Spin is highly prevalent in abstracts of systematic reviews of PRP in the treatment of hip OA. Several associations were found between spin types and the study characteristics of AMSTAR 2 rating, Scopus CiteScore, Journal Impact Factor, and PROSPERO preregistration. When present, spin in the abstracts of reviewed studies tended to favor the use of PRP in hip OA.

Disclosures

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: F.A.P. has received consulting fees and/or hospitality payments from Stryker, Exactech, Micromed, Smith & Nephew, Zimmer Biomet, Flexion Therapeutics, DePuy Synthes. J.N.L. has received consulting fees and/or hospitality payments from Stryker Corp, Smith & Nephew, Merz North America Inc, and Neuracrine Biosciences and reports American Shoulder and Elbow Surgeons and Arthroscopy Association of North America (Board or committee member). Full ICMJE author disclosure forms are available for this article online, as supplementary material.