Microfracture: State of the Art in Cartilage Surgery?

Abstract

Objective

This study is a literature review from 2010 to 2014 concerning the quality of evidence in clinical trials about microfracture in attempt to repair articular cartilage. We have decided to focus on microfracturing, since this seems to be the best documented technique. Interest in evaluation of publication quality has risen in orthopaedic sports medicine recently. Therefore, we think it is necessary to evaluate recent clinical trials being rated for their evidence-based medicine (EBM) quality. We also compared the mean impact factor of the journals publishing the different studies as an indicator of the study’s citation and evaluated for a change over the studied time frame.

Design

To measure the EBM level, we applied the modified Coleman Methodology Score (CMS) introduced by Jakobsen. The impact factor, which is a measurement of the yearly average number of citations of articles recently published in that journal, was evaluated according to self-reported values on the corresponding journal’s website.

Results

We found that the mean CMS has not changed between 2010 and 2014. The mean impact factor has also not changed between 2010 and 2014. The CMS variance was high, pointing to different qualities in the evaluated studies. There is no evidence that microfracturing is superior compared to other cartilage repair procedures.

Conclusion

Microfracture cannot be seen as an evidence based procedure. Further research needs to be done and a standardization of the operating method is desirable. There need to be more substantial studies on microfracturing alone without additional therapies.

Introduction

This study is a literature review concerning the quality of evidence in clinical trials about microfracture surgery in attempt to repair articular cartilage. As there are many different joint resurfacing procedures, we decided to focus on microfracturing, since this seems to be the most widely performed and best documented technique. Other subchondral methods such as nanofracturing are further developments of microfracturing.¹

Currently, no gold standard seems to exist as to which surgical standard may be set^2,3 even though there are clinical trials comparing the different surgical approaches with each other. Those studies’ scientific level of evidence is being questioned by several publications.^4-7 Interest in evaluation of publication quality has risen in orthopaedic sports medicine research.⁸ Therefore, we think it necessary to evaluate recent clinical trials being rated for their evidence-based medicine (EBM) quality. To measure the EBM level, we intend to use the modified Coleman Methodology Score (CMS) introduced by Jakobsen et al. in 2005.²

We postulate the following theses:

1. Mean CMS has not changed between 2010 and 2014.

2. Mean impact factor has not changed between 2010 and 2014.

3. CMS variance will be high.

We feel it is of some importance to base therapy decisions on solid evidence. This is true in every medical specialty, including surgery. With patient emancipation becoming more and more developed and the patient’s interest to get the best treatment available a physician or surgeon should be able to give advice not only based on his or her opinion or experience but on scientifically sound data and research. Also—with keeping rising health care costs in mind—health insurance companies and their customers are more and more concerned about the efficacy of treatment, for example, the price/performance ratio.

Our goal with this literature review is to find out whether there is evidence for cartilage repair through microfracture or not. Should we find out that the quality of recent published clinical trials is high, this would open the possibility for others to give definitive treatment recommendations based on meta-analysis of available studies.

Material and Methods

In order to identify potential studies, we performed a search across multiple databases (Pubmed/MEDLINE, EMBASE by Elsevier, Cochrane Library) using a range of keywords. These keywords were the same for all 3 searched databases. The goal was to find all clinical studies including microfracture. The exact term entered with logical search operators can be seen below.

Database Keywords

The exact terms, including logical operators, were as follows: (subchondral drilling) OR microfracture OR microfracturing or (micro fracture) OR (micro fracturing) OR (cartilage and repair) OR (membrane AND collagen)

Filter

According to our needs, filters were applied to each search process. Those filters were the same for every database. We were only looking for clinical trials on humans with full text available ranging from January 1, 2010 to December 31, 2014. A language filter then was applied to sort out studies not written in English, German, or French.

Databases

The Pubmed/MEDLINE online database maintained by the US National Library of Medicine was searched first. After manually screening the search results and downloading eligible studies, the search process was repeated in the EMBASE online database by Elsevier. The EMBASE search yielded a lot of duplicates along with 2 additional clinical trials, which we could include. The search process then was again repeated using the Cochrane Library online database by the Cochrane Collaboration. Alongside many duplicates that were sorted out this showed 1 additional study.

Number of Studies

We could identify 20 clinical trials, including microfracture from 2010 to 2014. Seventeen of them (85%) were available through Pubmed/MEDLINE. Eight of those studies included only microfracture and its variants, while the other 12 were comparisons of microfracture with other cartilage repair techniques.

Modified Coleman Methodical Score

The CMS was first introduced by Coleman et al.⁹ in 2000 to assess the quality of clinical trials concerning the surgical treatment of patellar tendinopathy. It was then modified by Jakobsen in 2005 with the purpose of evaluating cartilage repair studies. Less emphasis on postoperative rehabilitation compliance was laid because this information was seldom given in evaluated articles.² A maximum of 100 points can be reached. According to Jakobsen et al.,² “A score of 100 indicates that the study largely avoids chance, various biases, and confounding factors.”

In their original review, Jakobsen et al.² calculated an average CMS of 43.5, which they interpreted as a generally low methodological quality. Another publication by Benthien et al.¹⁰ from 2010 found an average CMS of 58, which was also interpreted as generally low.

The modified CMS provides an adequate tool to assess the scientific quality of cartilage repair studies on a deeper level. It is important to compare one study with others because of the high variance and different emphases those studies have.¹¹

Details of the score can be seen in Table 1.

Table 1.

The Modified Coleman Methodology Score.

Section	Number or Factor	Score
Part A (only 1 score for each section possible)
1.  Study size: number of patients	>60	10
	41-60	7
	20-40	4
	<20 or not stated	0
2.  Mean follow-up (months)	>24	5
	12-24	2
	<12 or not stated or unclear	0
3.  Number of different surgical procedures included in each reported outcome	One surgical procedure	10
	More than one surgical procedure, but >90% of subjects undergoing the one procedure	7
	Not stated or unclear or <90% of subjects undergoing the one procedure	0
4.  Type of study	Randomized controlled trial	15
	Prospective cohort study	10
	Retrospective cohort study	0
5.  Diagnostic certainty	In all	5
	In >80%	3
	In <80%	0
6.  Description of surgical procedure given	Adequate (technique stated and necessary details of that type of procedure given)	5
	Fair (technique only stated without elaboration)	3
	Inadequate or not stated or unclear	0
7.  Description of postoperative rehabilitation	Well described	10
	Not adequately described	5
	Protocol not reported	0
Part B (scores possible for each option in each section)
1.  Outcome criteria	Outcome measures clearly defined	2
	Timing of outcome assessment clearly stated	2
	Use of outcome criteria that has reported good reliability	3
	Use of outcome with good sensitivity	3
2.  Procedure for assessing outcomes	Subjects recruited (results not taken from surgeon’s files)	5
	Investigator independent of surgeon	4
	Written assessment	3
	Completion of assessment by subjects themselves with minimal investigator assistance	3
4.  Description of subject selection process	Selection criteria reported and unbiased	5
	Recruitment rate reported >80%	5
	Recruitment rate reported <80%	3
	Eligible subjects not included in the study satisfactorily accounted for or 100% recruitment	5

Impact Factor

The impact factor (IF) is used to compare journals with each other in order to rank them. It is a ratio calculated by dividing all citations of papers published in the span of 1 year by total number of papers published in the 2 previous years. It its published by the Journal Citation Reports.¹²

In this review, we associate each publishing journal’s IF with the corresponding study. This allows us to see if the mean IF of publications changes over the observed time, meaning an increase or decrease of quality for journals publishing trials about microfracture. The impact factor was evaluated per self-reported values on the corresponding journal’s website. Journals not rated by the Journal Citation Reports were assigned a hypothetical IF of 0.

Results

In the year 2010, 2 studies could be identified. The amount rises with 5 studies in 2011 and declines with 4 studies in 2012. In 2013 there were 6 and in 2014, 3 studies ( Fig. 1 ).

Figure 1.

Number of studies per year.

The study with the least amount of CMS points scored 34. No study reached the maximum of 100 points, the best ranked study scored 80 points. The exact distribution can be seen in Figure 2 .

Figure 2.

Number of studies ranked by the modified Coleman Methodology Score (CMS).

A mean CMS of 66.6 (64.4 if considering only those 8 studies solely about microfracture) has been reached in those 4 years. Mean CMS in 2010, 2011, 2012, 2013, and 2014 were 71, 66.2, 69.8, 65.5, and 62.3, respectively.

To prove our thesis, we compared the group from 2010 (2 studies with a mean of 71) with the group from 2014 (3 studies with a mean of 62.3). A 2-tailed 2-sample unequal variance Student t test was performed resulting in a P value of 0.60499. This means there is no significant difference between the mean CMS in 2010 and 2014 ( Fig. 3 ).

Figure 3.

Mean modified Coleman Methodology Score (CMS) per year.

A mean IF of 2.953 (2.6165 if considering only those 8 studies solely about microfracture) has been reached in those 4 years. Mean IFs in 2010, 2011, 2012, 2013, and 2014 were 2.676, 3.817, 2.902, 2.113, and 3.446, respectively.

To prove our thesis, we compared the group from 2010 (2 studies with a mean of 2.676) with the group from 2014 (3 studies with a mean of 3.446). A 2-tailed 2-sample equal variance Student t test was performed resulting in a P value of 0.399571. This means there is no significant difference between the mean IF in 2010 and 2014 ( Fig. 4 ).

Figure 4.

Mean impact factor (IF) per year.

We already saw that the lowest study scored 34 and the highest 80. The standard deviation is about 10.6 points (s² = 112.15). This equals about the difference between a randomized controlled trial and a retrospective cohort study in the CMS and should therefore be considered high. If we look separately at the 8 studies only concerning microfracture, the standard deviation is even higher with 12.8 points (s² = 163.41). In Figure 5 , all studies are listed by their CMS to further illustrate the variance.

Figure 5.

Modified Coleman Methodology Score (CMS) variance.

Discussion

Hypothesis 1: Mean CMS Has Not Changed from 2010 to 2014

This hypothesis turned out to be correct. The scientific quality level of the examined studies has remained the same over the years. It is hard to say whether that level is low or high, since the CMS does not really allow to make an absolute statement about the quality. We can simply note that on average, these studies fulfil 67% of the criteria that a perfect study would meet. We think that this is far from optimum and can be ameliorated. A good way to do this would be to take the CMS (or some other similar or even better guideline) prior to study design and build the study around it.

Hypothesis 2: Mean IF Has Not Changed from 2010 to 2014

This hypothesis turned out to be correct. Regarding the outcome of the previous hypothesis, it would have been alarming otherwise. The scientific quality did not increase; therefore, the higher rated journals did not increase their interest in publishing those studies. In order to publish in “better” journals, study authors could improve the scientific quality level of their studies. This seems logical but is also proven by the data in our review.

Hypothesis 3: The CMS Variance Will Be High

This hypothesis turned out to be correct. We expected a high variance because generally speaking, in every speciality there are better and worse studies. We feel that another cause for high variance lies within the lack of any standards. Surgical techniques are not standardised, there is no common rehabilitation protocol and there is no standard in how to assess the outcome of microfracture. There are also a lot of different foci made by the studies. Some report mainly on the rehabilitation, some rely heavily on magnetic resonance imaging while a few use histology. This could be avoided by further narrowing the inclusion criteria in our study, but a better way to achieve more homogenous and therefore more comparable studies would be some kind of standard of what to include and measure and what not. For example, it would be useful for every study to include a description of the surgical process, since there are a lot of potential significant differences between how a certain surgeon performs microfracture compared to another. Another example would be to include a detailed report on the rehabilitation protocol, which could also potentially affect the outcome of a study.

Comparison of Results with the Literature

In 2011, there was an article similar to our review, evaluating the quality of evidence in subchondral surgery.¹⁰ This study by Benthien et al. did include microfracture, autologous chondrocyte implantation (ACI), matrix-induced ACI (MACI), and osteoarticular transfer system (OATS). In 133 studies, they found a mean CMS of 58. We do not think that this mean is comparable to ours (e.g., saying that scientific quality improved over the past 5 years because we found a higher mean of 67 would be wrong), because there are a few quite subjective elements to the CMS—this will be discussed further under limitations. Benthien et al.¹⁰ did also find that different studies are hard to compare.

There is another review from 2009 by Mithoefer et al.⁴ evaluating the quality of studies regarding only microfracture also using the CMS. In 28 studies, they found an average CMS of 58. They also conclude that the variance in study design and outcome reporting leads to a limited quality of evidence. Contrary to what Benthien et al.¹⁰ and we found, Mithoefer et al.⁴ note that “the quality of clinical studies on microfracture has increased over time, confirming that the importance of methodological quality is increasingly recognized.” This might have been true in 2009, but since then we could not observe an increase in quality. They, however, agree with Benthien et al.¹⁰ with the demand for large, well-designed, multicenter studies in order to provide solid evidence.

At last, there is the 2005 study by Jakobsen et al.² In accordance with the literature and also our review, Jakobsen et al. note that studies are very heterogeneous and many different outcome measurement scales are being used.

Strengths of Our Study

The Modified Coleman Methodology Score

We feel that one of the strengths of this study lies in its well-documented material and methods part. With the given search terms, databases and filters, reproducibility is guaranteed. The broad search strategy made sure that every possible study available was included. Regarding the chosen databases, a prospective study by Woods and Trewheellar¹³ has shown that Medline and EMBASE complement each other in literature searches. Together they covered enough evidence to answer all questions used in this study.¹³ We also think that by using the CMS, we got the best currently available instrument to assess microfracture trials.

Relationship to Already Published Material

The state of evidence in microfracture surgery is an area of interest not only for orthopaedic surgeons, but also for health insurance providers and patients. The demand for better evidence in this field has been stated before,^2,4,10 and this review underlines the importance of this claim and shows its topicality today. By working with the CMS, we used the same tool like previous researchers and establish a methodological coherence. The fact that our work supports (and expands) older findings is in line with the scientific paradigm of EBM.

Limitations of Our Study

Generally, reviews concerning the quality of studies on microfracturing should consider that microfacturing is rarely performed as a single procedure in a patient. There are not enough studies on microfracturing performed without additional procedures to perform a validated review. This should be considered in discussing our results.

The Modified Coleman Methodology Score

We used this scoring system simply because we did not find another score tailored to our specific task and because it was being employed previously in similar studies. We think that the CMS has some flaws, one of them being subjective in some points. While a lot of items are objective, such as study size (number of patients) and mean follow-up (measured in months), there are also other items that are dependent on the assessor’s interpretation. For example, there is no standard for what is a “well-described” rehabilitation protocol. Therefore, we feel that the score points given in this category are less reliable than the ones from other, more objective categories. Another example of subjective items is the item “completion of assessment by subjects themselves with minimal investigator assistance.”² The term minimal assistance is not well defined. The demand for the use of outcome criteria with “good” sensitivity is difficult to answer as there is no explanation what standard an outcome criterion should fulfil in order to be classified as “good.” The criterion that selection should be unbiased also remains unclear to an extent as there is no definition of what counts as a bias in selection and what not.

Comparison of Studies Is Difficult due to Heterogeneous Designs

The high variance found in study design, used outcome measuring scores, and clinical focus is a problem that our study’s design had to consider. Applying a narrower set of filters would most likely have resulted in a much smaller sample size and would probably not have yielded any useful results. This is a problem that seems to lie within surgical clinical trials in general and of course also affects orthopaedic research.

Conclusion

Currently, although widely performed, microfracturing cannot be conceived as an EBM-based state-of-the-art procedure in joint resurfacing and cartilage regeneration. According to our investigations further and more homogenous level I and II studies are needed as is a common consensus about a standardization of this procedure.