Arthroscopic meniscectomy versus non-surgical or sham treatment in patients with MRI confirmed degenerative meniscus lesions: a protocol for an individual participant data meta-analysis

Abstract

Introduction

Arthroscopic partial meniscectomy (APM) after degenerative meniscus tears is one of the most frequently performed surgeries in orthopaedics. Although several randomised controlled trials (RCTs) have been published that showed no clear benefit compared with sham treatment or non-surgical treatment, the incidence of APM remains high. The common perception by most orthopaedic surgeons is that there are subgroups of patients that do need APM to improve, and they argue that each study sample of the existing trials is not representative for the day-to-day patients in the clinic. Therefore, the objective of this individual participant data meta-analysis (IPDMA) is to assess whether there are subgroups of patients with degenerative meniscus lesions who benefit from APM in comparison with non-surgical or sham treatment.

Methods and analysis

An existing systematic review will be updated to identify all RCTs worldwide that evaluated APM compared with sham treatment or non-surgical treatment in patients with knee symptoms and degenerative meniscus tears. Time and effort will be spent in contacting principal investigators of the original trials and encourage them to collaborate in this project by sharing their trial data. All individual participant data will be validated for missing data, internal data consistency, randomisation integrity and censoring patterns. After validation, all datasets will be combined and analysed using a one-staged and two-staged approach. The RCTs’ characteristics will be used for the assessment of clinical homogeneity and generalisability of the findings. The most important outcome will be the difference between APM and control groups in knee pain, function and quality of life 2 years after the intervention. Other outcomes of interest will include the difference in adverse events and mental health.

Ethics and dissemination

All trial data will be anonymised before it is shared with the authors. The data will be encrypted and stored on a secure server located in the Netherlands. No major ethical concerns remain. This IPDMA will provide the evidence base to update and tailor diagnostic and treatment protocols as well as (international) guidelines for patients for whom orthopaedic surgeons consider APM. The results will be submitted for publication in a peer-reviewed journal.

PROSPERO registration number

CRD42017067240.

Strengths and limitations of this study.

• To our knowledge, this is the first study that combines the individual participant data of randomised controlled trials (RCTs) performed on arthroscopic partial meniscectomy, maximising the capability to detect subgroups that may benefit from the surgery.

• The main advantage of an individual participant data meta-analysis is that no large-scale RCT is required, but instead the power of existing studies is combined to achieve large patient numbers.

• Trial data might not be available, not accessible or sharing is not possible due to a stringent informed consent that only enables the use of the data for the original study. This might limit the amount of trials we can include.

• We are dependent on the outcomes that have been used in the included studies. These can differ between studies.

Background

Arthroscopic partial meniscectomy (APM) is a regularly performed surgical procedure intended to treat symptoms believed to be caused by degenerative meniscus lesions.^1–3 Degenerative lesions are typically observed in middle-aged and older people, and are caused by chronic degenerative processes.^{4 5} Over the past decade, evidence has accumulated that questions both the rationale for, and the effectiveness of APM for degenerative meniscus lesions.^{6 7} Additionally, concerns have been expressed on the harms associated with the procedure^7–9 and the potential detrimental effect of the procedure on the progression of osteoarthritis.^10–12 Still, the number of surgical procedures performed in the treatment of degenerative meniscus lesions remains high.^13–17

Orthopaedic surgeons have expressed concerns about the generalisability of the trial results and point out that the study samples are not representative of the subjects they select for surgery in their day-to-day clinical practice.^18–24 The common perception by most surgeons is that there are subgroups of patients that do need the procedure to improve.⁸ Hence, applying the mean effects of randomised controlled trials (RCTs) to individual patients in day-to-day practice runs against the intuitive approach of doctors to use the specific characteristics of a particular patient to tailor management accordingly. Unfortunately, the identification of subgroups of patients that may/may not benefit from the procedure has been problematic, as the individual trials performed so far were too small to perform valid and reliable subgroup analyses.

An individual participant data meta-analysis (IPDMA), that is, a meta-analysis on the original individual participant data of previously performed trials, has been described as the gold standard of systematic review and meta-analysis. An IPDMA offers the unique opportunity to recode, and re-analyse all original trial data, and evaluate the effectiveness of surgical treatment of degenerative meniscus lesions and to identify potential subgroups more likely to benefit from the intervention. Identifying these subgroups can assist physicians to make personalised treatment decisions and thereby improving the overall quality of life of patients that are currently selected for APM.

Therefore, the objective of this IPDMA is to assess whether there are subgroups of patients with degenerative meniscus lesions who benefit from APM in comparison with non-surgical or sham treatment.

Methods

The protocol is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis protocols statement and is registered in PROSPERO with registration number: CRD42017067240.²⁵ The first part of the method section describes a regular systematic review to identify eligible papers and invite the study authors to collaborate and contribute data. The second part describes the analysis with the individual participant data.

Patient and public involvement

Patients and members of the public were not involved in development of the protocol. A panel of patient representatives will provide detailed input regarding outcomes and the interpretation of the results from this IPDMA.

Part 1: Identifying eligible papers and data collection

Eligibility criteria

This IPDMA will include RCTs that evaluated the effectiveness of (partial) meniscectomy compared with non-surgical or sham treatments in persons with MRI-verified degenerative meniscus lesions. Degenerative meniscus lesions are typically observed in middle-aged and older people and may be the result of early degenerative knee disease. Persistent knee symptoms may encompass knee pain, limitation of function and mechanical symptoms, such as the sensation of catching or locking of the knee. Non-surgical or sham treatments may include, but are not limited to, sham surgery, pain and/or anti-inflammatory medication, exercise programme and/or watchful waiting. Trials that included persons with traumatic meniscal lesions, defined as being the result of a specific traumatic incident will be excluded. There will be no restrictions on publication date, type of setting, length of follow-up or language.

Identification and selection of eligible trials

The search strategy described by Thorlund et al ⁷ will be adopted to systematically search for eligible trials in Medline, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science and the Cochrane Central Register of Controlled Trials (online supplementary additional file 1). The identified studies will be exported to Early Review Organizing Software (developed by Institute of Clinical Effectiveness and Health Policy, Buenos Aires, Argentina) to remove duplicates, and randomly allocate references to two independent reviewers responsible for screening and selection. The two reviewers will independently screen all titles and abstracts of identified reports for eligibility. Full-text copies of all publications regarded as potentially eligible for inclusion, or where there is any uncertainty, will subsequently be assessed. Trials will be included when they meet the eligibility criteria. Any discrepancies between reviewers will be resolved by discussion, and if necessary, a third reviewer will be consulted. In addition, the reference lists of included studies will be reviewed to identify additional eligible trials. The electronic database search will be supplemented by searching for additional eligible trials in the WHO International Clinical Trials Registry Platform search portal, which contains the trial registration datasets provided by several registries. This portal includes 16 national and international primary registries, including ClinicalTrials.gov, Australian New Zealand Clinical Trials Registry (ANZCTR), Japan Primary Registries Network (JPRN) and International Standard Randomised Controlled Trial Number (ISRCTN). The corresponding authors of eligible trials will be invited to collaborate in the current IPDMA by sharing their data.

Collection of individual participant data

Data collection and transfer

Time and effort will be spent in tracing and encouraging original investigators to share their trial data. If no reply is received on a first invitation, additional inquiries will be sent, including inquiries sent to alternative email addresses identified for the corresponding author, inquiries sent to listed co-authors and to the institution of the corresponding author listed in the original publication. The principal investigators of the original trials collaborating in the current project will be encouraged to actively participate in the IPDMA and discuss and finalise the definitions and outcomes to be assessed and the analytical processes proposed. Where possible, a face-to-face collaborator meeting will be scheduled, at which key decisions, including the project design, analysis plan and interpretation of findings will be discussed. Before sharing of the de-identified data, we will sign a data sharing agreement with those principal investigators of the original trials that are interested in collaboration, in which we will arrange that the research data will be used for the declared purposes and the data will be stored on secured servers located in the Netherlands.

Data check and risk of bias

All received data will first be validated to match the results of the original publication. Statistical tests will be repeated and analysed in R (R Foundation for Statistical Computing, Vienna, Austria). The trial data provided by original investigators will be checked for consistency, plausibility, integrity of randomisation and reproducibility of published trial results. The aims of checking data are to increase the probability that data supplied are accurate, and to confirm that trials were appropriately randomised. Inconsistencies will be discussed and resolved with the individual investigators. All checked and de-identified data of randomised participants will be entered into a pooled database, and every trial will be assigned a trial number. Data will include characteristics relating to the participants (age, gender and body mass index); radiographic information on knee osteoarthritis; onset, duration and severity of symptoms; generic and disease-specific health-related quality of life); non-surgical or sham procedure; trial (sample size, setting and allocation concealment); and outcome measures of interest. For eligible trials of which original data are not available the aggregated data from trial reports will be collected.

Checking the individual participant data (IPD) directly can provide more reliable investigations of key potential biases, some of which might be reduced or alleviated in the process. The risk of bias in included trials will be independently assessed by checking the IPD directly. Randomisation and allocation concealment will be assessed by checking if both treatment arms are balanced in every study.²⁶ The advantage of an IPDMA is that we can also include outcomes not reported by the original journal article, possibly reducing outcome reporting bias by checking all relevant outcomes at the same time. In order to avoid ecological bias, the within-trial information will be examined for individual predictors of treatment effect, separately from the across-trial information.²⁷

The potential for publication bias and small study effects will be examined, in the context of visual inspection, using a contour-enhanced funnel plot.^{28 29} To avoid availability bias, aggregated data from studies lacking individual participant data will be used to consider their potential impact.

To enable to assessment of homogeneity/heterogeneity between the included trials, the following characteristics of the included RCTs will be compared and described in a table: (1) selection of participants, (2) previous (conservative) treatment(s) before randomisation, (3) inclusion criteria and exclusion criteria, (4) description of clinical path prior to inclusion, (5) number of participants that declined participation, (6) diagnostic characteristics, including traumatic or non-traumatic injury and presence or absence of osteoarthrosis, (7) work characteristics, (8) socioeconomic characteristics, (9) intervention and control treatment, (10) cross-over, (11) adherence to the intervention in both treatment arms, (12) other healthcare services during follow-up and (13) outcome measures. These study characteristics will be used to assess which trials can enter the meta-analysis and to determine the generalisability of the results.

Missing data

The final IPDMA dataset will have a multilevel (ie, clustered) structure, where the individual trials are the levels (ie, clusters). A foreseen feature of this dataset will be that some variables will be systematically missing, that is missing for all individuals in one or more trials. Next, to systematically missing variables, we may also encounter sporadically missing variables, that is missing for some but not all individuals in one or more trials.

In order to optimally use the available participant data, reduce bias, and to increase statistical efficiency, incomplete data will be imputed using imputation methods that handle both systematically and sporadically missing covariates in a two-level structure using hierarchical multiple imputations by chained equations.^30–33

Outcomes variables

The most important outcomes according to surgeons and patients is treatment effect, determined as the difference between the intervention (surgery) and control group (non-surgical treatment) in knee pain, function and quality of life 2 years after the intervention. The preferred outcome measure instrument will be the Knee injury and Osteoarthritis Outcome Scale (KOOS). The KOOS is an instrument developed with the purpose of evaluating short-term and long-term symptoms and function in subjects with a knee injury and osteoarthritis. The KOOS consists of five subscales: pain, other symptoms, function in daily living, function in sport and recreation and knee-related quality of life and a composite score can be calculated (referred to as the KOOS5). The KOOS has been validated for several orthopaedic interventions such as anterior cruciate ligament reconstruction, meniscectomy and total knee replacement.³⁴ The score of the KOOS pain subscales will be compared between the intervention and control group across the included studies. Other measurement instrument targeted at quantifying pain will be standardised and combined in one single pain outcome. The functional outcome will be measured by using the KOOS function subscale or equivalent outcome aimed to measure function. The health quality of life will be measured by using the EuroQol-5 dimensions questionnaire or the 36-Item Short Form Survey.

Other outcomes of interest will include the difference between intervention and control group in adverse events (defined as deep venous thrombosis, pulmonary thromboembolism, venous thromboembolism, infection and death) associated with the intervention from baseline to 2 years after intervention; difference in mental health; difference in risk for future knee replacement surgery between groups (feasibility will depend on whether the follow-up of the trials will be long enough to capture the events), and the effect of follow-up time after the intervention on the treatment effect.

Part 2: Analysis

Treatment effect

The IPDMA will be used to assess the general effectiveness of APM in comparison with non-surgical or sham treatments in patients with knee symptoms and degenerative meniscal lesions. For this part, we will apply both a two-stage and a one-stage approach.³⁵ In the two-stage approach, we will perform regression analyses for each study to obtain effect estimates separately. Thereafter, these are pooled in a random effects model (to account for heterogeneity), such as a regular meta-analysis. The random effects models in the two-stage will be analysed using the restricted maximum likelihood (REML) approach with the Hartung-Knapp-Sidik-Jonkman method for continuous outcomes to account for uncertainty due to heterogeneity.³⁶ The two-stage approach is ideal for assessing pooled treatment effect and detect heterogeneity. However, it is difficult to detect non-linear trends or account for correlating covariates.

In the one-stage approach, the IPD from all studies will be analysed simultaneously by adopting a single statistical model (random effects model) that fully accounts for heterogeneity across studies, while accounting for the clustering of participants within studies. The one-stage approach is more flexible and more accurate compared with the two-stage approach but can face computational difficulties. Therefore, the results from the one-stage approach will be compared with the results of the two-stage approach and differences will be investigated. The random effects models in the one-stage approach will be analysed using the REML approach with the Kenward-Rogers approach for continuous outcome and the maximum likelihood method with quadrature for binary or survival outcomes.^{35 37 38} Heterogeneity will be addressed by I² and τ², reflecting the heterogeneity between studies. To reflect the variation of the treatment effect in a different setting, 95% prediction intervals will be reported to provide more information on the expected effect in future patients.³⁹

A key advantage of a one-stage approach is the flexibility in terms of the models that may be fitted compared with a two-stage approach. One-stage models allow for the inclusion of multiple covariates in a single model, multiple random effects on different parameters, correlation between covariates and the separation of within and across-trials information. It is this flexibility that is essential to the primary objective of this IPDMA.⁴⁰

Heterogeneity in treatment effect (subgroups)

To investigate which patients may benefit from (partial) meniscectomy we will assess whether the treatment effect is modified by baseline patient characteristics. First, relevant baseline patient characteristics will be identified. Modern regression procedures with penalisation of estimated regression coefficients will be applied for the selection of those characteristics that are independent predictors of the treatment effect.^{41 42} A full list of the baseline patient characteristics that will be taken into consideration is listed in online supplementary additional file 2. Second, it will be assessed whether these identified independent baseline predictors (individually or in combinations) modify the treatment effect. Effect modification will be assessed with a random effects model. In this model, a dummy for the particular study will be the random effect and APM (yes vs no), the potential effect modifier, and an interaction term (APM * potential effect modifier) will be included as fixed variables and the treatment effect as dependent variables.³³ The illustrated approach will allow the assessment of effect modification without overfitting the data and reducing the risk of type I errors.

In addition, we want to ensure that patient characteristics deemed to be of high clinical relevance in day-to-day clinical practice are ultimately evaluated for effect modification. For this purpose, alongside the procedure described above, we will also assess a set of predefined patient characteristics deemed to be of high clinical relevance for effect modification. To define these characteristics, the IPDMA collaborators will be asked to provide a top-3 of characteristics that they regard as most clinically relevant. Subsequently, it will be assessed whether the overall top-3 of these predefined patient characteristics modify the treatment effect. Predefining, these characteristics will be performed before actual analysis of the data.

Sensitivity analysis

To analyse the robustness of the results from the IPDMA, several sensitivity analyses will be performed. First, to evaluate the impact of including aggregated data of published trials (of which the IPD was not available in the meta-analysis), analyses will be performed in which we either only include studies with IPD available or only studies of which only aggregated data were available. Second, to determine the effect of imputation of missing values on the study outcome, analyses will be performed in which we impute either only systematic missing variables, only sporadically missing variables (within trials) or not impute at all. Third, we will study whether the persistence of complains is a relevant subgrouping variable.

All analyses will be performed according to the intention-to-treat principle.

Publication considerations

The draft version of the final manuscript will be circulated among the collaborators for further discussion prior to submission for publication. The authors of the IPDMA paper(s) will be the project team managing the IPDMA, followed by the collaborators, whom are the principal investigators that collaborated in the current project by sharing their trial data and commenting on the results and draft of the papers.

Study status

Currently, we are collecting the data and are contacting the original investigators of the included trials and encourage them to share the trial data. We have already received a part of the data and are still waiting on the data of a few trials. We expect to end data collection in Q1 2020. After validation of the data, we will start with the analyses. Our aim is to publish our results in 2020/2021.

Discussion

In the last decade, several RCTs found no or very little benefit of APM compared with non-operative or sham treatment in patients with MRI confirmed degenerative meniscus tears,^43–52 although there is some evidence that it is effective in middle-aged patients with degenerative meniscal symptoms.⁵³ These findings started a discussion on the effectiveness of the surgery and the methodology used in those RCTs by both orthopaedic surgeons and other healthcare professionals.^{19–23 54} The published studies were not able to adequately tease out whether or not there are subgroups that do additionally benefit from APM. This has resulted in a deadlock: APM is continued to be performed, despite level I evidence that discourage the treatment.¹³

The proposed IPDMA provides the opportunity to evaluate the relationship between potential clinically relevant baseline characteristics and the effectiveness of the APM of all patients that have been included in the trials, and to possibly detect subgroups that may benefit from APM. IPDMA is the gold standard of systematic review and meta-analysis that provides more power and is less prone to bias compared with meta-analysis on aggregated data. To our knowledge, this is the first study that combines the individual participant data of RCTs performed on APM, maximising the capability to detect subgroups that may benefit from the surgery. The main advantage of an IPDMA is that no large-scale RCT is required, but instead the power of existing studies is combined to achieve large patient numbers. This prevents additional trials and patient involvement. Moreover, combining individual patient data enable us to analyse the effects of within-study and between-study moderators of effect sizes, even though the original studies were too small to analyse such samples.

Although IPDMA is the best method to detect possible subgroups, performing an IPDMA also comes with several challenges. First, all individual patient data from the eligible trials have to be collected. This time-intensive task often requires us to contact the principal investigators multiple times to invite them to collaborate. Unfortunately, data are sometimes not available, not accessible or sharing is not possible due to a stringent informed consent that only enables the use of the data for the original study. While there are guiding principles for open data management and sharing (Findable, Accessible, Interoperable and Reusable data principles) these often conflict with the rules of the informed consent or national legislations, creating a tension between privacy and reuse of (anonymous) medical data.⁵⁵ This might limit number of studies that can be included in this IPDMA. Second, there are multiple ways to measure knee pain, knee function or general quality of life. Every researcher can or will use their own set of outcome parameters, dependent on the experience of the researcher with a certain questionnaire/scoring system, preference or the time-dependent academic insights of the optimal questionnaire/scoring system (especially if studies have been published in different time periods, ie, different research paradigm). As a result, we are dependent on the outcomes that have been used in the included studies in the IPDMA and cause systematic missing variables in the final pooled dataset of an IPDMA. These missing variables pose a methodological challenge, because they require advanced imputation that preserves the hierarchical structure of the data followed by the one-stage meta-analysis.⁵⁶

In conclusion, the aim of this project is to identify potential subgroups of patients with degenerative meniscus lesions who may benefit from APM. It will provide the evidence base to update and tailor diagnostic and treatment protocols as well as (international) guidelines for patients for whom orthopaedic surgeons consider APM. Identifying potential subgroups can improve the quality of life of patients who do truly benefit from the treatment, and can perhaps be implemented with a clinical decision aid. In case we do not find a subgroup that has additional benefits from the treatment, it can help to reduce the number of APMs, and thus less risk of, eg, complications.^{9 57}