Attitudes of patients and surgeons towards sham surgery trials: a protocol for a scoping review of attributes to inform a discrete choice experiment

Abstract

Introduction

In order to properly evaluate the efficacy of orthopaedic procedures, rigorous, randomised controlled sham surgery trial designs are necessary. However, randomised controlled trials (RCTs) for surgery involving a placebo are ethically debated and difficult to conduct with many failing to reach their desired sample size and power. A review of the literature on barriers and enablers to recruitment, and patient and surgeon attitudes and preferences towards sham surgery trials, will help to determine the characteristics necessary for successful recruitment.

Methods and analysis

This review will scope the diverse literature surrounding sham surgery trials with the aim of informing a discrete choice experiment to empirically test patient and surgeon preferences for different sham surgery trial designs. The scoping review will be conducted in accordance with the methodological framework described in Arksey and O’Malley (2005) and reported using the Preferred Reporting Items for Systematic Reviews and Meta-analyses Protocols extension for Scoping Reviews. The review will be informed by a systematic search of Medline, Embase, PsycInfo, CINAHL and EconLit databases (from database inception to 21 June 2019), a Google Scholar search, and hand searching of reference lists of relevant studies or reviews. Studies or opinion pieces that involve patient, surgeon or trial characteristics, which influence the decision to participate in a trial, will be included. Study selection will be carried out independently by two authors with discrepancies resolved by consensus among three authors. Data will be charted using a standardised form, and results tabulated and narratively summarised with reference to the research questions of the review.

Ethics and dissemination

The findings from this review will inform the design of a discrete choice experiment around willingness to participate in surgical trials, the outcomes of which can inform decision and cost-effectiveness models of sham surgery RCTs. The qualitative information from this review will also inform patient-centred outcomes research. The review will be published in a peer-reviewed journal.

Trial registration number

CRD42019133296.

Strengths and limitations of this study.

• This scoping review forms a critical stage of the development of a discrete choice experiment which will comprehensively and empirically evaluate patient and surgeon preferences to participation in sham surgery trials.

• To capture all possible factors affecting recruitment of randomised controlled trials, this review will not restrict the inclusion criteria to only surveys and interviews, which are limited in the information they provide by the questions they ask, but will also include published reviews and expert opinion, which can explore a greater number of themes.

• This scoping review will not collect grey literature, unpublished work or work written in languages other than English.

• The breadth of inclusion criteria, allowing published opinion pieces, introduces a risk that these pieces will primarily reflect one point of view.

Introduction

Background

The double-blinded randomised controlled trial (RCT) is the ‘gold standard’ of medical research and recommended across a range of fields for assessing new interventions.^1–3 This process is more difficult, however, when the intervention is surgical. The surgeon can never be fully blinded and the extent to which the placebo resembles the surgery is practically and ethically problematic.⁴ There is some inconsistency in the requirements of a ‘sham’ surgery; however, the definition we will use throughout this review is the same as Jonas et al (2015, p.3), who define an invasive surgical procedure as a procedure where “an instrument is inserted into the body … for the purpose of manipulating tissue or changing anatomy”,⁵ and a sham procedure as one that involves the same surgical or invasive procedure with associated instruments and ritual, but without the critical component of tissue manipulation.

The use of sham and placebo surgeries in surgical trials is controversial. Some commentators have argued that surgical placebos are essential, and that surgical interventions should be held to the same rigorous assessment as other medical procedures,^{6 7} particularly given the expense and risks involved with untested surgical procedures.^{8 9} However, others have argued it is misguided to consider sham surgery as analogous to sugar pills due to the increased risks and harm to participants without any benefit^10–12 and as such consider sham surgery to be unjustified and unethical.

These issues have become particularly significant in orthopaedic surgery, with the efficacy of some orthopaedic procedures being called into question. Recent observational studies have shown that a significant proportion of patients report no clinical meaningful improvement in pain or function after knee replacement surgery.^{13 14} Further, several recent systematic reviews of RCTs in orthopaedic surgery have suggested that surgery is no better than a sham procedure in reducing pain,^{5 9 15} although firm conclusions regarding efficacy are constrained by limitations of the trials including incomplete recruitment.⁵ In a number of countries worldwide, orthopaedic surgeries are high-volume and expensive procedures, with considerable risks and significant recovery times for patients.¹⁶ Therefore, there is increasing pressure to improve the evidence base around orthopaedic surgical procedures. In order to properly evaluate the efficacy of orthopaedic procedures, rigorous RCT designs, blinded through sham surgery, are necessary. There is evidence to suggest that the number of RCTs in surgery is increasing; however, their use in comparison with other study types remains low.^{17 18} Furthermore, RCTs for surgery involving a sham procedure are ethically debated and they are also difficult to conduct with most trials not reaching their desired sample size for sufficient power.⁸

A recent systematic review conducted by the authors on sham surgery trials in orthopaedics found that nine out of ten trials had some risk of bias, and in two trials the benefits of the sham procedure were not considered to outweigh the risks involved.⁸ Furthermore, it was found that participant recruitment within reasonable timeframes and low threshold for crossover were consistent challenges. This highlighted that there are considerable issues around the feasibility of such trials. This is symptomatic of clinical trials generally, with up to 50% of funded trials not publishing within their funding period.¹⁹ As one of the main reasons for patient participation in sham surgery trials is contribution to research,²⁰ if the trial is subsequently abandoned, participants are exposed to potential harm for no benefit to society, which presents an ethical dilemma.

In order to ensure that research funds as well as patient, surgeon and researcher time are not wasted, it is important to understand both the challenges and feasibility of conducting a sham surgery trial within the anticipated timeframe, and the barriers and enablers to participation in sham surgery RCTs of orthopaedic procedures. In order to more thoroughly explore the feasibility of sham surgery trials, a follow-up systematic review by members of this research team is currently being conducted to investigate any disparity between planned (protocol and trial registry) and actual (outcome papers) outcomes from sham surgery trials. This review will identify potential trial characteristics associated with differences in recruitment rates; however, it is also considered important to identify information regarding patient and surgeon preferences for trial participation.

Rationale

To date, there has been no study which comprehensively evaluates the effect of the variety of factors related to sham surgery trials that influence participation decisions of patients and surgeons. Through investigating the preferences of patients and surgeons regarding participation, we can determine the various factors necessary for a successful trial in this area; improving design, acceptability and feasibility, and ultimately patient and surgeon participation. The best practice method of evaluating the effect of multiple factors on decision-making is a discrete choice experiment (DCE).²¹

DCEs are used extensively in economics and psychology to elicit individuals’ preferences by presenting a series of pairs of hypothetical options. These may be different treatment options, or in this case, different types of RCT designs. These options differ by several characteristics, referred to as attributes, for example, the type of surgical procedure being tested, or the definition of sham or placebo provided to patients in participant information. The usefulness of a DCE relies heavily on appropriate design, in particular the selection of suitable attributes. The recommended guidelines for the design of DCEs indicate that the selection of attributes should be informed by multiple sources, including literature reviews and focus groups.^22–24

Therefore, this paper outlines a protocol for a scoping review mapping factors that influence patient and surgeon decision-making around sham surgical trials in orthopaedics, which will ultimately be used to inform a DCE to empirically test patient and surgeon preferences for different sham surgery trial designs. In keeping with the methodological recommendations for the conduct of DCEs, a series of qualitative interviews and focus groups with key stakeholders, including orthopaedic patients and surgeons, will then inform the selection of identified attributes to be included in the final DCE. To our knowledge, evidence synthesis around preferences for surgical trials has not been previously published. Therefore, this scoping review will also inform future patient-centred outcomes research around sham surgery trials.

Aims and objectives

Broadly, the objective of this study is to identify and assess the literature reporting on the preferences of surgeons and patients around sham controlled trials in surgery. Specifically, this scoping review will inform an important stage in DCE development, namely the identification and selection of appropriate attributes.^25–27 The scoping review will provide qualitative information regarding a variety of factors that affect trial recruitment. The DCE will then provide easily translatable, quantitative information on the relative importance of each attribute in determining willingness to participate. Beyond informing DCE design, data on patient and surgeon preferences are valuable to decision-makers.^{28 29} For example, preferences for outcomes can be used to inform decision and cost-effectiveness models, while preferences for treatments can inform patient-centred outcomes research and future trial design.

Methods and analysis

Protocol design

We have chosen a scoping review because it is a suitable method for summarising findings from a body of knowledge that is heterogeneous in methods or discipline.³⁰ The framework for our scoping review will be compiled in accordance with Arksey and O’Malley,³¹ Levac et al³² and Peters et al.³³ This requires identifying the research question; identifying relevant studies; study selection; charting of the data; and collating, summarising and reporting the results. To optimise reporting, this scoping review protocol uses the Preferred Reporting Items for Systematic Reviews and Meta-analyses Protocols extension for Scoping Reviews (PRISMA-ScR) checklist.³⁰ The final review will also be reported in accordance with PRISMA-ScR. The five stages of our planned scoping review are detailed below. Title and abstract screening commenced on 26 June 2019 and data charting is to be completed by 31 January 2020.

Stage 1: identifying the research question

The research questions will inform a future DCE, as well as provide important information for surgical trial design, by mapping the available evidence around the factors that affect patient and surgeon recruitment into randomised, sham-controlled, surgical trials. The key questions in this scoping review include

1. What characteristics of surgical trials (eg, trial design, method of randomisation, information provided to participants) affect patient and surgeon decisions to participate in a sham surgical trial?

2. What patient characteristics (eg, level of pain, perceived time costs) affect their decision to participate in a sham surgical trial?

3. What surgeon characteristics (eg, seniority, previous participation in research) affect their decision to participate in a sham surgical trial?

Stage 2: identifying relevant studies

We conducted a systematic search of the Medline, Embase, PsycInfo, Cinahl and EconLit databases from database inception to 21 June (Medline, Embase and PyscInfo) and 23 June (Cinahl and EconLit) 2019. The search strategy was developed around the two key concepts of sham surgery trials, and patient and surgeon attitudes to identify the relevant literature in this area. Subject headings, keywords and keyword phrases were developed for each of the search concepts. The two concepts were then combined using the ‘AND’ operator. The search strategy was developed in consultation with a senior research librarian in Medline before being translated to the other databases. The search was limited to human studies and English-language citations. Some publication types including comments, editorials, letters and news were excluded from the search results. In addition to the databases mentioned, Google Scholar was searched using the relevant keywords informed by the initial Medline strategy. The first 200 results were screened for inclusion. In addition, the reference lists of relevant included studies were reviewed for any additional citations which may not have been captured by the search strategy. After de-duplication, the search identified 2322 studies. Full details of the draft search strategies are shown in the online supplementary appendix.

Stage 3: study selection

The eligibility criteria were decided prior to screening and then finalised through a pilot title and abstract screen. The inclusion and exclusion criteria are shown in table 1. Inclusion criteria will include published primary research studies, reviews and opinion pieces. Exclusion criteria will include any primary research study which does not specifically deal with sham or placebo surgery, or any review or opinion publication which does not specifically address issues related to placebos or participation in RCTs. Studies included in the scoping review will not be limited by year of publication. Studies identified via the search strategy will be exported to EndNote X9.1 and imported to Covidence software³⁴ for the screening of titles and abstracts, and for full-text review. All titles and abstracts will be independently assessed by two study authors using the selection criteria (outlined in table 1), with conflicts resolved by consensus between three authors. Following title and abstract screening, full texts of potentially relevant studies will be independently assessed by two authors, with conflicts resolved by consensus between three authors. Reviewers will communicate regularly throughout the screening processes to support the iterative nature of the scoping review format, with updates to eligibility criteria as required. Any alterations made to the selection criteria as the review progresses will be recorded. The study selection process will be reported using a PRISMA flow chart diagram.

Table 1.

Table of inclusion and exclusion criteria for the title and Abstract, and full text screening

	Inclusion	Exclusion
	Primary research studies
Publication type	Peer-reviewed papers and protocols	Grey literature including theses and books
Population	All adult patients involved in a surgical trial	Children
Intervention	Surgical procedure (skin incision and change to anatomy)	Non-surgical procedure (no skin incision)
Examples	Vertebroplasty; arthroscopy	Radiofrequency; transcutaneous electrical stimulation; ultrasound/microwave therapy; photodynamic therapy; acupuncture
Comparison type	Sham or placebo surgery	Other (not sham surgery)
Examples	General anaesthesia and skin incision only; general anaesthesia, skin incision and insertion of scope only	Device implementation but not activation; other surgical procedure (eg, standard surgery); other non-surgical intervention (eg, physiotherapy)
	Reviews and opinion pieces
Publication type	Peer-reviewed journal articles	Grey literature including news articles, blogs etc
Population	Patients, practitioners, researchers or ethicists	Null
Outcome	Information on factors affecting recruitment in surgical or randomised trials	Information and opinion on ethics of sham surgery or randomised trial designs

Stage 4: data charting

As stated in Arksey and O’Malley³¹ and Tricco et al,³⁰ data charting is the method used for data extraction in scoping reviews. Data charting allows for a descriptive summary of the results to be collated, with amendments made to accommodate additional unforeseen data as extraction progresses. Data will be charted using a standardised electronic form, developed based on the research questions outlined previously. Three authors will pilot the data charting form using the first eight studies included to ensure the data extracted addresses the research questions. This process will also ensure the data charting process is reliable among coauthors. The remainder of the data will be charted by three authors, with data extracted by one checked by a second author. Any discrepancies will be resolved by consensus among authors. The iterative nature of scoping reviews means the data charting form may require adaptation during the data charting process, which will be recorded for inclusion in the study publication.

Details and outcomes from each publication will be chosen guided by both the research questions, as well as recommendations of items to consider in data extraction published by the Cochrane collaboration³⁵ and Arksey and O’Malley.³¹ Where available, these will include bibliographic details, study type, description of dataset used, relevant trial attributes and/or patient/surgeon attributes. Any data that inform the research questions (ie, characteristics of trials, surgeons or patients likely to affect trial recruitment) will be prioritised. Additional categories may be added as necessary during the full-text screening, in consultation with authors. Any other key findings or recommendations not captured through the data charting process, but which are relevant to the research questions, will be included.

All publications included in the scoping review will be critically appraised for quality. Although not an essential part of a scoping review,³⁰ we will undertake this step to generate a level of confidence around the results, as well as to identify any gaps in the literature. Studies will not be excluded on the basis of quality appraisal score. Quantitative studies will be assessed using the Cochrane collaboration’s tool for assessing risk of bias in RCTs.³⁶ Qualitative studies will be assessed using the Consolidated criteria for reporting qualitative studies.³⁷ Critical appraisal will be carried out by two reviewers independently, with discrepancies resolved by consensus among three authors. We recognise that in some cases, an assessment of study quality may be difficult due to the heterogeneous nature of the data sources likely to be included in this review. In the case a systematic assessment of study quality is unable to be applied to a source, we will qualitatively comment on the quality of the reference included.

Stage 5: data summary and synthesis of results

The outcomes of this scoping review will be reported according to the PRISMA Extension for Scoping Reviews (PRISMA-ScR) checklist.³⁰ Charted results will be summarised in the first instance to address the research questions. All data will be thematically explored and tabulated. Results of the quality assessment will be tabulated. Narrative description will also be used to aid interpretation and synthesis of results. The implications of the evidence, and any gaps in existing knowledge, will also be summarised narratively. The key findings from this review will inform the development of attributes and levels for a DCE investigating factors precluding participation in RCTs of orthopaedic procedures.

The inclusion of evidence across a wide range of sources ensures that this review will report a widespread range of attributes relating to trials, patients and surgeons likely to affect recruitment to clinical trials of surgery. The results of this review may be used more broadly to inform future trial design, both within and outside of surgery.

Strengths and limitations

A strength of this scoping review is the breadth and diversity of articles to be included. Previous reviews on factors affecting recruitment of RCTs have only included experimental studies, surveys, questionnaires and interviews. Although these generally provide higher quality data than expert opinion, they are limited in the data they provide by what questions were asked. More general review or opinion pieces are not restricted to focus on only a few key items and thus can explore a greater number of themes that may potentially affect recruitment. As the results of this scoping review are intended to be the primary information in the design of a comprehensive DCE (supplemented by qualitative interviews and a related systematic review), it is important that all possible barriers and enablers to recruitment are included. This will ensure that the DCE is optimally designed.

A limitation of this scoping review is that we will not collect grey literature, unpublished work or work written in languages other than English. There is a risk that some crucial attributes will not have been published in English, in peer-reviewed journals or in any form previously and therefore will not be captured in this review. Further, there is a risk that published opinion pieces will primarily reflect one point of view. However, the supplementary patient and surgeon qualitative interviews should help to address these limitations. These and other limitations identified throughout the review process will be acknowledged in the review publication.

Patient and public involvement

There will be no patient or public involvement in this scoping review.

Ethics and dissemination

The final scoping review will be submitted for publication in a peer-review journal and may also be presented at appropriate forums or conferences. Findings will be used to inform the development of a DCE to investigate patient and surgeon preferences around RCTs in orthopaedic surgery. As a synthesis of published studies and other references, no ethics approval is required for this scoping review. The mentioned qualitative interviews have been approved through St Vincent’s Hospital (Melbourne) HREC (LRR 072/19), registered with the University of Newcastle HREC (H-2019-0222), and governance approval has been provided by St Vincent’s Hospital Melbourne and John Hunter Hospital. There are no data currently available for this paper as it is a protocol; however, the data sourced throughout the scoping review will be made available in an open repository at the time of publication.

Discussion

Sham clinical trials in orthopaedic surgery are becoming more frequent worldwide, and it is of vital importance to address the poor recruitment and retention rates commonly associated with them. At present, sham surgical trials which do not reach their minimum recruitment levels are associated with significant concerns about their validity and applicability, and therefore cannot answer questions about the efficacy of some procedures. To our knowledge, there have been no other similar reviews of attributes affecting recruitment into sham surgery trials. The results of this review will have implications beyond informing our future DCE. The comprehensive inclusion of previous studies, opinions and ethical viewpoints will also benefit other researchers wishing to design DCEs regarding sham surgery, but also more broadly provide a summary of trial, patient and surgeon characteristics relevant to sham surgery participation, which can inform future development of RCTs.