The development of a cryptoglandular Anal Fistula Core Outcome Set (AFCOS): an international Delphi study protocol

Abstract

Purpose

Cryptoglandular anal fistula is a disorder with an incidence of around 1 per 5,000 people per year in European countries. Many studies have been conducted to evaluate the effectiveness of interventions for anal fistula. However, there is considerable heterogeneity in the outcomes assessed and reported in these studies. This limits research quality and complicates evidence synthesis. A solution for heterogeneity in outcome reporting is the development of a Core Outcome Set (COS). This paper describes the protocol for the development of a European COS for Anal Fistula (AFCOS).

Methods

The first step will be a systematic review of the literature to identify potential outcomes that may be included in the COS. Patient interviews will be conducted in The United Kingdom and The Netherlands to ensure that both clinician-important and patient-important outcomes are captured. The outcomes will be categorized using the COMET taxonomy and taken forward to a Delphi consensus exercise. In up to three web-based Delphi surveys the outcomes will be prioritized by patients, clinicians (surgeons, gastroenterologists, and radiologists), and (clinical) researchers. The responses will be summarized and reported anonymously in subsequent round(s) facilitating convergence to a consensus opinion. The final COS will be decided during a face-to-face consensus meeting with patients, clinicians, and (clinical) researchers.

Discussion

This study protocol describes the development of a European COS for anal fistula to improve research quality, evidence synthesis, and patient care.

Introduction

An anal fistula is defined as an abnormal tract connecting the anorectal canal and the perianal skin and is lined with granulation tissue.¹ It is a commonly encountered problem with an incidence of around 1 per 5,000 people per year in European countries.² Men are more likely to be affected than women, with a peak incidence around the age of 40.³ An anorectal abscess can be considered as the acute and an anal fistula as the chronic state of the same disease process.¹ However, not all anorectal abscesses lead to anal fistulas and it remains unclear which patients develop a fistula following an abscess.⁴ Over 90% of the anal fistulas are due to infected anal glands in the anal canal.³ These fistulas are called cryptoglandular anal fistulas. The most commonly reported symptoms are pain and discharge, causing social embarrassment and loss of quality of life.

Treatment goals are to eradicate the fistula tract(s) and to prevent recurrence while maintaining fecal continence. However, complete and lasting cessation of discharge is rarely achieved. Simple fistulas, passing through <30% of the sphincter complex, may be relatively easy to manage by fistulotomy with a low risk of fecal incontinence.⁵ The approach to complex fistulas is more difficult as complex fistulas pass through >30% of the sphincter complex, resulting in greater risk of recurrence and fecal incontinence.⁶ Therefore, there has been a considerable interest in the development of sphincter-sparing repair procedures as fistulotomy in complex fistulas will result in fecal incontinence.⁷

The interventional studies evaluating the effectiveness of interventions for anal fistula show heterogeneity in outcome reporting. In addition, there is considerable variability in outcome definitions, outcome measurement instruments, and timing of outcome assessment. This limits research quality and complicates evidence synthesis.^8–10 The development of a Core Outcome Set (COS), encouraged by the Core Outcome Measures in Effectiveness Trials (COMET) initiative (http://www.comet-initiative.org/), will reduce heterogeneity in outcome reporting and minimize outcome reporting bias.^11–12 A COS represents an agreed minimum standardized set of outcomes that should be assessed and reported in all interventional studies for a specific condition.¹³

This paper describes the protocol for the development of a European COS that can be used in interventional studies for anal fistula to evaluate treatment effectiveness. The aim of the COS is to be as representative as possible for patients, clinicians, and (clinical) researchers across Europe.

Methods

This study will follow a multi-step approach as recommended by the guidelines for COS development published by COMET initiative.¹³ In the first step potential clinician-important and patient-important outcomes will be identified by a systematic review of the literature and patient interviews.¹⁴ These outcomes will in the second step be prioritized by patients, clinicians, and (clinical) researchers in up to three web-based Delphi surveys. The third step is a face-to-face consensus meeting to decide on the definitive COS.

The outcomes which are currently reported in interventional studies for anal fistula are predominantly determined by clinicians and (clinical) researchers and may lack outcomes that are important from the patient’s perspective. To ensure that the outcomes included in the COS are important for patients, clinicians, and (clinical) researchers, all groups will be involved in the development of the COS for anal fistula as recommended by the Core Outcome Set-STAndards for Reporting (COS-STAR).¹⁵ The COMET Checklist for Public Research Partners will be used to optimize the involvement of patients. The Study Management Group (AM, NI, MK, UG, PT, SB and patient representatives from The United Kingdom and The Netherlands) is responsible for the design and conduct of this study.

Step 1: Identification of candidate outcomes

A systematic review of the literature will be performed to identify potential clinician-reported and patient-reported outcomes, including outcome definitions, outcome measurement instruments, and timing of outcome assessment. Patient interviews will be performed to identify any additional outcomes that are relevant to patients but are not captured with the systematic review of the literature.

Systematic review of the literature

A systematic review of the literature will be conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement.^16–17

Search strategy

A broad search strategy will be used to identify studies evaluating medical, surgical, and combined (medical-surgical) interventions for adult patients with anal fistula. The search strategy will be reviewed by an information specialist prior to execution. The databases Embase (Ovid), The Cochrane Library, and Medline (Ovid) will be searched using validated terms for perianal fistula (including synonyms and variants). The search will be limited to studies conducted in human adults aged ≥18 years old published from January 2008 to present.

Study selection

All prospective and retrospective interventional studies with ≥10 participants will be included. In addition, systematic reviews will be included and the systematic review itself and the individual studies will be reviewed to ensure complete capture of outcomes. Studies will be excluded if they report interventions for fistulas that are not perianal or not cryptoglandular of origin. Full-text review will be performed for potentially relevant articles based on the initial screening or articles for which there is insufficient data in title and abstract to assess study eligibility. The study selection process will be done independently by two of the authors (AM, NI) using predefined selection criteria. Any disagreement on study eligibility will be resolved through discussion with recourse to the senior authors (PT, SB) if necessary. Covidence Systematic Review Software (Veritas Health Innovation, Melbourne, Australia, available at https://www.covidence.org/home) will be used to screen the studies.¹⁸ A PRISMA flow diagram will be presented including the number of studies included and excluded with reasons provided for exclusion.

Data extraction

A predefined data extraction form implemented in Microsoft Excel will be used. Extracted data will include: 1) study characteristics, 2) outcomes and outcome definitions provided by the authors, and 3) outcome measurement instruments and timing of outcome assessment. All data will be extracted verbatim by two authors (AM, NI). The extracted data will be discussed to assess consensus and to ensure that all outcomes, outcome definitions, and outcome measurement instruments have been identified. Disagreement will be resolved through discussion with recourse to the senior authors if necessary. It is expected that various definitions and outcome measurement instruments are used in the studies. These definitions and outcome measurement instruments will be grouped together under the specific outcome.

Patient interviews

Semi-structured patient interviews will be conducted to identify any additional outcomes that are relevant to patients but are not captured with the systematic review of the literature. Semi-structured interviews will ensure that certain topics are discussed, allowing patients the flexibility to go into details when needed. During patient interviews information about health status, experiences and expectations regarding treatment, and outcomes relevant from the patients’ perspective will be collected. Patients will be reassured at the start of the interview that they can opt out at any moment during the interview should they feel unable or uncomfortable to continue. Patient interviews will be conducted by telephone and are expected to take between 30 and 60 minutes. Interviews will be audio-recorded, transcribed verbatim, and analyzed for content independently by two authors (AM, NI). Interview transcripts will be fully anonymized, and no patient identifiable information will be shared between institutions. The patient-important outcomes extracted from the interview transcripts will be reviewed by all members of the Study Management Group.

Sampling method patient interviews

Patient interviews will be conducted in The United Kingdom and The Netherlands to ensure that representative views of patients are included. Adult patients aged ≥18 years old with anal fistula will be eligible for inclusion. Purposive sampling will be used, aiming to recruit equal numbers of men and women, with a range of ages and from a variety of ethnic groups. Roughly equal numbers of patients will be recruited from St Mark’s Hospital (The United Kingdom) and Maastricht University Medical Centre+ (The Netherlands) to ensure balanced groups. The estimated sample size for the patient interviews is 20 patients, but saturation will be used as a criterion for discontinuing the patient interviews. Patients will be identified from medical records at both institutions. Eligible patients will receive a copy of the patient information sheet and two informed consent forms from their treating clinician. If the patient agrees to be contacted by the authors (AM, NI) to discuss possible participation, the patient will be contacted by telephone to elaborate on the aim and broad content of the interview, and to answer any questions the patient may have. If the patient agrees to participate, the patient will be asked to return the signed informed consent forms prior to the actual interview. The informed consent forms will be countersigned by the treating clinician. The patients will receive one of the signed informed consent forms by return.

Categorization of candidate outcomes

Clinician-reported and patient-reported outcomes resulting from the systematic review of the literature and patient interviews will be categorized into domains using the COMET taxonomy.¹⁹ The outcomes will be mapped by the Study Management Group to better understand the spread of outcomes but will not necessarily guide the formation of the COS. Disagreement will be resolved through discussion. Next, the outcomes will be formatted into questions and taken forward to the Delphi consensus exercise. To ensure appropriate phrasing and understanding for all stakeholders, outcomes will be translated, and lay definitions will be provided in different languages. The surveys will be pilot tested.

Step 2: Delphi consensus exercise

The Delphi methodology, comprising multiple web-based surveys answered anonymously by patients, clinicians, and (clinical) researchers, will be used to facilitate convergence to a consensus opinion about which outcomes should be included in the COS. The Delphi methodology is advantageous because it comprises anonymous surveys and participants do not interact directly with each other when reviewing their responses. This limits the effects of dominant individuals on individual responses.²⁰ The Delphi consensus exercise will be web-based to increase the feasibility of international sampling.¹³

Sampling method Delphi consensus exercise

Patients with anal fistula are managed by a multidisciplinary team, including surgeons, gastroenterologists (in several European countries) and radiologists. It is expected that they may differ in their views on the relevance of outcomes in evaluating treatment effectiveness. Therefore, stakeholders will be divided into multiple panels to ensure that the views of different stakeholders are equal weighted in the Delphi consensus exercise. The following panels will be generated: 1) patients, 2) clinicians, subdivided into 2a) surgeons, 2b) gastroenterologists, and 2c) radiologists, and 3) (clinical) researchers.¹⁵ There is no consensus on the preferred sample size for a Delphi consensus exercise.¹³ Considering an anticipated response rate of 40% and some attrition between the Delphi rounds, 80 patients, 80 clinicians, and 40 (clinical) researchers will be invited to ensure at least 80 participants per Delphi round. Participants will be contacted by email and provided with information about the study and participant information which underlines voluntary participation, anonymity, and the right to withdraw from the study at any moment. Informed consent to participate in the study will be implied through completion of the initial registration via an embedded link in the email. In the initial registration survey participants will be asked to specify to which stakeholder panel they belong, and demographic data will be captured. Patients will be asked additional information about their health status (e.g. duration of disease and current status).

Patients

Adult patients aged ≥18 years old with a confirmed diagnosis of anal fistula will be eligible for inclusion. Patients from all European countries will be invited by their national representative from the European Society of ColoProctology to participate in the Delphi consensus exercise. In addition, patients will be identified from medical records by their treating clinician and recruited via social media posts made by patient support groups and professional organizations.

Clinicians and (clinical) researchers

Clinicians (surgeons, radiologists, and gastroenterologists) with an interest in anal fistula will be recruited via social media posts and via national and international organizations such as the European Society of ColoProctology. In addition, recently published papers regarding anal fistula will be screened and authors will be contacted to ensure complete capture of experts in this field. Each expert will also be asked to provide the names and contact details of anyone else they believe would be suitable for inclusion in the study.

Delphi consensus rounds

The candidate outcomes resulting from the systematic review of the literature and the patient interviews will be prioritized by the participants in up to three web-based Delphi rounds. The results will be summarized and fed back anonymously after each round to facilitate convergence to a consensus opinion. Stakeholders will be asked to score the candidate outcomes on a 9-point Likert scale with respect to how important they think the outcomes are in evaluating treatment effectiveness. The scale, proposed by the GRADE group (http://www.gradeworkinggroup.org/), labels ‘1–3’ as an outcome of limited importance (not important), ‘4–6’ as important but not critical (fairly important), and ‘7–9’ as critical (really important).²¹ The candidate outcomes will be listed alphabetically to minimize bias. Lay definitions in several languages will be available for all outcomes and will be reviewed by patients included in the Study Management Group. Participants will not be able to progress through the survey unless all questions are answered, but an ‘unable to score’ option will be made available to allow for the fact that some participants may not have the level of expertise to score certain outcomes.¹³ Each Delphi round will be open for a period of 4 weeks. A reminder email will be sent to participants who have not completed the Delphi round after 2 weeks. Another reminder email will be sent if participants have not returned the completed survey after the 3-week deadline to ask whether they are experiencing any difficulties in completing the survey or decided not to participate in the study. The number of participants invited and the number of participants within each stakeholder panel who completed each round will be documented. Participants who register for the survey will be given a unique number to allow the Study Management Group to track attrition. To minimize the attrition rate between the subsequent Delphi rounds, acknowledgement will be offered to the clinicians and (clinical) researchers who completed all rounds.

Delphi round 1

In the first Delphi round stakeholders will be asked to score all candidate outcomes. The survey starts with an open question asking the participant to state up to three outcomes that they feel are most important for evaluating treatment effectiveness. What follows are the candidate outcomes as extracted from the systematic review of the literature and patient interviews. At the end of the survey stakeholders will be given the opportunity to propose additional outcomes they think are missing and to submit any feedback on the survey. Any new outcomes will be reviewed by the Study Management Group to ensure they represent true new outcomes. Proposed outcomes will be taken forward to the second Delphi round for assessment. At the end of the first Delphi round, the Study Management Group will analyze the data. The candidate outcomes will be classified as ‘consensus in’, ‘consensus out’, or ‘no consensus’ (Table 1). Consensus in will be defined as >70% of the participants in each panel rating the outcome ‘7–9’. Outcomes will be classified as consensus out if <70% of the clinicians and (clinical) researchers rate the outcome ‘7–9’ and the patient group rating average was <7. Outcomes rated ‘7–9’ by <70% of the clinicians and (clinical) researchers but with a patient average rating of >7 are considered patient-important and will be classified as consensus in. All other outcomes will be considered as not achieving consensus in either direction. The ‘no consensus’ outcomes will be carried forward to the second Delphi round for another round of feedback.²² Participants who completed the first round will be eligible for entry into the second round.

Table 1.

Definition of consensus.

Consensus classification	Definition	Description
Consensus in	>70% of the participants in each panel rating the outcome ‘7–9’ OR <70% of the clinicians and (clinical) researchers rating the outcome ‘7–9’ but an average patient rating of >7	Important outcome
Consensus out	<70% of the clinicians and (clinical) researchers rating the outcome ‘7–9’ AND an average patient rating of <7	Not important outcome
No consensus	Anything else	Fairly important outcome

Delphi round 2

In the second Delphi round the results from the first round will be summarized and fed back anonymously using a numerical and graphical representation. Participants will be shown their previous responses and the responses from all stakeholder panels.²⁰ This will allow the participants to review their original answers after seeing how others ordered the outcomes. This will facilitate convergence to a consensus opinion between the different stakeholder panels.²³ At the end of the second Delphi round, the Study Management Group will analyze the data using predefined consensus criteria (Table 1). Participants who completed the second round will be eligible for entry in the additional third round and/or face-to-face consensus meeting.

Delphi round 3 (optional)

If many additional outcomes are suggested by participants in the first round, a third round may be needed to allow for these outcomes to be appropriately assessed (i.e. the ability to review the responses from all stakeholders panels to facilitate convergence to a consensus opinion).

Consensus meeting

Following the web-based Delphi rounds, patients, clinicians, and (clinical) researchers will be invited to participate in a face-to-face meeting to establish the final COS. In the first Delphi survey a question will be included about willingness to participate in the consensus meeting. Participants who completed all Delphi rounds will be eligible for participation. Equal numbers of participants will be randomly selected from each stakeholder panel to minimize imbalance in stakeholder representation. Considering practical constraints, it is anticipated that approximately 40 participants will attend the consensus meeting. Attendance via teleconference may be considered, if needed to ensure balance in stakeholder representation. During the consensus meeting, the results of the Delphi rounds will be presented and any disagreement in ‘consensus in’ or ‘consensus out’ can be discussed. In addition, the importance of the ‘no consensus’ outcomes will be discussed. Following discussion, the ‘no consensus’ outcomes will be classified as ‘consensus in’ or ‘consensus out’. Predefined criteria will be used to prevent any bias of the results. To maintain anonymity during the face-to-face meeting an electronic voting system will be used and the scores per voting round will be projected on a screen.

Discussion

Studies evaluating the effectiveness of interventions for anal fistula have used a wide variety of outcomes. The development of a COS for anal fistula important to both patients, clinicians, and (clinical) researchers is a crucial step to be able to progress evidence-based treatment and will make a profound contribution to patient care. This study protocol provides a detailed description of the development of a European COS for anal fistula. Once the COS has been developed, it will be important to determine how and when the outcomes should be measured. Therefore, future work will include the evaluation of potential outcome measurement instruments. This will be done following a consensus-based guideline for the selection of outcome measurement instruments for outcomes included in a COS.²⁴ This is a multi-step process including the assessment of the suitability of the specific measurement instruments using the checklist developed by The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative.²⁵

Trial status

The project is expected to be completed by the end of 2020. The face-to-face consensus meeting will be held during the European Society of Coloproctology’s annual meeting in Vilnius, Lithuania (September 2020) allowing international representation of clinicians.

Trial registration

This project is registered in the Core Outcome Measures in Effectiveness Trials (COMET) database on May 1 2018, available at: http://www.comet-initiative.org/studies/details/1145?result=true. The systematic review is registered in the PROSPERO database (PROSPERO-ID: CRD42018102778) on July 9 2018, available at https://www.crd.york.ac.uk/prospero/#recordDetails.

Author contributions

Astrid J.H.M. Machielsen, B.Sc.: Design of the study; drafting and revision of the article; final approval of the article. Nusrat Iqbal, BSc (Hons), MBBS, MRCS: Design of the study; revision of the article; final approval of the article. Merel L. Kimman, Ph.D.: Design of the study; revision of the article; final approval of the article. Kapil Sahnan, BSc (Hons), MBBS, MRCS: Design of the study; revision of the article; final approval of the article. Samuel O. Adegbola, BSc (Hons), MBBS, MRCS: Design of the study; revision of the article; final approval of the article. Jos Kleijnen, M.D., Ph.D., Prof.: Design of the study; revision of the article; final approval of the article. Carolynne J. Vaizey, MD, MBChB, FRCS, FCS (SA): Design of the study; revision of the article; final approval of the article. Ugo Grossi, MD, Ph.D.: Design of the study; revision of the article; final approval of the article. Phil J. Tozer, FRCS, MD (Res): Conception and design of the study; revision of the article; final approval of the article. Stéphanie O. Breukink, M.D., Ph.D.: Conception and design of the study; revision of the article; final approval of the article.

Declaration of conflicting interests

The authors declare that they have no conflict of interest.

Ethical approval

The ethical committee of Maastricht University Medical Centre+ and Maastricht University confirmed that the Medical Research Involving Human Subjects Act (WMO) does not apply to the above-mentioned study and that an official approval of this study by the committee is not required (METC 2018-0913). HRA and Health and Care Research Wales (HCRW) has given approval for the above referenced study (IRAS project ID 259864).  Informed consent: Informed consent will be obtained from all individual participants included in the study.

Funding

This study is not funded.