The STROCSS statement: Strengthening the Reporting of Cohort Studies in Surgery

Abstract

Introduction

The development of reporting guidelines over the past 20 years represents a major advance in scholarly publishing with recent evidence showing positive impacts. Whilst over 350 reporting guidelines exist, there are few that are specific to surgery. Here we describe the development of the STROCSS guideline (Strengthening the Reporting of Cohort Studies in Surgery).

Methods and analysis

We published our protocol apriori. Current guidelines for case series (PROCESS), cohort studies (STROBE) and randomised controlled trials (CONSORT) were analysed to compile a list of items which were used as baseline material for developing a suitable checklist for surgical cohort guidelines. These were then put forward in a Delphi consensus exercise to an expert panel of 74 surgeons and academics via Google Forms.

Results

The Delphi exercise was completed by 62% (46/74) of the participants. All the items were passed in a single round to create a STROCSS guideline consisting of 17 items.

Conclusion

We present the STROCSS guideline for surgical cohort, cross-sectional and case-control studies consisting of a 17-item checklist. We hope its use will increase the transparency and reporting quality of such studies. This guideline is also suitable for cross-sectional and case control studies. We encourage authors, reviewers, journal editors and publishers to adopt these guidelines.

1. Introduction

Cohort studies have a major investigative role in medical and surgical literature. A cohort study is a comparative study typically involving two or more groups of patients that are sampled on the basis of a specific exposure or intervention [1]. Cohort studies permit the comparison of multiple treatments by conducting a comparative analysis of different cohorts treated with different interventions. They are still relevant today despite the growth in RCTs [2].

Cohort studies are popular in the surgical literature. In 2013 studies of level 2 or 3 evidence, including cohort studies, accounted for 20–55% of published research in the top three journals (by impact factor) across six surgical specialties, a significant rise compared with the 2003 figures of 17–50% [3]. However, work by our group has shown cohort studies to be poorly reported in Plastic Surgery [4].

The development of reporting guidelines over the past 20 years represents a major advance in scholarly publishing. Recent evidence shows guidelines have had a positive impact on reporting transparency. Whilst over 350 reporting guidelines exist there are few that are specific to surgery. The recent development of surgery specific guidelines for case reports and case series has underscored how surgical and procedural interventions require significant additional detail.

Reporting surgical studies in sufficient detail is necessary for readers to have a complete, clear, transparent and reproducible understanding critical for clinical practice to change and healthcare to advance. Here we describe the development of the STROCSS guideline (Strengthening the Reporting of Cohort Studies in Surgery) using a DELPHI consensus exercise.

2. Methods

A similar methodological approach was taken to that used for developing the SCARE and PROCESS guidelines by our group, including Moher et al.’s guidance on developing reporting guidelines [5,6]. Relevant points from the PROCESS [7], STROBE [8] and CONSORT guidelines [9], were used to create a list of applicable items which were used as baseline material in the Delphi rounds. Guidance from articles appraising cohort studies was also considered [10–12].

2.1. The Delphi process

The Delphi questionnaire was administered using Google Forms (https://www.google.co.uk/forms/about/). The questionnaire was conducted using standard Delphi Methodology [13]. The same questionnaire was completed by all stakeholders throughout the process. Participants were invited to recommend adaptations to the draft items and suggest new items important in reporting surgical cohort studies.

Participants rated the importance of reporting each item according to a nine-point Likert scale, where 1 indicates not important, and 9 indicates critically important. This methodology follows the recommendations outlined by the Grading Recommendations, Assessment, Development, and Evaluations (GRADE) working group [14]. Items scoring: 1 to 3 - indicates the item is of little important; 4 to 6 - indicate an item is important but not critical; 7 to 9 - indicates the items is critically important.

Items proceeded to the reporting guideline if they scored between 7 and 9 by 70% of respondents and between 1 and 3 by fewer than 30% of respondents. Likewise, items would not proceed to the reporting guideline if they scored between 1 and 3 by 70% of respondents, and between 7 and 9 by 30% or more of respondents. The entire process will be conducted electronically. There was no predetermined number of Delphi rounds and rounds were terminated when agreement for all items was unanimous.

3. Participant selection

The 59-strong panel of experienced reviewers and researchers who helped develop the SCARE and PROCESS guidelines were invited to participate in developing the STROCSS guidelines in this study. Further participants were drawn from the Editorial Board of the International Journal of Surgery after an open call, leading to a total of 74 participating doctors on the panel. The 74 participants represented 25 countries across North and South America, Europe, Africa, Asia and Australasia. Participants represented ten surgical specialties as well as allied specialties including; dermatology, pathology, oncology, clinical pharmacology, acute care surgery with many participants being experienced researchers, authors, journal reviewers, editorial board members and editors.

4. Results

In a single round over a 10-day period, 62% (46/74) of the panel completed the DELPHI and consensus was achieved for all 17 items (Table 1). There was over 70% agreement between respondents for all but one item (4c), which achieved 67% agreement on its inclusion. It was accepted given it was so close to the threshold set in our protocol and only 4% scored it 1–3 on the Likert scale.

Table 1.

The STROCSS items and the level of agreement achieved.

Item no.	Item description	Agreement (% scoring 7–9 on Likert scale)
1	Title. The words “cohort” and the area of focus should appear in the title (e.g. disease, exposure/intervention or outcome). Whether the study is retrospective or prospective should also be stated.	91% (42/46)
2a	Abstract - Introduction What is the background and scientific rationale for the research question.	100% (46/46)
2b	Abstract - Methods - Describe the study design (cohort design, retrospective or prospective, single or multi-centre, etc), what was done to each group, how, when was it done and by whom.	96% (44/46)
2c	Abstract - Results What was found. Give the results for the main outcomes.	98% (45/46)
2d	Abstract - Conclusion - What have we learned and what does it mean. Where should future research go.	94% (43/46)
3	Explain the scientific background and rationale for the cohort study. What are objectives, research questions and the hypotheses.	98% (45/46)
4a	Registration and ethics State the research registry number in accordance with the declaration of Helsinki - “Every research study involving human subjects must be registered in a publicly accessible database before recruitment of the first subject" (this can be obtained from; ResearchRegistry.com or ClinicalTrials. gov or ISRCTN). Even retrospective studies should be registered prior to submission.	80% (37/46)
4b	Ethical Approval - State whether ethical approval was needed and if so, what the relevant judgement reference from the IRB or local ethics committee was? If ethical approval was not needed, state why.	96% (44/46)
4c	Protocol - Was a research protocol developed apriori? Where can it be accessed. Was it published in a journal e.g. IJS Protocols, BMJ Open, etc, if so, provide the reference.	67% (31/46)
5a	Study design - State the research is a cohort study and whether prospective or retrospective in design, whether single or multi-centre.	98% (45/46)
5b	Setting - Describe the setting(s)and nature of the institution in which the patient was managed; academic, community or private practice setting? Location(s), and relevant dates, including periods of recruitment, exposure, follow-up, and data collection	91% (42/46)
5c	Cohort Groups - State the number of groups in the study. What interventions will each group receive?	96% (44/46)
5d	Sub-group – Analysis. Any planned sub-group analyses are specified/Describe any methods used to examine subgroups and interactions.	94% (43/46)
6a	Participants - State any eligibility (inclusion/exclusion) criteria and the sources and methods of selection of participants. Describe length and methods of follow-up.	94% (43/46)
6b	Recruitment - State the methods of how patients or participants were recruited to each group, over what time periods.	96% (44/46)
6c	Sample size calculation Whether there was calculation of margin of error or a prior analysis to determine study population, or mention of how appropriate study sample was determined.	80% (37/46)
7a	Pre-intervention considerations - e.g. Patient optimisation: measures taken prior to surgery or other intervention e.g. treating hypothermia/hypovolaemia/hypotension in burns patients, ICU care for sepsis, dealing with anticoagulation/other medications and so on.	87% (40/46)
7b	Types of intervention(s) deployed - To include reasoning behind treatment offered (pharmacological, surgical, physiotherapy, psychological, preventive) and concurrent treatments (antibiotics, analgesia, anti-emetics, nil by mouth, VTE prophylaxis, etc). Medical devices should have manufacturer and model specifically mentioned.	85% (39/46)
7c	Peri-intervention considerations - Administration of intervention (what, where, when and how was it done, including details for surgery; anaesthesia, patient position, use of tourniquet and other relevant equipment, preparation used, sutures, devices, surgical stage (1 or 2 stage, etc) and operative time. Pharmacological therapies should include formulation, dosage, strength, route and duration). Authors are encouraged to use figures, diagrams, photos, video and other multimedia to explain their intervention.	87% (40/46)
7d	Who performed the procedure(s) - Operator experience for each group (position on the learning curve for the technique if established, specialisation and prior relevant training).	87% (40/46)
7e	Quality control - What measures were taken to reduce inter or intra-operator variation. What measures were taken to ensure quality and consistency in the delivery of the intervention e.g. independent observers, lymph node counts, etc	91% (42/46)
7f	Post-intervention considerations - e.g. post-operative instructions and place of care. Important follow-up measures - diagnostic and other test results. Future surveillance requirements - e.g. imaging surveillance of endovascular aneurysm repair (EVAR) or clinical exam/ultrasound of regional lymph nodes for skin cancer.	83% (38/46)
8	Outcomes - What primary and secondary (if any) outcomes will be assessed and how are they defined. Definitions should be clear and precise. Appropriate references to validation of outcome measures used should be provided if they exist.	98% (45/46)
9	Statistical methods - Clearly outlined statistical tests used to compare the outcomes between an intervention group and a comparison group, state whether pre-existing differences and known confounders were controlled.	96% (44/46)
10a	Participants recruited with a flow diagram - Report numbers involved in each group and use a flow diagram to show recruitment, non-participation, cross-over, withdrawal from the study with reasons.	94% (43/46)
10b	Comparison between groups including a table - Provide a table comparing the demographic, clinical/prognostic features (co- morbidities, tumour staging, smoking status, etc) and relevant socioeconomic characteristics of each group and whether numerical differences are significant (using p-values and/or confidence intervals as appropriate). Were the groups matched and if so, how.	94% (43/46)
10c	Changes - Any changes in the interventions during the course of the study (how has it evolved, been altered or tinkered with, what learning occurred, etc) together with rationale and a diagram if appropriate. Degree of novelty for a surgical technique/device should be mentioned and a comment on learning curves should be made for new techniques/devices.	96% (44/46)
11a	Outcomes and follow-up - Clinician assessed and patient- reported outcomes (when appropriate) should be stated for each group (size of effect with raw numbers and percentages) with inclusion of the time periods at which assessed. Relevant photographs/radiological images should be provided e.g. 12- month follow-up. Make it clear which confounders were adjusted for and which were not.	94% (43/46)
11b	Intervention adherence/compliance and tolerability - How was this assessed. Describe loss to follow-up (express as a percentage and a fraction) or cross-over between group and any explanations for them.	96% (44/46)
11c	Complications and adverse or unanticipated events - Described in detail and ideally categorised in accordance with the Clavien-Dindo Classification. How they were prevented, mitigated, diagnosed and managed. Blood loss, wound complications, re- exploration/revision surgery, 30-day post-op and long-term morbidity/mortality may need to be specified.	98% (45/46)
12	Summarise key results	98% (45/46)
13	Discussion of the relevance of the findings and rationale for conclusions - Relevant literature, implications for clinical practice guidelines, how have the indications for a new technique/device been refined and how do outcomes compare with established therapies and the prevailing gold standard should one exist and any relevant hypothesis generation. The rationale for any conclusions.	96% (44/46)
14	Strengths and limitations of the study	98% (45/46)
15	State what needs to be done next, further research with what study design(s).	89% (41/46)
16	State the key conclusions from the study and key directions for future research	96% (44/46)
17a	State any conflicts of interest	100% (46/46)
17b	State any sources of funding	98% (45/46)

The majority of participants had no other additions to the above criteria. One participant suggested that the statistical package used should be stated. Another participant stated that reasons for non-participation during the recruitment process should also be documented. We think these are reasonable as minor additions and have added them to the final criteria (items 9 and 6b respectively).

4.1. STROCSS guideline

Table 1 constitutes the STROCSS guideline and this is provided again in an appendix, together with a column in which the author can state the page number on which the criterion was met. All authors submitting surgical cohort studies should include a completed STROCSS checklist with their manuscript, allowing review by journal reviewers and editors. In addition, they should explicitly state in their manuscript that they are reporting in line with the STROCSS guideline, which they can cite. Whilst we set out to utilise these guidelines purely for Cohort studies, like the STROBE guideline, STROCSS guideline can apply to cross-sectional and case-control studies as well, because of their similar observational methodology. It is also important to underscore the point that the guideline represents the minimum detail that should be reported. If something wasn’t done, it should be stated, to aid transparency.

4.2. Endorsement

The STROCSS guideline has been endorsed by the IJS Publishing Group journals which include:

• International Journal of Surgery (IJS)

• IJS Case Reports

• IJS Open

• IJS Protocols

• IJS Oncology

• IJS Short Reports

• Annals of Medicine and Surgery

A dedicated website has been formed to facilitate distribution of the guideline: www.strocssguideline.com. The authors hope that more journals will endorse the guideline in due course and incorporate it as part of routine manuscript submission for cohort, cross-sectional and case-control studies.

5. Conclusion

Here we present the STROCSS Guideline to the international surgical and academic community. It follows a rigorous DELPHI consensus exercise amongst experts from a broad cross-section of the field of surgery internationally. We look forward to feedback from the wider community as well as studies of its implementation and effectiveness as a tool to improve reporting quality and to help inform any future revision.

HIGHLIGHTS.

• There are currently no guidelines specific to surgical cohort studies.

• A Delphi consensus exercise was put to an expert panel of 74 surgeons and academics.

• 17 items were agreed critical to adequately report surgical cohort studies.