Development of ‘Core Outcome Sets’ for Meningioma in Clinical Studies (The COSMIC Project): protocol for two systematic literature reviews, eDelphi surveys and online consensus meetings

Abstract

Introduction

Meningioma is the most common primary intracranial tumour in adults. The majority are non-malignant, but a proportion behave more aggressively. Incidental/minimally symptomatic meningioma are often managed by serial imaging. Symptomatic meningioma, those that threaten neurovascular structures, or demonstrate radiological growth, are usually resected as first-line management strategy. For patients in poor clinical condition, or with inoperable, residual or recurrent disease, radiotherapy is often used as primary or adjuvant treatment. Effective pharmacotherapy treatments do not currently exist. There is heterogeneity in the outcomes measured and reported in meningioma clinical studies. Two ‘Core Outcome Sets’ (COS) will be developed: (COSMIC: Intervention) for use in meningioma clinical effectiveness trials and (COSMIC: Observation) for use in clinical studies of incidental/untreated meningioma.

Methods and analysis

Two systematic literature reviews and trial registry searches will identify outcomes measured and reported in published and ongoing (1) meningioma clinical effectiveness trials, and (2) clinical studies of incidental/untreated meningioma. Outcomes include those that are clinician reported, patient reported, caregiver reported and based on objective tests (eg, neurocognitive tests), as well as measures of progression and survival. Outcomes will be deduplicated and categorised to generate two long lists. The two long lists will be prioritised through two, two-round, international, modified eDelphi surveys including patients with meningioma, healthcare professionals, researchers and those in caring/supporting roles. The two final COS will be ratified through two 1-day online consensus meetings, with representation from all stakeholder groups.

Ethics and dissemination

Institutional review board (University of Liverpool) approval was obtained for the conduct of this study. Participant eConsent will be obtained prior to participation in the eDelphi surveys and consensus meetings. The two systematic literature reviews and two final COS will be published and freely available.

Trial registration number

COMET study ID 1508

Strengths and limitations of this study.

• Comprehensive systematic literature reviews and trial registry searches will identify outcomes measured and reported in published and ongoing intracranial meningioma clinical effectiveness trials and clinical studies of incidental/untreated intracranial meningioma (reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines).

• Unique outcomes (classified according to the Core Outcome Measures in Effectiveness Trials taxonomy) will be prioritised through consensus methodology including two, two-round, international, multistakeholder, modified eDelphi surveys, followed by two online consensus meetings, to ratify the COSMIC: Intervention and COSMIC: Observation Core Outcome Sets (COS).

• COS for intracranial meningioma do not exist but are urgently needed to ensure core outcomes relevant to patients with meningioma, healthcare professionals, researchers and other key stakeholders are measured and reported in future meningioma clinical trials and studies.

• ‘How’ and ‘when’ each core outcome is measured is beyond the scope of this work and will be the focus of future research.

Introduction

Meningioma is the most common primary intracranial tumour accounting for approximately 38% of all primary tumours of the central nervous system, with an estimated age-adjusted incidence of 8.8 per 100 000 population per year.¹ Meningioma are more than twice as common in females (12 vs 5.3 per 100 000 population), although boys are more likely to be affected than girls.¹ Median age at diagnosis is 66 years, and incidence increases with age.¹ The WHO classification of tumours of the central nervous system describes three grades of meningioma, with the most recent distribution by grade as follows: 80.4% benign (WHO grade 1), 17.9% atypical (WHO grade 2) and 1.6% malignant (WHO grade 3).^{1 2} All meningioma have a long-term risk of recurrence, as well as progression to a higher tumour grade.

For symptomatic meningioma, those that threaten neurovascular structures or demonstrate growth on interval imaging, a treatment intervention is warranted. Surgical resection is often the preferred first-line management strategy; however, for poor surgical candidates, patients with inoperable, residual or recurrent disease, radiotherapy may be used as primary or adjuvant treatment to obtain disease control. Despite studies investigating different agents, there are no effective pharmacotherapy treatments.^{3 4} On the other hand, incidental intracranial meningioma may never require treatment. International consensus guidelines recommend interval MRI monitoring; however, details surrounding the intervals and duration of follow-up, and indications for treatment are lacking.⁵ A very low percentage of patients with an incidental intracranial meningioma develop symptoms during follow-up; 0%–8%, however, the risk of growth has been reported to be between 10% and 70%.^{6 7} This heterogeneity in imaging behaviour leads to management decisions recommended to patients varying between active long-term MRI and clinical monitoring or upfront treatment with surgery or radiotherapy.⁸

Intracranial meningioma clinical effectiveness trials

Clinical effectiveness trials for intracranial meningioma are sparse, but important research questions remain to be answered, especially for recurrent and high-grade meningioma. Two phase II studies investigating the efficacy of adjuvant radiotherapy following surgical resection of high-grade meningioma have been reported: Radiation Therapy Oncology Group 0539⁹ and the European Organisation for Research and Treatment of Cancer (EORTC) 22042,¹⁰ as well as a phase II trial of trabectedin for recurrent grade 2/3 meningioma.¹¹ There are currently two phase III randomised controlled trials underway to establish the role of radiotherapy after gross total resection of WHO grade 2 meningioma: Radiation versus Observation following surgical resection of Atypical Meningioma (ROAM)/EORTC 1308¹² and NRG-BN003,¹³ as well as a phase II trial of vismodegib, the focal adhesion kinase inhibitor GSK2256098, capivasertib, and abemaciclib for progressive meningioma.¹⁴ There are other clinical effectiveness trials in development, such as Surgeons Trial of Prophylaxis for Epilepsy in seizure-naïve Patients with Meningioma: a randomised controlled trial (STOP'EM), which will aim to establish the role of prophylactic levetiracetam in seizure-naïve patients undergoing resection of meningioma.¹⁵ However, the outcomes measured and reported in meningioma clinical effectiveness trials are not standardised.

Clinical studies of incidental and untreated intracranial meningioma

Clinical studies of incidental and untreated intracranial meningioma are rare. Recent work has attempted to accurately define risk factors for untreated meningioma growth. The Asan Intracranial Meningioma Scoring System and the Incidental Meningioma: Prognostic Analysis Using Patient Comorbidity and MRI Tests calculator stratify patients based on the imaging features of a meningioma into risk groups.^{16 17} Although both scoring systems require validation with external data sets, they underpin clinical equipoise in patients with an untreated meningioma and pave the way for prospective clinical studies. Patients with meningioma at high risk of progression, for example, ≥3 cm, T2 hyperintense and with peritumoral signal change (indicative of vasogenic oedema), are likely to benefit from an intervention trial, whereas patients with low-risk or medium-risk meningiomas may draw more benefits from trials that compare different monitoring strategies. Similarly, the outcomes measured and reported in clinical studies of incidental and untreated intracranial meningioma are also not standardised.

Rationale for the development of core outcome sets for intracranial meningioma

Interest in meningioma is increasing in part due to the ‘meningiomics’ revolution, which offers the prospect of treatment arm stratification by molecular and genomic aberration, and the potential for personalised management options.^{14 18} With this comes the difficulty of recruitment of a sufficient number of patients into treatment arms, for instance, when stratification is by a single-point mutation present in only 5%–10% of what is already a rare disease. For this reason, future meningioma clinical effectiveness trials will need to be (1) global, multi-institutional efforts, (2) allow meaningful comparison across studies in order to determine comparative efficacy, (3) measure the outcomes that are important to all stakeholders including patients with meningioma, and (4) resourceful and not performed in duplicate, or near duplicate with different outcomes measured and reported for similar research questions. The development of a core outcome set (COS) for meningioma to be used in future clinical effectiveness trials will enable the alignment of these aims.

There is also increasing interest in asymptomatic patients with an incidentally discovered intracranial meningioma, who may never require treatment. The balance between observation and intervention, and the benefit versus harm of each strategy, is not yet clear. Future prospective clinical studies could benefit from the implementation of a COS that is specific to this patient group, in recognition of the specific outcomes that are likely to be considered core.

A COS is defined as the minimum set of outcomes that should be measured and reported in all clinical trials for a specific health condition/health area.¹⁹ To date, over 400 COS have been developed, and over 300 in progress, and they are increasingly recognised as critical to the design of clinical research.²⁰ No COS have been developed within the field of neuro-oncology thus far.²¹ The aim of this project is to develop two COS for intracranial meningioma; one for clinical effectiveness trials (COSMIC: Intervention) and one for observational studies (COSMIC: Observation). This novel methodological approach has been chosen because meningioma is a highly heterogeneous disease, and we assume that the outcomes likely to be considered core by key stakeholders will be somewhat different for a COS developed for interventional in comparison to observational studies. This protocol describes the development of both COS. The COS should be appropriately used in future intracranial meningioma clinical effectiveness trials across the breath of interventions being tested, and future clinical studies of incidental and untreated intracranial meningioma.

The specific objectives of this project are as follows:

• Identify outcomes reported in ongoing and published meningioma clinical effectiveness trials and clinical studies of incidental and untreated intracranial meningioma through trial registry searches and two systematic literature reviews.

• Recruit patients with an intracranial meningioma, healthcare professionals, researchers and other key stakeholders in caring or supporting roles to one of, or both, two-round, international, modified eDelphi surveys to reduce long lists of potentially relevant unique outcomes.

• Conduct two independent, 1-day, international, multistakeholder, online consensus meetings to ratify the COSMIC: Intervention and COSMIC: Observation COS.

• Make freely available and disseminate widely a COS for use in all future intracranial meningioma clinical effectiveness trials, and a COS for use in all future clinical studies of incidental and untreated intracranial meningioma.

Scope of the COS

For a COS to be selected and used by clinical trialists, the scope must be clear (research or practice setting(s) in which the COS is to be applied, and the health condition(s), populations(s) and intervention(s) covered by the COS). A COS with a broad scope may lack relevance for heterogeneous disease entities, but if scope is too narrow, it may never be used. Core Outcome Set-Standards for Development (COS-STAD) recommendations have been described; the product of an international consensus process involving experienced COS developers.²² The purpose of these 11 minimum standards is to facilitate COS development by providing a framework to consider when project planning.

Such is the importance of scope for the successful development and uptake of a COS, that The COSMIC Project encompasses the development of two distinct COS for the same health condition. The scope of both COS is defined within the 11 minimum COS-STAD recommendations (table 1). In summary, the COSMIC: Intervention COS will be developed for use in phase II and later, intracranial meningioma clinical effectiveness trials in adults, which are designed to inform clinical decision-making and improve clinical care for patients. The COS will be applicable to all interventions used to treat the disease including surgical resection, radiotherapy, stereotactic radiosurgery, pharmacotherapy, perioperative care and supportive treatments, any of which may be used in isolation or in combination. Conversely, the COSMIC: Observation COS will be developed for use in observational clinical studies of incidental and untreated intracranial meningioma that are designed to inform monitoring and decision to treat strategies.

Table 1.

Core Outcome Set-Standards for Development recommendations as applied to both COSMIC: Intervention and COSMIC: Observation COS

Domain	Standard	Methodology	COSMIC: Intervention	COSMIC: Observation
Scope specification	1	The research or practice setting(s) in which the COS is to be applied.	Later phase clinical effectiveness trials that will inform clinical decision-making.	Clinical studies of incidental and untreated intracranial meningioma that will inform clinical decision-making.
	2	The health condition(s) covered by the COS.	Sporadic intracranial meningioma requiring intervention, including multiple meningioma and those with SMARCE1-related familial meningioma, but excluding NF2-associated meningioma.	Incidental and untreated intracranial meningioma (including those which are minimally symptomatic).
	3	The population(s) covered by the COS.	Human adults aged 18 or above.	Human adults aged 18 or above.
	4	The intervention(s) covered by the COS.	Interventions including surgical resection, radiotherapy, stereotactic radiosurgery, chemotherapy, perioperative care and supportive treatments; any of which may be in isolation or in combination with each other.	Active monitoring only as an intervention, but not treatment for an intracranial meningioma.
Stakeholders involved	5	Those who will use the COS in research.	Clinical trialists who manage patients with intracranial meningioma. They are included in standard 6.	Clinical trialists who manage patients with intracranial meningioma. They are included in standard 6.
	6	Healthcare professionals with experience of patients with the condition.	This will include clinicians from multiple subspecialties and non-clinician healthcare professionals with active involvement in the care of patients with intracranial meningioma.	This will include clinicians from multiple subspecialties and non-clinician healthcare professionals with active involvement in the care of patients with intracranial meningioma.
	7	Patients with the condition or their representatives.	Patients with a diagnosis of intracranial meningioma who have received treatment will be included, along with relatives and carers of such patients.	Patients with a diagnosis of incidental intracranial meningioma who have not received treatment will be included, along with relatives and carers of such patients.
Consensus process	8	The initial list of outcomes considered both healthcare professionals and patients’ views.	A trial registry search and systematic literature review of intracranial meningioma trial outcomes will consider healthcare professionals’ views, while patient research partner input and published semistructured interviews with patients will consider patients’ views.	A trial registry search and systematic literature review of clinical studies of incidental and untreated intracranial meningioma will consider healthcare professionals’ views, while patient research partner input and published semistructured interviews with patients will consider patients’ views.
	9	A scoring process and consensus definition were described a priori.	Described in the ‘Scoring’ and ‘Analysis’ sections of this protocol.	Described in the ‘Scoring’ and ‘Analysis’ sections of this protocol.
	10	Criteria for including/dropping/adding outcomes were described a priori.	Described in the ‘Analysis’ section of this protocol.	Described in the ‘Analysis’ section of this protocol.
	11	Care was taken to avoid ambiguity of language used in the list of outcomes.	Both study content and study materials will use plain language summaries and clinical explanations where necessary. All materials will be reviewed with patient research partners and pilot tested with patients and healthcare professionals.	Both study content and study materials will use plain language summaries and clinical explanations where necessary. All materials will be reviewed with patient research partners and pilot tested with patients and healthcare professionals.

COS, core outcome set; NF2, neurofibromatosis type 2.

Registration

The study is registered with the Core Outcome Measures in Effectiveness Trials (COMET) database as study 1508 accessible at https://www.comet-initiative.org/Studies/Details/1508. This study has University of Liverpool sponsorship and ethical approval (Ref UoL001601).

Methods and analysis

Development of the COSMIC: Intervention and COSMIC: Observation COS consists of two distinct phases. Phase I concerns the generation of two long lists of unique outcomes that are potentially relevant to key stakeholders. The long lists will be generated by extracting outcomes measured and reported in ongoing and published intracranial meningioma clinical effectiveness trials and clinical studies of incidental and untreated intracranial meningioma. Ongoing studies will be identified from trial registry searches while published studies will be identified from systematic reviews of the literature. Phase I commenced after protocol manuscript submission in September 2021 and will be complete by May 2022.

Phase II concerns the prioritisation of the unique outcome long lists developed in phase I. Two 2-round, international, multistakeholder eDelphi surveys will be administered to achieve a degree of consensus on which unique outcomes should be included or excluded in the final COSMIC: Intervention and COSMIC: Observation COS, according to preset criteria. Those unique outcomes without a conclusive consensus decision will be discussed in one of two online consensus meetings to ratify the final COSMIC: Intervention and COSMIC: Observation COS. A study flow chart summarises the key steps of this project (figure 1). Phase II will commence in May 2022 and will be complete by October 2022.

Figure 1.

A flow chart summarising the five stages of The COSMIC Project. COS, core outcome set.

Phase I: generation of two long lists of outcomes of potential relevance to key stakeholder groups

The purpose of phase I is to generate two long lists of unique outcomes measured and reported in ongoing and published intracranial meningioma clinical effectiveness trials and clinical studies of incidental and untreated intracranial meningioma. The long lists will consist of unique outcomes that are of potential relevance to key stakeholders.

Trial registry search and systematic review of the literature to identify ongoing and published intracranial meningioma clinical effectiveness trials

Research question

What outcomes are measured and reported in ongoing and published clinical trials assessing the effectiveness of interventions, including surgery, radiotherapy, stereotactic radiosurgery, pharmacotherapy, perioperative care and supportive treatments used in isolation or in combination for adult intracranial meningioma?

Types of studies

This systematic review of the literature will identify published full texts that describe phase II, III and IV clinical trials (including single-arm studies) that assess the effectiveness of an intervention for patients with an intracranial meningioma. Articles will be required to describe trial results and have a minimum of 20 adult patients with intracranial meningioma. If multiple publications exist in relation to an individual study, for example, an interim analysis of a clinical trial, final results, as well as an additional prognostic paper, then the publications will be considered together as one study, and repetition of data extraction would not be performed. Online international trial registries will be searched to identify ongoing trials. Only online trial registry entries and published trials written in the English language will be included due to resource limitations.

Types of interventions

Eligible interventions include the full breadth investigated in intracranial meningioma clinical effectiveness trials. Broadly speaking, this will include surgical interventions (including modified techniques, approaches and adjuncts), fractionated radiotherapy (in any form including conformal three-dimensional and intensity-modulated radiotherapy), stereotactic radiosurgery (single fraction, hypofractionated or fractionated), pharmacotherapy (whereby the investigators include outcomes related to the effectiveness of the drug, and not simply the tolerability of the drug), perioperative care (including medical therapies, anaesthetic considerations and general aspects of the care of patients with intracranial meningioma in and around the time of treatment) and supportive treatments (eg, neurorehabilitation and ongoing medical therapies for symptom control). Studies will be included if they investigate an intervention in isolation or in any combination, for example, surgical resection plus a specific radiotherapy and/or chemotherapy regime.

Types of participants

Participants will comprise adults (18 years and above) of either sex, with a diagnosis of sporadic intracranial meningioma, including multiple meningioma and SMARCE1-related familial meningioma. The diagnosis need not be made histopathologically, as participants may potentially be recruited into trials based on a radiological diagnosis of an intracranial meningioma.

Exclusion criteria

We will not include clinical efficacy studies, or studies of a purely experimental nature, for example, exploratory studies to identify biomarkers. Some studies identify themselves as combined phases, for instance, phase 0/II, phase I/II. These studies will be evaluated and discussed between members of the study management group (SMG) to establish where the focus of the work sits. Studies with a primarily phase 0 or I component will be excluded. Case studies and case series (with fewer than 20 participants) will be excluded. Meningioma within the spinal column are outside the scope of The COSMIC Project and will be excluded. Studies investigating meningioma secondary to radiation (eg, administered in childhood as an intervention for cancer) will not be included as this is considered a different disease entity. Similarly, studies investigating meningioma in cohorts of patients with the genetic condition neurofibromatosis type 2 (NF2) would not be included as, again, this is also considered to be a different disease entity, given the predisposition to manifest at an early age, meningioma and schwannoma of the cranial and peripheral nerves, ependymoma, astrocytoma, as well as skin and ocular findings. To include studies of spinal column, radiation-induced and NF2-associated meningioma would identify outcomes that are likely to be of no/limited relevance to key stakeholders contributing to a consensus study to establish a COS for sporadic intracranial meningioma. Studies with a mix of brain tumour types and at least 20 patients with an intracranial meningioma would be included, however, on the assumption that the outcomes measured and reported could be of relevance to key stakeholders.

Search strategy

A detailed search strategy using the search strings ‘meningioma’ AND ‘trial’ has been developed and translated to interrogate the following electronic bibliographic databases: PubMed, EMBASE, MEDLINE, CINAHL via EBSCO and Web of Science. In addition, simple searches of the following trial registries will be conducted: Cochrane Central Register of Controlled Trials, ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform. The search strategies are provided in online supplemental appendix 1. Prior to completing the review, the searches will be rerun to identify new records published since the original search.

Identification of eligible studies

Search results will be downloaded from their respective online databases, and uploaded to the online platform Rayyan.²³ Following deduplication, two review authors (CPM and SMK) will independently screen all titles and abstracts retrieved, according to the inclusion and exclusion criteria. Screening will be performed on the Rayyan platform independently, with each review author blinded to the screening choice of the other. Full-text copies of all titles which appear to meet the inclusion criteria will be obtained, but also titles where a decision cannot be confidently made based on title and abstract alone. The same two review authors will independently screen all full-text copies to assess for eligibility. A lack of agreement at screening or full-text eligibility check will initially be discussed between the two review authors and if agreement is not reached, the issue will be escalated to the senior review author (MDJ). The complete reference list of full-text titles included will be screened to identify titles not identified through the searches. In addition, trial registries will be searched independently by the same two reviewers implementing the same procedures to identify ongoing studies not yet published which describe outcomes that will be reported.

Trial registry search and systematic review of the literature to identify ongoing and published clinical studies of incidental and untreated intracranial meningioma

Research question

What outcomes are measured and reported in ongoing and published clinical studies describing cohorts of adults with incidental and untreated intracranial meningioma?

Types of studies

This systematic review of the literature will identify published full texts that describe any cohort of adults with incidental and untreated intracranial meningioma, with a minimum of 20 patients. Studies are likely to be observational in design. Again, multiple publications relating to one study cohort will be considered together and online international trial registries will be searched to identify ongoing studies.

Types of interventions

This systematic review is concerned with patients who have not received a treatment intervention, but have undergone active monitoring of an incidental intracranial meningioma. Studies will be included if they present outcomes for patient cohorts who have received a radiological diagnosis of an intracranial meningioma, but no treatment intervention. For the purposes of this review, active monitoring is therefore considered to be an intervention and may include clinical review (including history and clinical examination), testing (for instance, to obtain patient-reported, caregiver-reported or performance outcomes) and imaging (using any modality and with any frequency).

Types of participants

Participants will comprise adults (18 years and above) of either sex, with a radiological diagnosis of an incidental intracranial meningioma or an untreated intracranial meningioma (asymptomatic or minimally symptomatic), including both patients with multiple meningiomas and SMARCE1-related familial meningioma.

Exclusion criteria

Clinical studies with fewer than 20 participants will be excluded. Again, studies investigating spinal column, radiation-induced and NF2-associated meningioma are outside the scope of this review and The COSMIC Project, and will be excluded.

Search strategy

A detailed search strategy using the search strings ‘meningioma’ AND ‘incidental’ OR ‘untreated’ has been developed and again translated to interrogate bibliographic databases, as well as simple searches for trial registries. A rerun of searches will again be performed. These search strategies are also provided in online supplemental appendix 1.

Identification of eligible studies

Again, search results will be processed in the online platform Rayyan.²³ Following deduplication, two review authors (CPM and AII) will independently screen all titles and abstracts retrieved, according to the inclusion and exclusion criteria. Full-text evaluation for eligibility, screening for additional titles and trial registry searches will be performed according to the aforementioned methodology.

Extraction of outcomes measured and reported in ongoing and published clinical studies

Definition of an outcome

A trial or study outcome is a measurable variable examined in response to a treatment or intervention. Active monitoring of an intracranial meningioma shall be considered an intervention. For the purposes of this study, a trial outcome will be defined as ‘one that has original meaning and context’ and so different phrasing or spelling of a word, or an idea that addresses the same concept will be categorised as one outcome.²⁴ This study is therefore concerned with any measured and reported variable (trial/study outcome) that attempts to assess response or condition. The US Food and Drug Administration describes four types of clinical outcome assessment that may be reported, namely patient-reported (eg, health-related quality of life), clinician-reported (eg, adverse events), observer-reported (eg, input from informal caregivers on activities of daily living) or performance outcomes (eg, neuropsychological tests). Other traditional outcomes of relevance include those that relate to progression (and its measurement) and survival.

Data extraction

Data will be extracted from eligible articles and trial registry entries by a single review author (CPM) into one of two custom-designed and piloted spreadsheets in Microsoft Excel (V.16.34, Microsoft, Washington, DC, USA) following best practice described by COMET.^{19 20} The first 10% of included titles will be dual extracted by a second review author to assess for consistency and accuracy of extraction (AII). If differences exist in the verbatim outcomes extracted by the two review authors, this will be discussed, resolved and a further 10% will be extracted until concordance is established. For the purposes of this study, concordance is defined as less than 5% difference between both review authors. If disagreements cannot be resolved, these will be escalated to the senior review author (MDJ).

The following data will be extracted from each study as recommended by COMET^{19 20}: study type, study population, first author, year and journal of publication, intervention(s) under investigation, each outcome reported (recorded verbatim) from the study abstract, methods or results, the definition of the outcome, whether outcome is a primary or secondary outcome, the indicator and/or tool(s) used to operationalise or measure the outcome and the time points or time period at which the outcome was measured. The number of verbatim outcomes per trial/study will be recorded.

Matching outcomes that have been measured at multiple time points will not be recorded as different outcomes. As previously described, eligible articles that relate to an individual study will be considered together as one study. For example, an outcome measured and reported in exactly the same way in both an interim analysis and final results report would only be extracted once in relation to that study. If a new outcome was measured and reported in the final results report, this would be considered a new outcome and extracted in addition to all outcomes in the interim analysis.

Data analysis

Tabulation and descriptive data analysis will be performed in Microsoft Excel (V.16.34, Microsoft) with the aim of deduplicating outcomes to generate two lists of unique outcomes measured and reported across the breath of studies identified. Given that there exists considerable heterogeneity in the definition of what constitutes a unique outcome and the difficulties in prioritising and achieving consensus when similar outcomes are advanced, we describe our method of data analysis as per Young et al,²⁴ and classify outcomes according to the outcome framework proposed by COMET.^{19 25}

For patient-reported outcome measures (PROMs), content analysis will be performed according to the method described by Macefield et al.²⁶ For each PROM identified, the PROM development paper will be identified and reviewed in order to establish whether or not the PROM is validated for use in meningioma. Frequency of use of a PROM and the studies using it will be listed. The number of single-item and multi-item scales for each PROM will be recorded. For each single-item or multi-item PROM scale, verbatim scale name, verbatim scale component name and verbatim scale component or single-item description will be recorded. Verbatim scale component and single-item descriptions will be classified according to the COMET taxonomy into core areas and outcome domains, followed by a subdomain category.²⁵ This process will be reviewed by members of the study management group (SMG) to ensure rigorous and consistent application of the classification system.

Creation of two unique long lists of outcomes potentially relevant to stakeholder groups of COSMIC: Intervention and COSMIC: Observation

Outcomes extracted in relation to intracranial meningioma clinical effectiveness trials will be used to generate the long list for the COSMIC: Intervention eDelphi survey, while outcomes extracted in relation to clinical studies of incidental and untreated intracranial meningioma will be used to generate the long list for the COSMIC: Observation eDelphi survey. Within each long list, exact matching outcomes will be deduplicated. Those outcomes that remain will be grouped for further deduplication when similarities in spelling, meaning or context are judged to exist. For instance, the outcomes ‘seizure’ and ‘fit’ may be considered synonymous and could therefore be deduplicated to ‘seizure’. While the two long lists remain separate, a consistent approach will be applied to both. For instance, if the outcomes ‘seizure’ and ‘fit’ are within both long lists, and the outcome was deduplicated to ‘seizure’, this term would be selected for both long lists. This process will be performed jointly by three members of the SMG (CPM, AII, and MDJ), then reviewed by two further SMG members (AGM and PRW), prior to final review by SAG members and patient research partners (PRPs).

The data that describe ‘how’ and ‘when’ each extracted outcome is measured will be used for subsequent COS work to inform ‘how’ and ‘when’ the outcomes constituting both COS could be measured. The review will be reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines where applicable,²⁷ which for the purposes of this study would exclude the following items (11, 12, 13e, 13f, 14, 15, 18, 19, 20, 21, 22).

Phase II: outcome consolidation and consensus building

Phase II incorporates key stakeholders into The COSMIC Project which includes patients with lived experience of intracranial meningioma, along with other key stakeholder groups. The aim of phase II is to first reduce the two long lists of unique outcomes from phase I by the use of two modified eDelphi surveys, and then to ratify the final COSMIC: Intervention and COSMIC: Observation COS at two independent, 1-day online consensus meetings.²⁸

eDelphi surveys

The eDelphi surveys will adhere to the standards described by COMET.¹⁹ As previously described, the SAG will scrutinise the two long lists developed by the SMG to ensure both are fit for purpose. This may include removal of obsolete outcomes, further rationalisation of outcomes or the addition of new outcomes felt to be of importance. The COSMIC Project does not include primary, semistructured interviews with patients with an intracranial meningioma. However, patient-centred outcomes have been reported in published, semistructured interviews conducted with patients with intracranial meningioma that explored the relevance of issues, and issues not addressed, in existing health-related quality of life questionnaires.²⁹ Additional, unique patient-centred outcomes reported in this source will be considered by the SAG for supplementation of the two long lists if deemed necessary.

Each unique outcome to be included in the eDelphi survey will be ascribed a lay definition. The two final long lists of unique, COMET-classified outcomes with associated lay definitions will be reviewed by the study PRPs to ensure there is clarity of meaning, a lack of replication of outcomes within each list and lists which are acceptable to patient participants with respect to length.³⁰

Research question

Which outcomes do patients with an intracranial meningioma, healthcare professionals, researchers and other key stakeholders believe should be included in a COS for use in future meningioma clinical effectiveness trials and in a COS for use in future clinical studies of incidental and/or untreated intracranial meningioma?

Methods

Key stakeholders, including patients with a radiological or histological diagnosis of intracranial meningioma, will be invited to participate in one or both of the eDelphi surveys. The eDelphi surveys will use a ‘modified’ approach³¹ as opposed to a ‘traditional’ approach,³² whereby the outcomes obtained from phase I of the study are presented in the first round of the eDelphi surveys for rating. The first round of the eDelphi surveys will also offer the opportunity for participants to suggest outcomes that have not been presented. These outcomes would not be scored but will be reviewed prior to the second round of the eDelphi surveys by the SMG and considered for inclusion in the second round, should the outcome be judged as unique and appropriate. Two rounds will be used to reduce attrition bias, along with two panels (panel 1—healthcare professionals and researchers, panel 2—patients and relatives/patient support roles).³³ Participant-level data will be pseudoanonymised. A participant’s identity is not revealed, and responses made by an individual participant are not identifiable.

Inclusion criteria

Participants will be recruited to phase II of the study from three key stakeholder groups: healthcare professionals and researchers who will use the COS, patients with a radiological or histological diagnosis of intracranial meningioma and other stakeholders in a caring or supporting role to a patient with an intracranial meningioma. All participants must be over the age of 18 and able to complete the online survey/s in English.

Patients with meningioma

Patients who have completed or are receiving treatment for an intracranial meningioma with surgery, radiotherapy, stereotactic radiosurgery or pharmacotherapy, either in isolation or in combination, are eligible to participate in the COSMIC: Intervention eDelphi survey only. Patients who have not received treatment for a radiologically diagnosed intracranial meningioma are eligible to participate in the COSMIC: Observation eDelphi survey only.

Healthcare professionals and researchers

Any member of the clinical team directly responsible for the care of patients with a meningioma. The neuro-oncology multidisciplinary team (MDT; also known as the ‘tumour board’ in other countries) consists of the following roles who are all eligible to participate: neurosurgeons and ear, nose and throat surgeons who operate on meningiomas, neuro-oncology specialist nurses, radiation oncologists, medical oncologists, neurologists, neuropathologists and neuroradiologists. While it is anticipated that the majority of researchers likely to use both COS will also be healthcare professionals directly involved in the care of patients with a meningioma, those not directly involved in care but who are likely to use the COS will also be eligible to participate (eg, neuropsychologist or epidemiologist). Healthcare professionals are eligible and will be encouraged to participate in both eDelphi surveys.

Caring or supporting roles

Individuals who provide a regular and involved caring or supporting role to a patient with a meningioma will be eligible to participate, including the following: primary carers, family members and charity/support group representatives as these participants will likely offer a different but important perspective on outcomes that matter to patients with meningioma. Individuals in a caring or supporting role are eligible to participate in one or both eDelphi surveys dependent on the individual participant’s experience. For instance, a relative in a supporting role to a patient with an incidental and/or untreated meningioma would only be eligible to participate in the COSMIC: Observation eDelphi surveys. However, a charity support worker who has provided input to both patients who have received treatment, and patients who have not, would be eligible and encouraged to participate in the eDelphi surveys for both COSMIC: Intervention and COSMIC: Observation.

Sampling and recruitment

Healthcare professionals

Healthcare professional participants will be recruited locally, nationally and internationally. The main study site (The Walton Centre NHS Foundation Trust) has a weekly neuro-oncology MDT meeting which will be contacted to recruit local healthcare professional participants. Neuro-oncology MDTs or similar will be contacted at all other UK neurosurgical centres to maximise national recruitment. Where personal contacts of the SAG exist, these will also be used. National recruitment will also be sought by advertisement through national professional societies, including the British-Irish Meningioma Society, the British Neuro-Oncology Society, the Society of British Neurological Surgeons and the British Skull Base Society.

International recruitment of healthcare professional participants will be driven again by personal contacts of the SAG, but also through a number of international professional societies, including the EORTC Brain Tumour Group, the European Association of Neuro-Oncology, the International Consortium on Meningioma, the Response Assessment in Neuro-Oncology Patient-Reported Outcome Group and the Society for Neuro-Oncology. Key international collaborators will be asked to distribute the recruitment email within their own neuro-oncology MDT or tumour board to maximise healthcare professional recruitment. To promote participation by healthcare professionals at the forefront of meningioma clinical research, the chief investigators of published trials and studies conducted in more recent years that are identified through the systematic reviews, along with the chief investigators of ongoing clinical trials and studies, will also be contacted and invited to participate.

Patients and those in caring or supporting roles

Patients will be invited to participate in this study through charities, support groups and social media platforms/forums. Charities and support groups will be contacted and a named contact for each will be sourced. This contact will circulate the participant invitation email, which will include a link to the study website (thecosmicproject.org) and the online DelphiManager platform. We will encourage named contacts to share recruitment details on social media in order to recruit participants who may not be on a charity or support group mailing list, but who may interact with a social media account of the same organisation. The International Brain Tumour Alliance, The Brain Tumour Charity (TBTC), Brainstrust–the brain cancer people and the Brain Tumour Foundation of Canada will all likely contribute the majority of opportunities to recruit patient participants as they each maintain a database of patients with intracranial meningioma. Study social media accounts will also be created to interact directly with potential participants and thereby increase patient participant recruitment.

Sample size

No specific requirements exist for the minimum number of participants to be included in an eDelphi survey in order to gain consensus,³⁴ but it is generally considered that having more participants increases the reliability of the group’s judgement.³⁵ However, for the purposes of this study, a minimum of 20 participants will be required for each panel of the eDelphi surveys (panel 1—healthcare professionals, panel 2—patients and those in caring or supporting roles). We will not limit the number of participants who may wish to register to complete the eDelphi surveys. However, study registration after closure of round 1 of the eDelphi surveys will not be permitted.

Registration

The study website (thecosmicproject.org) will contain all necessary information about the study. Registration to participate as an individual from any stakeholder group will only be possible via the online DelphiManager platform. This is accessed through an emailed link or through the study website. On attempting to register, a number of screening questions will be asked. First, registering participants will be asked to identify to which stakeholder group they belong (healthcare professional and/or researcher, patient with a meningioma, primary carer, family member, charity/support group representatives or other stakeholder with a supporting role). Further screening questions will ensure the eligibility criteria are met within each stakeholder group as previously described. Specific information will then be collected depending on the stakeholder group chosen.

For healthcare professionals, job role will be identified (categorised) along with years in practice (categorised). Country of clinical practice will be recorded to analyse the international contribution of healthcare professionals, including differences in outcome scoring by continent or region. This could have implications for dissemination of the final COS.

For patients with an intracranial meningioma (all of which rely on a self-reported diagnosis), baseline demographics will be recorded in order to ensure the patient cohort is representative of the demography of this disease (age, sex). The number of years since diagnosis (categorised) will be requested to analyse whether this variable affects the scoring of outcomes. Finally, the level of treatment will be recorded for instance (incidental/untreated, surgical intervention only, radiotherapy or stereotactic radiosurgery only, pharmacotherapy use, or a combination of surgery and radiotherapy or stereotactic radiosurgery and/or pharmacotherapy) and number of years since diagnosis and treatment (categorised). The response to the level of treatment question will determine which eDelphi survey the patient will be encouraged to complete.

For patients, three further pieces of information will be obtained in order to evaluate methods and motivations for registration for studies such as this. These will include (a) the format by which recruitment was achieved, (b) the principal motivator for registration, and (c) the most important factors within the recruitment advert for initiating registration. These data will subsequently be used to analyse and draw conclusions on how best to recruit to eDelphi surveys in the future.

For carers or those in a supporting role to patients with a diagnosis of intracranial meningioma, baseline demographics will be recorded in order to evaluate if differences affect the scoring of outcomes (age, sex). The specific role will be requested (categorised). Whether the role relates to patients with incidental and/or untreated or treated intracranial meningioma will be requested and this response will determine if the participant will be encouraged to complete one of or both eDelphi surveys. The number of years in this role (categorised) will also be requested to analyse whether this variable affects the scoring of outcomes.

Consent

Consent to participate in the eDelphi survey/s will be obtained as eConsent by all participants at the point of registration. A participant information leaflet will be provided for download from the study website, as well as an email attachment to accompany e-invitations. Sufficient time will be available for participants to choose to partake in this study prior to closure of the first round of the eDelphi surveys. Similarly, consent to participate in the consensus meeting will be obtained as eConsent by all participants prior to the online consensus meetings. A participant information leaflet will again be provided. Finally, participants will be offered the opportunity to consent to be listed as a named individual within a collaborative authorship group: The COSMIC: Intervention Collaborative and/or The COSMIC: Observation Collaborative.

Surveys

The two eDelphi surveys will be constructed and delivered through the online DelphiManager platform. The software was developed by the COMET initiative for this specific purpose. The eDelphi surveys will be piloted with members of the SMG, PRPs and lay contributors. At the beginning of the eDelphi survey, instructions will be provided on how to complete the survey. Plain language summaries and videos developed by the COMET ‘Patient Participation, Involvement and engagement group’ will be used during the registration and eDelphi administration process to facilitate understanding.

Data collection will last for a period of 4 weeks for both rounds of both eDelphi surveys. Participants completing the COSMIC: Intervention and/or COSMIC: Observation eDelphi surveys will complete both first rounds consecutively. Reminders will be sent to participants who have not completed the survey/s following registration, and following a request to complete round 2 of the survey. Reminders will be sent 2 weeks, 1 week and 48 hours prior to closure of the surveys. Failure to complete the survey within the 4-week period would be recorded as a failure to complete that round of the eDelphi survey.

Scoring

Inclusion of an outcome in either the COSMIC: Intervention or COSMIC: Observation COS requires a large majority agreement from both panels of its critical importance.³⁶ During round 1 of the eDelphi surveys, participants will rate the importance of each outcome presented using the 9-point Likert scale. It shall be explained to participants that the following scores represent outcome importance, whereby (1–3) is of limited importance, (4–6) is important but not critical and (7–9) is critically important. Previous studies have demonstrated that a 9-point scale provides adequate discrimination, does not overburden and is suitable when a subsequent eDelphi round or consensus meeting will take place.19 34 37 38 Outcomes will be grouped by domain so that similar outcomes are viewed together. Lay terms and definitions will be used with medical terms given in brackets (using PRP review). Ordering of grouped outcomes will be randomised to prevent question order from impacting the results.³⁹

All items from round 1 will be carried forward to the second round of each eDelphi survey. In the second round of the eDelphi surveys, round 1 response from each panel will be presented graphically for each outcome in order to demonstrate the distribution of allocated scores across the Likert scale. This method facilitates consensus building by allowing participants to consider the aggregate responses of their own and the alternative panel.^40–42 Participants will again rate on a 9-point Likert scale. A change in score will prompt the participant to be offered the chance to explain their reasoning, but this is not mandatory. At each round of the eDelphi surveys and at the consensus meetings, data will be recorded on number of participants invited, number completing the study section and the measure of response to each outcome. The results of the second round of each eDelphi survey will be used to determine what outcomes are dropped and what outcomes are included in the final COSMIC: Intervention and COSMIC: Observation COS, or discussed at the relevant consensus meeting if undecided.

Analysis

The definition of consensus which will include an outcome (consensus in) beyond round 2 of the eDelphi surveys and during the consensus meetings is 80% or more of participants from both panels scoring an outcome as critical (7–9). Should an outcome be rated as critical by only 50% or less of participants from both panels, the outcome would be dropped (consensus out). Consensus percentage will be calculated for each panel as follows: number of participants scoring particular outcome as critical/total number of participants scoring that outcome × 100). Outcomes to be discussed and voted on at the consensus meetings will be those that are neither included or dropped. The same definition of consensus will apply. All participants who complete both rounds of an eDelphi survey will be eligible to take part in the consensus meeting associated with that eDelphi survey. Participants eligible to complete both eDelphi surveys, who subsequently complete both rounds of both eDelphi surveys, will be eligible to take part in both consensus meetings.

Attrition between rounds

While we endeavour to retain as many participants as possible between rounds 1 and 2 of the eDelphi surveys, it is expected that a proportion will not complete the second round. The attrition rate will be calculated between rounds. In order to assess for attrition bias, the mean round 1 scores for the participants completing both rounds of an eDelphi will be compared with those that only complete the first round. The importance of completing both rounds will be emphasised. This will be recorded and analysed to compare views of those completing one versus two rounds and discussed at the consensus meeting.

Consensus meetings

Research question

Can two COS be ratified for subsequent use in clinical effectiveness trials for patients with intracranial meningioma and clinical studies of incidental and untreated intracranial meningioma?

Methods

Two independent online consensus meetings will take place. Each consensus meeting will commence with a brief presentation to reaffirm the purpose of the meeting. During each consensus meeting, outcomes categorised as ‘consensus-in’ and ‘consensus-out’ across all stakeholder groups will be reviewed first. This will provide consensus meeting participants the opportunity to discuss those outcomes that have a preliminary decision following the eDelphi surveys. Review of these outcomes may prompt further rationalisation and/or refinement of definitions. The primary focus of each meeting will be to discuss those outcomes that are yet to achieve ‘consensus-in’ or ‘consensus-out’. These outcomes will be presented in batches and discussion encouraged between all consensus meeting participants in order to move towards consensus. Each outcome will be voted on and further discussion encouraged if necessary.

Sampling and recruitment

During registration for the eDelphi surveys, participants will be informed of and asked if they would like to be considered for invitation to either one or both consensus meetings (where eligible). To be eligible, it will be mandatory that both rounds of the appropriate eDelphi survey/s are completed. A minimum of thirty participants will be invited from the UK and internationally. We will apply stratified purposive sampling based on the judgement of the SAG to select attendees in order to balance stakeholders’ specialities, participant’s disease severity and level of intervention where applicable.

Scoring and analysis

After each round of discussion, confidential electronic voting will take place involving all consensus meeting participants. Participants will again be required to vote on a 9-point Likert scale. The same consensus criteria applied after round 2 of the eDelphi surveys will also be applied at the consensus meetings. This will include the requirement that participants from both panels are in independent agreement. Only those outcomes that achieve ‘consensus-in’ will be included in the COSMIC: Intervention or COSMIC: Observation COS. Should either consensus meeting not achieve ratification of a COS, a further meeting would be arranged to achieve this.

Patient and public involvement

The SAG was formed to guide the management of The COSMIC Project. The SAG is formed from key stakeholder representatives. The purpose of the SAG is to ensure the aims of the study are delivered at all stages. This is achieved by obtaining feedback on proposed study methodology, delivery and research output.

The PRP team at TBTC were contacted to identify potential PRPs. One patient volunteered to join the SAG and has been involved in the design of this study. Two further PRPs were put into contact with the SMG by a member of the SAG. The scope of both COS has been decided with PRP input and the PRPs have a critical role in delivering this study. All materials associated with this study will be reviewed by the PRPs for clarity and understanding, including recruitment materials, consent forms, participant information leaflets and the study website. The outcome long lists will be reviewed by the PRPs for potential missing outcomes, as well as clarity, length and meaning prior to commencement of the eDelphi surveys.³⁰ Two additional PRPs have been recruited to facilitate this specific aim. The eDelphi surveys will be pilot tested with the PRPs. The PRPs will be remunerated for their time. Dissemination plans will be discussed and reviewed by the PRPs.

Ethics and dissemination

This study is registered with the COMET database as study 1508 and accessible at https://www.comet-initiative.org/Studies/Details/1508. Institutional review board (University of Liverpool) sponsorship and ethical approval has been obtained for The COSMIC Project (Ref UoL001601). Participant eConsent will be obtained prior to participation in the eDelphi surveys and online consensus meetings.

The systematic literature reviews and trial registry searches will be published, as well as the final COSMIC: Intervention and COSMIC: Observation COS, and will be freely available. All eDelphi participants completing both rounds of the eDelphi survey for COSMIC: Intervention and/or COSMIC: Observation will be offered the opportunity to be listed as a named individual within a collaborative authorship group: The COSMIC: Intervention Collaborative and/or The COSMIC: Observation Collaborative. Results will be distributed to relevant professional organisations as well as charity and support groups. National and international presentation of results will take place after ratification of both COS. This project will continue in order to define how and when each core outcome should be measured. Both COS will be subject to refinement with the passage of time, in order to reflect changes in key stakeholder opinion as and when treatment paradigms change.

Multiple organisations support the use of COS (National Institute for Health Research (NIHR) Health Technology Assessment and WHO handbook for guideline development). The NIHR states in their application form: ‘where established core outcomes exist they should be included among the list of outcomes unless there is good reason to do otherwise’. This protocol adheres to the 13 minimum Core Outcome Set-Standardised Protocol Items recommendations.⁴³

Ethics statements

Patient consent for publication

Not required.