Abstract
This is a protocol for a Cochrane Review (Prognosis). The objectives are as follows:
Primary objective
The primary objective of this review is to determine which technique (test) for assessing MGMT methylation status best predicts overall survival in people diagnosed with glioblastoma who are treated with temozolomide. We will consider each MGMT test as a separate prognostic factor.
See Table 2 for the review question in population, index prognostic factor, comparator prognostic factor(s), outcome, timing, and setting (PICOTS) format.
Secondary objective
We will undertake a full integrated economic review to identify economic evaluations in relation to the different methods of assessing MGMT methylation status effect on overall survival. Furthermore, we will develop a simple cost‐effectiveness decision model exploring the cost‐effectiveness of alternative approaches to assessing MGMT methylation status.
Investigation of sources of heterogeneity
If we identify a sufficient number of studies for inclusion, we will examine for each technique/test whether any of the following features is best associated with overall survival.
Promoter region/ 5'‐C‐phosphate‐G‐3' (CpG)s analysed (or the antibody used in the case of immunohistochemistry)
Cut‐off used (where relevant)
Type of tumour sample (FFPE or frozen)
We will also investigate the effect of population characteristics including the following.
Age
Extent of tumour resection
Karnofsky performance status
IDH status
Recurrent versus first diagnosis
We are assuming constant HRs. To confirm the validity of this assumption we will investigate length of follow‐up as a source of heterogeneity. If studies have started follow‐up for overall survival from different timepoints, we will also investigate this as a source of heterogeneity.
Background
Description of the health condition and context
Glioblastoma is an aggressive form of brain cancer. Approximately five of every 100 people with glioblastoma survives for five years past diagnosis (Ostrom 2014). Glioblastomas that have a particular modification to their DNA (called methylation) in a particular region (the O6‐methylguanine–DNA methyltransferase (MGMT) promoter) respond better to treatment with chemotherapy using a drug called temozolomide. Although we know that modification of this DNA region is important, we don't know the best way to measure it. In this Cochrane Review we aim to assess which way of measuring methylation of the MGMT promoter best predicts survival when people with glioblastoma are treated with temozolomide.
Gliomas are a group of brain tumours that share some features with glial cells, which are the cells that support and insulate neurons. The World Health Organization (WHO) divides gliomas into astrocytic, oligodendroglial, and ependymal tumours, and other rarer subtypes depending on the type of glial cell the tumour shares features with (Louis 2016). Glioblastoma is the most malignant (aggressive) type of astrocytic tumour (Louis 2016), and the most common primary brain tumour among adults. Age‐adjusted incidence of primary (isocitrate dehydrogenase (IDH)‐wild type) glioblastoma (ICD‐O‐3 morphology codes 9440 to 9442, WHO grade IV) ranges from 0.59 to 3.69 per 100,000 people (Ostrom 2014). IDH‐wildtype glioblastomas increase in incidence with age, peaking in the 74 to 84‐year old age group (Ostrom 2014). These glioblastomas are associated with poor prognosis, with a five‐year relative survival of approximately 5% (Ostrom 2014). The median overall survival is 9.9 months for people treated with surgery plus radiotherapy, and 15 months for people treated with surgery plus radiotherapy plus chemotherapy (Louis 2016). For people with secondary (IDH‐mutant) glioblastomas, median overall survival is 24 months for people treated with surgery plus radiotherapy, and 31 months for people treated with surgery plus radiotherapy plus chemotherapy (Louis 2016).
Most people presenting with neurological symptoms are referred to their local neurosurgical multidisciplinary team following imaging with computerized tomography (CT) and magnetic resonance imaging (MRI) scans of the brain. If appropriate, the person proceeds to diagnostic biopsy or resection (surgical removal) of the tumour to confirm a histopathological diagnosis. Once the diagnosis has been confirmed, the decision is made for subsequent treatment, which can include radiotherapy and chemotherapy. For newly diagnosed glioblastoma, maximal surgical resection followed by radiation therapy with concomitant and adjuvant temozolomide is standard therapy (Stupp 2005). Temozolomide is a chemotherapeutic, more specifically an alkylating agent, that causes DNA damage. This DNA damage results in inhibition of DNA replication. Not all people respond to temozolomide therapy to the same extent. There is evidence that people with newly diagnosed glioblastoma who start treatment with radiation therapy and temozolomide greater than six weeks after neurosurgery have worse overall survival than people who start treatment within six weeks (Sun 2015).
In the UK, it is estimated that on average just over 20 years of life are lost per person with a brain tumour, the most of any form of cancer (Burnet 2005). Olesen 2012 estimated the total annual costs of brain tumours in Europe to be EUR 5.2 billion, based upon Purchasing Power Parity (PPP) rates for 2010.
Description of the prognostic factors
MGMT is a DNA repair enzyme in tumour cells that can repair the damage caused by alkylating agents, such as temozolomide. Methylation of MGMT in the tumour cell stops the repair enzyme working and the tumour cell can't repair itself and therefore the cancer cell then dies. If the MGMT in the tumour cell is unmethylated, then the cancer cell can repair the damage caused by temozolomide and therefore temozolomide is ineffective. Epigenetic silencing of the MGMT gene by promoter methylation is associated with longer overall survival in people with glioblastoma receiving alkylating therapy in addition to radiotherapy (Esteller 2000; Hegi 2004; Hegi 2005). A retrospective analysis of a randomized phase III trial found that treatment with temozolomide and radiotherapy conferred a significant survival benefit versus radiotherapy alone in people with MGMT promoter methylation (median survival 21.7 months, 95% confidence interval (CI) 17.4 to 30.4 versus 15.3 months 95% CI 13.0 to 20.9, P = 0.007), whereas a smaller difference in survival between treatment groups was seen in people with unmethylated MGMT (median survival 12.7 months, 95% CI 11.6 to 14.4 versus 11.8 months 95% CI 9.7 to 14.1) (Hegi 2005).
It is thought that glioblastomas that have silenced the MGMT gene are less capable of repairing the damage caused by temozolomide, and therefore more sensitive to alkylating therapy (Brandner 2015). However, MGMT methylation status does not always reflect gene expression, so the exact mechanism by which MGMT promoter methylation improves response to alkylating therapy is still unknown.
MGMT promoter methylation status testing is clearly important in treatment decisions in elderly people, as treating tumours with an unmethylated MGMT promoter with temozolomide is detrimental (when single agent temozolomide chemotherapy was compared to radiotherapy) (Malmström 2012; Wick 2012). On the basis of these findings professional bodies, such as the European Association for Neuro‐Oncology (EANO), recommend evaluation of MGMT promoter methylation status in elderly people (Weller 2017). The National Institute for Health and Care Excellence (NICE) recommends that all high‐grade gliomas are tested for MGMT promoter methylation to inform prognosis and guide treatment in their guideline on primary brain tumours and brain metastases in adults (NICE 2018). Most non‐elderly (aged under 65 years) people are treated with temozolomide chemotherapy irrespective of MGMT promoter status, possibly due to the lack of alternative treatments (Hegi 2015). However, MGMT promoter status is still a useful prognostic marker which may impact clinical management, and may also be used for recruitment into clinical trials for novel therapies.
There are many ways of assessing methylation status. These include the following.
Quantitative real time polymerase chain reaction (PCR) or MethylLight methylation‐specific quantitative PCR
Methylation‐specific PCR (MSP)
Methylation specific sequencing, including pyrosequencing
Bead array
Methylation‐specific multiplex ligation‐dependent probe amplification (MS‐MLPA)
PCR with high resolution melting (HRM)
Co‐amplification at lower denaturation temperature (COLD)‐PCR
Digestion‐based assays
We have briefly described these techniques in Table 1. In addition, protein expression or enzymatic activity may be used as a proxy for methylation status. However, internationally accepted consensus about the most appropriate diagnostic method for MGMT promoter status is lacking (Brandner 2015). MSP was used to assess MGMT promoter status in the landmark study by Hegi 2005. The choice of technique used to assess MGMT promoter status in practice may depend on the amount and quality of the DNA sample(s) (e.g. Formalin‐Fixed Paraffin‐Embedded (FFPE) versus frozen tissue‐derived DNA), the robustness and simplicity of the method, the availability of equipment and reagents necessary for each of the techniques, cost, and experience. In the last UK National Quality Assessment (UK NEQAS) External Quality Assessment report, of 18 UK laboratories 10 used pyrosequencing, five MSP, two HRM, and one MS‐MLPA. In addition, these techniques can only investigate methylation status in specific regions within the MGMT promoter (which may be different even when the same technique is used), and the effect of methylation status at different sites on prognosis is not well understood. In addition, some of the techniques quantify the amount of methylation present, and there is no consensus regarding the cut‐off for categorizing methylation status.
Table 1.
Methods of determining methylation status
| Test | Brief description |
| MSP | In MSP, DNA is extracted from tumour tissue and then treated with sodium bisulfite. Sodium bisulfite causes changes in the sequence of unmethylated DNA, as it changes the DNA base cytosine into uracil. Methylated DNA is protected and remains unchanged. Regions of DNA can then be amplified using PCR in a manner that is dependent on whether the changed (containing uracil) or original sequence (containing cytosine) is present. |
| Quantitative real time PCR | This technique is very similar to MSP, but there is a measure of the amount of changed and original DNA sequence. |
| Methylation‐specific sequencing | In methylation‐specific sequencing, DNA is again extracted from tumour tissue and treated with sodium bisulfite, which changes unmethylated DNA. The DNA can then be sequenced to see if it contains the changed or original sequence, i.e. whether it contains uracil in place of cytosine. There are many ways of sequencing DNA, but one commonly used method is called pyrosequencing. |
| Bead array | In this technique DNA is again extracted from tumour tissue and treated with sodium bisulfite, which changes unmethylated DNA. The DNA is then hybridized to sequences that are either complementary to the original sequence or changed sequence. The hybridization produces a signal which can be measured. |
| MS‐MLPA | In MLPA the DNA is treated with an enzyme that cleaves unmethylated DNA at specific sequences, but methylated DNA is protected. PCR to amplify regions of DNA is then performed. Amplification will only occur if the DNA was not cleaved. |
| PCR with HRM | This technique relies on the fact that the changes to DNA caused by sodium bisulfite (i.e. the replacement of cytosine by uracil) lead to it having a lower melting temperature, which is the temperature at which the two different DNA strands come apart. Methylated DNA will have a higher melting temperature. A dye that changes fluorescence depending on whether the DNA strands are together or apart can be added. |
| COLD‐PCR | This technique relies on the same principle as PCR with HRM. In this case only sequences with low melting temperatures will be amplified. This means that only unmethylated regions will be amplified. |
| Digestion‐based assays | This technique also relies on enzymes that cleaves unmethylated DNA at specific sequences, but methylated DNA is protected. |
Abbreviations: COLD: Co‐amplification at lower denaturation temperature; DNA: Deoxyribonucleic acid; HRM: high resolution melting; MLPA: multiplex ligation‐dependent probe amplification; MS‐MLPA: Methylation‐specific multiplex ligation‐dependent probe amplification; MSP: methylation‐specific PCR; PCR: polymerase chain reaction.
The result of each technique/test for MGMT status can be considered a separate prognostic factor for predicting overall survival in people with glioblastoma treated with temozolomide.
Health outcomes
The health outcome of interest for this review is overall survival.
We are not limiting the period of follow‐up. Glioblastomas are associated with poor prognosis, so we anticipate that most studies will assess overall survival within five years of diagnosis.
Why it is important to do this review
It is important to reach a consensus regarding which is the best method for assessing MGMT methylation status based on the prognostic value of each method in predicting overall survival in people with glioblastoma treated with temozolomide. The regions of the promoter that need to be analysed and the most relevant cut‐offs for quantitative tests need to be established. Systematic reviews have assessed the prognostic value of MGMT promoter status assessed by a specific technique, for example by pyrosequencing (Zhao 2016), or MSP (Zhang 2013). However, no review has determined which method is best correlated with prognosis (although a review, Dullea 2016, aimed to do this but provided no quantitative synthesis of the results, only a narrative overview).
In this Cochrane Review we will seek to determine which technique, assessing which regions, and (if relevant) which cut‐off is best associated with overall survival in people with glioblastoma treated with temozolomide. We will consider each MGMT test as a separate prognostic factor. We will extract or calculate (where possible) hazard ratios (HRs) for those who test positive compared to those who test negative. A test that is not better than flipping a coin is expected to have a HR of one. The better the test can distinguish between those with a good overall survival versus those with poor overall survival, the further the HR value will be from one.
By seeking to determine which technique, assessing which regions, and (if relevant) which cut‐off is best associated with overall survival in people with glioblastoma treated with temozolomide, this review will partially answer the question “Do molecular subtyping techniques improve treatment selection, prediction and prognostication in people with brain and spinal cord tumours”, one of the top 10 topics identified by the James Lind Alliance Neuro‐Oncology Priority Setting Partnership (JLA PSP 2018). The James Lind Alliance is an organization that brings patients, carers, and clinicians together to set research priorities. The National Cancer Research Institute Brain Tumour Clinical Studies Group has also identified this as an area for future research.
It is also important to consider the cost effectiveness of alternative methods of assessing MGMT promoter methylation status. Each method of assessment will incur costs, such as laboratory costs, clinic costs, and subsequent treatment costs. The benefits of targeting treatment may include greater survival and less exposure to potentially toxic treatments, as well as potential cost‐savings from the avoidance of waste from the use of ineffective drugs. This review will consider the costs alongside the consequences of the prognostic tests to understand the value that they provide to the healthcare system.
Objectives
Primary objective
The primary objective of this review is to determine which technique (test) for assessing MGMT methylation status best predicts overall survival in people diagnosed with glioblastoma who are treated with temozolomide. We will consider each MGMT test as a separate prognostic factor.
See Table 2 for the review question in population, index prognostic factor, comparator prognostic factor(s), outcome, timing, and setting (PICOTS) format.
Table 2.
Review question in PICOTS format
| Population | Patients with diagnosed glioblastoma (at any point after diagnosis) who go onto be treated with temozolomide |
| Index prognostic factors | Tests for MGMT promoter methylation. We will consider each test as a separate prognostic factor |
| Outcome | Overall survival |
| Timing | The outcome is to be predicted at any point after the start of treatment |
| Setting | To give prognostic information before the start of treatment with temozolomide |
Abbreviations: MGMT: O6‐methylguanine–DNA methyltransferase; PICOTS: Population, Index prognostic factor, Comparator prognostic factor(s), Outcome, Timing, Setting.
Secondary objective
We will undertake a full integrated economic review to identify economic evaluations in relation to the different methods of assessing MGMT methylation status effect on overall survival. Furthermore, we will develop a simple cost‐effectiveness decision model exploring the cost‐effectiveness of alternative approaches to assessing MGMT methylation status.
Investigation of sources of heterogeneity
If we identify a sufficient number of studies for inclusion, we will examine for each technique/test whether any of the following features is best associated with overall survival.
Promoter region/ 5'‐C‐phosphate‐G‐3' (CpG)s analysed (or the antibody used in the case of immunohistochemistry)
Cut‐off used (where relevant)
Type of tumour sample (FFPE or frozen)
We will also investigate the effect of population characteristics including the following.
Age
Extent of tumour resection
Karnofsky performance status
IDH status
Recurrent versus first diagnosis
We are assuming constant HRs. To confirm the validity of this assumption we will investigate length of follow‐up as a source of heterogeneity. If studies have started follow‐up for overall survival from different timepoints, we will also investigate this as a source of heterogeneity.
Methods
Criteria for considering studies in this review
Types of studies
We will include longitudinal studies of adults with diagnosed glioblastoma treated with temozolomide with/without radiation therapy/surgery that have related MGMT status in tumour tissue assessed by one or more technique with overall survival. This includes the temozolomide treated arms of RCTs. We will also include nested case‐control studies. We will exclude cohort studies performed exclusively in people who have survived a particular amount of time.
We will exclude case reports.
To be included, studies must have determined MGMT status from samples taken prior to the initiation of treatment. We will include studies with any length of follow‐up.
We will only include studies if HRs are reported or can be calculated from the data reported.
Types of studies for the economic component
We will include economic evaluations conducted alongside trials, modelling studies, and cost analysis to inform the identification of cost effectiveness outcomes.
Targeted participants
We will include studies of adults with diagnosed glioblastoma treated with temozolomide with/without radiation therapy/surgery.
If studies included people with other forms of glioma (and we cannot extract results for the population with glioblastoma), we will include these if other forms of glioma make up less than 10% of the population.
We will exclude studies performed exclusively in paediatric populations (under 18 years of age). We will include studies of participants with either first diagnosis or recurrent glioblastoma. Participants in eligible studies could receive concomitant and adjuvant therapies in addition to temozolomide (e.g. surgery or radiation therapy, or both). If not all participants received temozolomide (e.g. in the context of a RCT), we will include data on people who did receive temozolomide if this is available.
Types of prognostic factors
We will include studies that assess MGMT promoter methylation status in tumour tissue by one or more techniques.
We will treat each test for MGMT as a separate prognostic factor.
Eligible techniques will include MSP; quantitative real time PCR or MethylLight methylation‐specific quantitative PCR; methylation‐specific sequencing, including pyrosequencing; bead array; MS‐MLPA; PCR with HRM; COLD‐PCR; and digestion‐based assays.
We will include testing strategies that look at MGMT expression (e.g. immunohistochemistry for protein expression, or tests looking at messenger ribonucleic acid (mRNA) levels), or MGMT enzymatic activity.
Eligible techniques must be performed directly on tumour tissue. We will exclude studies that assess MGMT promoter methylation status from blood samples. We will only include molecular techniques. We will exclude studies that infer MGMT methylation status due to macroscopic morphological changes that can be detected by, for example, imaging (i.e. MRI, CT, positron emission tomography (PET)).
We will exclude studies if the method of determining MGMT promoter methylation status is not reported, as this information is essential for this review.
Types of outcome to be predicted
Overall survival
Outcomes of the economic component
Resources use, costs, cost effectiveness, and cost‐utility of different methods of assessing MGMT promoter methylation status based on full economic review
Relative efficiency of each method of testing for MGMT promoter methylation status based on a decision model using the outcomes from the review of effectiveness and from the full integrated economic review
Search methods for identification of studies
Electronic searches
We will search the following databases:
MEDLINE Ovid (from 1946 to current date)
Embase Ovid (from 1980 to current date)
The MEDLINE search strategy is in Appendix 1. We will adapt this for Embase.
We will search the BIOSIS Citation Index (from 1969 to current date) using the search strategy in Appendix 2.
We will also search for studies available in PubMed that are not available in MEDLINE using the syntax ‘pubmednotmedline[sb]'.
There will be no restrictions based on language or date of publication.
Searching other resources
We will search Open Grey (www.opengrey.eu/) using the free text terms from our MEDLINE search ((glioblastom* or GBM or astrocytom* or gliosarcom*) AND (((methylguanin* or methyl guanin* or alkylguanin* or alkyl guanin*) AND (methyltransferas* or methyl transferas* or alkyltransferas* or alkyl transferas*)) or AGT or MGMT or AGAT) AND (prognos* or predict* or mortality or death* or surviv*)).
We will search for relevant material in dissertations and theses using ProQuest Dissertations & Theses Global (search.proquest.com/pqdtglobal/dissertations/), again using the same strategy as for Open Grey but limiting to all fields except full text. We will also search the Networked Digital Library of Theses and Dissertations (search.ndltd.org/index.php) using the same strategy as for Open Grey.
The Society of Neuro‐Oncology (SNO), and its partner associations the EANO and the Japan Society of Neuro‐Oncology, hold meetings where relevant research may be presented. We will search for abstracts from these meetings and other relevant conferences via the Web of Science Conference Proceedings Citation Index (CPCI‐S) (from 1990 to current date). We will translate the BIOSIS search for CPCI‐S as both databases are hosted on Web of Science.
We will examine the reference lists of included studies, and of systematic reviews that have assessed the prognostic value of MGMT promoter status overall (Binabaj 2018), or as assessed by a specific technique; for example by pyrosequencing (Zhao 2016), or MSP (Zhang 2013).
Search methods for the economic component
We will perform searches for economic evaluation studies in MEDLINE and Embase from January 2015 to current date. In addition, we will search the NHS Economic Evaluation Database (EED) up to the end of December 2014, when the last records were added to that database. The NHS EED was based on a comprehensive search of bibliographic databases including MEDLINE and Embase, so searches of MEDLINE and Embase before 1 January 2015 are not required. We will also consider relevant grey literature (such as health technology assessments, reports, and working papers) for inclusion.
Data collection
We will use EPPI‐Reviewer 4 to perform the review.
Selection of studies
Two review authors will independently screen titles and abstracts of all identified search results. We will retrieve the full‐text of any article(s) that either review author deems relevant, or whose relevance cannot be determined from the abstract. Two review authors will then independently assess the full‐text articles. We will resolve any disagreements by consensus, or by consulting a third review author if necessary. Articles excluded during full‐text assessment will be listed in a 'characteristics of excluded studies' table. We will construct a preferred reporting items for systematic reviews and meta‐analyses (PRISMA) flow diagram to depict the flow of information through the different phases of the review.
We will perform full data extraction, risk of bias assessment and synthesis on studies that evaluated MGMT promoter methylation status of the same patients using two or more techniques, and will perform more limited data extraction on studies that evaluated MGMT promoter methylation status using a single technique.
Selection of studies for the economic component
We will include full economic evaluation studies in this review. This includes evaluations alongside trials and model‐based evaluations and cost analysis.
Data extraction and management
Two review authors will independently perform data extraction. We will resolve any disagreements by consensus, and will consult a third review author if necessary.
For studies that evaluated MGMT promoter methylation status of the same patients using at least two techniques, we will extract data on the following items relevant to prognostic factor studies, derived from the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) (Moons 2014).
Study characteristics
Author
Year
Country
Length of follow‐up
Study dates
Study design
Population characteristics
Number of participants
Population source and setting
Timing of MGMT promoter methylation assessment
Inclusion/exclusion criteria
Tumour type
Age
Gender
Karnofsky performance status
Extent of resection
Treatment regimen
Length of time between neurosurgery and start of treatment
IDH mutation status
First diagnosis or recurrent disease
Deaths during follow‐up
Prevalence of MGMT promoter methylation (by each technique)
Method(s) of MGMT promoter methylation assessment
Technique
Tumour sample type (i.e. FFPE or frozen tissue)
Region/CpGs analysed (for PCR‐based tests); antibody used (for immunohistochemistry)
Cut‐off/threshold used to determine MGMT promoter methylation status
Method of determining threshold and whether it was prespecified
Outcome assessment
Timepoint from which overall survival is measured
Missing data
Number of participants with any missing data
Association between MGMT methylation status and overall survival
We will extract unadjusted HRs and variances, with a HR value less than one indicating favourable outcomes in people with a methylated MGMT promoter. If HR values are not directly reported, we will calculate these, where possible, using the techniques described in Tierney 2007 and Parmar 1998.
Adjusted HRs and variances (where reported), and factors the result is adjusted for. We will extract these to confirm that MGMT tests have added prognostic value in addition to easier to measure prognostic factors such as age, gender, disease stage at diagnosis, and comorbidity.
For studies that evaluated MGMT promoter status using a single technique we will extract details on author, year, country, length of follow‐up, number of participants, tumour type, IDH mutation status and technique used for MGMT promoter methylation assessment.
In addition to the above, we will collect the following data from the economic evaluation studies.
Type of evaluations
Sources of effectiveness data
Cost data
Sources of cost data
Sources of outcome valuations
Analytical approach
Assessment of risk of bias in included studies
We will assess risk of bias of studies that evaluated MGMT promoter methylation status of the same patients using at least two techniques.
The quality in prognosis studies (QUIPS) tool is designed to assess risk of bias in prognostic factor studies (Hayden 2013). It assesses bias across six domains: study participation, study attrition, prognostic factor measurement, outcome measurement, study confounding, and statistical analysis and reporting. We will assess risk of bias across QUIPS domains, although we have combined the domains of study participation and study attrition into one domain (participant selection), we have added a domain on subsequent treatment, we have renamed the study confounding domain to adjustment for other potential prognostic factors, and we have limited the domain about statistical analysis and reporting to selective reporting. We have replaced the prompting items and considerations, which mainly assessed reporting, with signalling questions to help us come to domain‐level judgements. The domain modifications and signalling questions have been informed by the CHARMS checklist (Moons 2014), a framework for assessing internal validity of articles dealing with prognosis described in Altman 2001, as well as ROBINS‐I (risk of bias in non‐randomized studies of interventions) (Sterne 2016) and QUADAS‐2 (Whiting 2011). In addition, for each domain apart from selective reporting, we have added questions assessing the applicability of the study as in QUADAS‐2 (Whiting 2011) and PROBAST (Wolff 2019).
We will judge risk of bias and concerns regarding applicability as either high, low, or unclear. The tool is detailed in Appendix 3. Two review authors will independently perform assessments, and will aim to reach a consensus judgement. We will resolve any disagreements by consulting a third review author.
Assessment of risk of bias in studies included in the economic component
Assessment of the quality of the economic evaluations that are captured in this review will take place in two stages. The first stage is assessing the quality of the clinical effectiveness evidence that informs the evaluation. If the economic evaluation has been carried out alongside a single study, then we will use the bespoke tool described in Appendix 3 for quality assessment. If the economic evaluation is based on multiple sources, such as a modelling study, then we will use the ROBIS (risk of bias in systematic reviews) tool to assess the quality (Whiting 2016).
The second stage involves assessing the quality of the economic component of the evaluation. We will use both the consolidated health economic evaluation reporting standards (CHEERS) (Husereau 2013) and the Evers checklist (Evers 2005) to assess the quality of the economic evidence.
Assessment of heterogeneity
We will quantify heterogeneity across results of the studies using an estimate of the between‐study variance in log HRs and will portray these using prediction intervals if we perform meta‐analyses. We will report between study variance (τ2 ). In addition, we will describe the extent of inconsistency in the findings using the I2 statistic, which describes the percentage of variation across studies that is due to heterogeneity rather than chance (Higgins 2002).
Assessment of reporting bias
For each meta‐analysis that contains 10 or more studies, we will examine the symmetry of funnel plots and test for asymmetry using Debray’s‐Funnel inverse variance test based on HRs (Debray 2018). Asymmetry may be an indicator of publication bias.
Data synthesis
Data synthesis and meta‐analysis approaches
To assess the relative prognostic ability of the different techniques we will focus on data from direct, within‐study comparisons, where the MGMT promoter methylation status of the same series of people is evaluated in multiple ways and the results correlated with overall survival. Full data extraction, risk of bias assessment, and synthesis on studies will be undertaken only for this subset of studies. The prognostic value of each test may be dependent on other prognostic factors of overall survival, and these may have been adjusted for. We will also aim to extract and meta‐analyse adjusted results, to confirm that the tests have added prognostic value in addition to (easier to measure) prognostic factors such as age, gender, disease stage at diagnosis, and comorbidity.
We also expect to identify studies that have evaluated MGMT promoter using only one technique. We will present only brief details of these studies. At a later date we may investigate these studies further to supplement inferences from the comparative studies. Specifically, there may be a possibility of comparing techniques indirectly across studies. Indirect comparisons rely on the assumption that the studies assessing each test for MGMT promoter methylation are similar for all important characteristics, i.e. that they have been conducted on similar populations that have been given similar treatments (or that these factors are adjusted for) and that the risk of bias is similar.
We will present the results from the full economic review as a narrative analysis, describing the results of the economic evaluations identified by the search. In addition to the narrative summary of the economic evaluations, we will use both the clinical and economic outcomes to inform a decision model to estimate the cost effectiveness of assessing MGMT status in the management of glioma.
Subgroup analysis and investigations of heterogeneity
We will investigate the potential sources of heterogeneity in the results for each method using subgroup analyses or meta‐regression, depending on the number of studies identified and the nature of the source of heterogeneity.
If we identify a sufficient number of studies, we will examine for each technique/test whether any of the following features is best associated with overall survival.
The promoter region/CpGs analysed (or the antibody used in the case of immunohistochemistry)
The cut‐off used (where relevant)
The type of tumour sample (FFPE or frozen).
We will also investigate the effect of population characteristics including:
Age
Extent of tumour resection
Karnofsky performance status
IDH status
Recurrent versus first diagnosis.
We will assume constant HRs. To test the validity of this assumption, we will investigate length of follow‐up as a source of heterogeneity. If studies have started follow‐up for overall survival from different timepoints, we will also investigate this as a source of heterogeneity.
Sensitivity analyses
We plan to perform a sensitivity analysis restricting the analysis to studies we judge to be at low or unclear risk of bias.
Decision model
We will create an economic model using outcomes from both the clinical and economic evidence we identify. We will use the extracted data to populate a decision analytic model, which will assess the cost‐effectiveness of different methods of testing for MGMT promoter methylation status in people with glioma. The effect of the different methods of assessing MGMT promoter methylation status (including not assessing for promoter methylation status at all) will be compared in terms probability of effectiveness and overall survival. This will be from a UK NHS perspective in a population aged 65 years or over. The time horizon of the model in terms of costs considered will be six weeks until the start of temozolomide treatment and we will assess parameter uncertainty using a sensitivity analysis.
Summary of findings
We will present the prognostic value of each test on overall survival in a ‘Summary of findings' table. We will assess confidence in each result by following the GRADE approach (Guyatt 2008). Guidance on the use of GRADE for prognostic factor studies has not yet been published, although adaptations have been proposed (Huguet 2013). We will rate the overall strength of evidence as either ‘high’, ‘moderate’, ‘low’ or ‘very low’. We will consider risk of bias, indirectness, inconsistency, imprecision, and publication bias, which may lead to downgrading of the strength of the evidence; and size of effect, which may lead to upgrading of the strength of the evidence (see Appendix 4).
Acknowledgements
We thank Robin Grant (Co‐ordinating Editor) and Gail Quinn (Managing Editor) for the Cochrane Gynaecological, Neuro‐oncology and Orphan Cancer Group (GNOC) for editorial guidance. We also thank Joanne Platt (Information Specialist, GNOC) for aiding with the development of the search strategy.
We thank all of our external peer reviewers, including Karel Moons and Joie Ensor from the Cochrane Prognostic Methods Group, Catherine McBain, Andrew Bryant, Helen Bulbeck and Colin Smith.
This project was supported by the National Institute for Health Research (NIHR), via Cochrane Programme Grant funding (16/144) to the Cochrane Gynaecological, Neuro‐oncology and Orphan Cancers Group. AM and JPTH are supported in part by Cancer Research UK (grant number C18281/A19169). The views and opinions expressed therein are those of the review authors and do not necessarily reflect those of the Systematic Reviews Programme, the NIHR, the National Health Service (NHS), the Department of Health, or Cancer Research UK.
Appendices
Appendix 1. MEDLINE search strategy
1. glioma/ or astrocytoma/ or glioblastoma/ 2. (glioblastom* or GBM or astrocytom* or gliosarcom*).mp. 3. 1 or 2 4. "O(6)‐Methylguanine‐DNA Methyltransferase"/ 5. ((methylguanin* or methyl guanin* or alkylguanin* or alkyl guanin*) adj5 (methyltransferas* or methyl transferas* or alkyltransferas* or alkyl transferas* or transmethylas* or trans methylas*)).mp. 6. (methyl* DNA protein cystein* adj (methyltransferas* or methyl transferas*)).mp. 7. (AGT or MGMT or AGAT).ti,ab,kf,ot. 8. or/4‐7 9. exp Prognosis/ 10. (prognos* or predict*).mp. 11. exp mortality/ 12. survival/ 13. survival rate/ 14. exp survival analysis/ 15. (mortality or death* or surviv*).mp. 16. Follow‐Up Studies/ 17. ((followup or follow‐up) adj (study or studies)).ti,ab,kf. 18. or/9‐17 19. 3 and 8 and 18 20. exp animals/ not humans.sh. 21. 19 not 20
Appendix 2. BIOSIS Citation Index search strategy
#1 ((TS=((glioblastoma* OR GBM* OR astrocytom*) NEAR (methylguanin* OR “methyl guanin*” OR alkylguanin* OR “alkyl guanin*” OR AGT OR MGMT OR AGAT)) OR TS=((gliosarcom*) AND (methylguanin* OR “methyl guanin*” OR alkylguanin* OR “alkyl guanin*” OR AGT OR MGMT OR AGAT))) AND (TS=(prognos* or predict* or mortalit* or death* or surviv*)))
#2 TS=((prognos* OR predict* OR mortalit* OR death* OR surviv*) NEAR (methylguanin* OR “methyl guanin*” OR alkylguanin* OR “alkyl guanin*” OR AGT OR MGMT OR AGAT)) AND TS=(glioblastom* OR GBM* OR astrocytom* OR gliosarcom*)
#3 ((TS=((prognos OR predict* OR mortalit* OR death* OR surviv*) NEAR (glioblastom* OR GBM* OR astrocytom*)) OR TS=((prognos OR predict* OR mortalit* OR death* OR surviv*) AND gliosarcom*)) AND (TS=("O(6)‐Methylguanine‐DNA Methyltransferase" OR "O‐6‐Methylguanine‐DNA Methyltransferase" or “methylated DNA protein cysteine methyltransferase” OR AGT or MGMT or AGAT) OR TS=((methylguanin* or “methyl guanin*” or alkylguanin* or “alkyl guanin*”) NEAR (methyltransferas* or “methyl transferas*” or alkyltransferas* or “alkyl transferas*” or transmethylas* or “trans methylas*”))))
#4 (TS=(glioblastom* or GBM or astrocytom* or gliosarcom*) AND (TS=("O(6)‐Methylguanine‐DNA Methyltransferase" or "O‐6‐Methylguanine‐DNA Methyltransferase" or “methylated DNA protein cysteine methyltransferase” or AGT or MGMT or AGAT) OR TS=((methylguanin* or “methyl guanin*” or alkylguanin* or “alkyl guanin*”) NEAR (methyltransferas* or “methyl transferas*” or alkyltransferas* or “alkyl transferas*” or transmethylas* or “trans methylas*”))) AND (TS=(prognos* or predict* or mortality or death or deaths or surviv*)) AND (TS= methylat* OR TS=((amount or amounts or express* or level or levels or activ* or status) NEAR (protein* or AGT or MGMT or AGAT or methylguanin* or “methyl guanin*” or alkylguanin* or “alkyl guanin*”))))
#5 (#4 or #3 or #2 or #1)
Appendix 3. Risk of bias and applicability assessment
Bespoke tool to assess risk of bias and applicability of prognostic factor studies. SQ = Signalling Question.
| Domain 1: Participant selection | |
| Risk of bias | SQ1: Was a consecutive or random sample of people enrolled? |
| SQ2: Was a case‐control or cross‐sectional design avoided? | |
| SQ3: Did the study avoid inappropriate exclusions? | |
| SQ4: Were all participants included in the analysis? | |
| If no to SQ4: SQ5 Were there important differences between participants who completed the study/were included in the analysis and those who were not? | |
| Applicability | Are there concerns that the included participants and setting do not match the review question? |
| Domain 2: Prognostic factor measurement | |
| Risk of bias | SQ1: Was the method and setting of measurement of the prognostic factor the same for all participants? |
| SQ2: Was the prognostic factor objective or measured without knowledge of the outcome or risk of the outcome? | |
| SQ3: If a threshold was used, was it pre‐specified? | |
| Applicability | Are there concerns that prognostic factor, the way that it was measured, or the way that it was interpreted, differ from the review question? |
| Domain 3: Outcome measurement | |
| Risk of bias | SQ1: Was the method of outcome measurement used adequately valid and reliable? |
| SQ2: Was the method and setting of outcome measurement the same for all study participants? | |
| SQ3: Was the outcome objective or assessed without knowledge of the prognostic factor? | |
| SQ4: Do the prognostic factors investigated form part of the outcome? | |
| Applicability | Are there concerns that outcome does not match the question and/or that follow‐up was not of sufficient duration? |
| Domain 4: Subsequent treatment | |
| Risk of bias | SQ1: Did treatment vary across participants? (or “Was treatment either standardized or randomized?") |
| Applicability | Are there concerns that treatments received do not match the review question? |
| Domain 5: Adjustment for other potential prognostic factors (where relevant) | |
| Risk of bias | Were other potential prognostic factors measured adequately and reliably and in a similar manner for all participants, and is the method of adding them to the model appropriate? |
| Applicability | Did the prognostic factors adjusted for match the review question? |
| Domain 6: Selective reporting | |
| Risk of bias | Is the reported estimate likely to be selected on the basis of the results from: multiple outcome measurements, multiple analyses of the prognostic factor‐outcome relationship, and/or from different subgroups? |
Appendix 4. Domains to be considered when judging the strength of the body of evidence
We will consider the following domains when we assess the strength of the body of evidence, based on the GRADE approach (Guyatt 2008).
| Domain | Explanation |
| Risk of bias | Based on results of ‘Risk of bias' assessments, we will downgrade confidence in the evidence base if most evidence is from studies that we judge to be at high risk of bias. |
| Indirectness | We will downgrade confidence in the evidence base if we have concerns that the study sample, the prognostic factor, the outcome, and/or the other factors in the models in the primary studies do not reflect the review question. |
| Inconsistency | We will downgrade confidence in the evidence base if there is unexplained heterogeneity or variability in results across studies. |
| Imprecision | We will downgrade confidence in the evidence base if the estimate of the effect size from a meta‐analysis is not precise or, if no meta‐analysis is performed, if the estimate of the size of effect from individual studies is not precise. |
| Publication bias | Studies showing no association are likely to be unpublished, unless part of a larger study that specifically aimed to compare tests. We will downgrade our confidence in the evidence base if we have reason to suspect publication bias from our assessments of reporting bias. |
| Size of effect | We will upgrade our confidence in the evidence base if the size of effect is moderate or large. If a meta‐analysis is not possible, we will upgrade if the size of effect is moderate or large for most included studies. |
Contributions of authors
AM drafted the prognosis sections of the protocol. JPTH provided methodological guidance. SJ, SB, and KMK provided clinical guidance. AK and LV drafted the economic sections of the protocol. SD provided guidance on searching. AM, AH, AK, CLF, SD, CW, SJ, SB, LV, JPTH and KMK commented on and approved the protocol.
Sources of support
Internal sources
University of Bristol, UK.
External sources
-
National Institute for Health Research (NIHR), UK.
This work is supported by a NIHR Systematic Reviews Cochrane Programme Grant 16/144
-
Cancer Research UK, UK.
AM and JPTH are supported in part by Cancer Research UK (grant number C18281/A19169).
The views and opinions expressed are those of the review authors and do not necessarily reflect those of the NIHR, its Systematic Reviews Programme, the National Health Service (NHS), the Department of Health, or Cancer Research UK.
Declarations of interest
Alexandra McAleenan: none known Amy Howell: none known Ashleigh Kernohan: none known Claire L Faulkner: none known Sarah Dawson: none known Christopher Wragg: none known Sarah Jefferies: member of the National Institute for Health and Care Excellence (NICE) Primary Brain Tumours Guideline Committee Sebastian Brandner: member of the NICE Primary Brain Tumours Guideline Committee Luke Vale: none known Julian PT Higgins: none known Kathreena M Kurian: none known
New
References
Additional references
- Altman DG. Systematic reviews of evaluations of prognostic variables. BMJ 2001;323(7306):224‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Binabaj MM, Bahrami A, ShahidSales S, Joodi M, Joudi Mashhad M, Hassanian SM, et al. The prognostic value of MGMT promoter methylation in glioblastoma: a meta‐analysis of clinical trials. Journal of Cellular Physiology 2018;233(1):378‐86. [DOI] [PubMed] [Google Scholar]
- Brandner S, Deimling A. Diagnostic, prognostic and predictive relevance of molecular markers in gliomas. Neuropathology and Applied Neurobiology 2015;41(6):694‐720. [DOI] [PubMed] [Google Scholar]
- Burnet NG, Jefferies SJ, Benson RJ, Hunt DP, Treasure FP. Years of life lost (YLL) from cancer is an important measure of population burden and should be considered when allocating research funds. British Journal of Cancer 2005;92(2):241‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Debray TPA, Moons KGM, Riley RD. Detecting small‐study effects and funnel plot asymmetry in meta‐analysis of survival data: a comparison of new and existing tests. Research Synthesis Methods 2018;9(1):41‐50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dullea A, Marignol L. MGMT testing allows for personalised therapy in the temozolomide era. Tumor Biology 2016;37(1):87‐96. [DOI] [PubMed] [Google Scholar]
- Thomas J, Brunton J, Graziosi S. EPPI‐Reviewer 4: software for research synthesis. EPPI‐Centre Software. London: Social Science Research Unit, UCL Institute of Education, 2010.
- Esteller M, Garcia‐Foncillas J, Andion E, Goodman SN, Hidalgo OF, Vanaclocha V, et al. Inactivation of the DNA‐repair gene MGMT and the clinical response of gliomas to alkylating agents. New England Journal of Medicine 2000;343(19):1350‐4. [DOI] [PubMed] [Google Scholar]
- Evers S, Goossens M, Vet H, Tulder M, Ament A. Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria. International Journal of Technology Assessment in Health Care 2005;21(2):240‐5. [PubMed] [Google Scholar]
- Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck‐Ytter Y, Alonso‐Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336(7650):924‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hayden JA, Windt DA, Cartwright JL, Côté P, Bombardier C. Assessing bias in studies of prognostic factors. Annals of Internal Medicine 2013;158(4):280‐6. [DOI] [PubMed] [Google Scholar]
- Hegi ME, Diserens AC, Godard S, Dietrich PY, Regli L, Ostermann S, et al. Clinical trial substantiates the predictive value of O‐6‐methylguanine‐DNA methyltransferase promoter methylation in glioblastoma patients treated with temozolomide. Clinical Cancer Research 2004;10(6):1871‐4. [DOI] [PubMed] [Google Scholar]
- Hegi ME, Diserens AC, Gorlia T, Hamou MF, Tribolet N, Weller M, et al. MGMT gene silencing and benefit from temozolomide in glioblastoma. New England Journal of Medicine 2005;352(10):997‐1003. [DOI] [PubMed] [Google Scholar]
- Hegi ME, Stupp R. Withholding temozolomide in glioblastoma patients with unmethylated MGMT promoter‐still a dilemma?. Neuro‐Oncology 2015;17(11):1425‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Higgins JP, Thompson SG. Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine 2002;21(11):1539‐58. [DOI] [PubMed] [Google Scholar]
- Huguet A, Hayden JA, Stinson J, McGrath PJ, Chambers CT, Tougas ME, et al. Judging the quality of evidence in reviews of prognostic factor research: adapting the GRADE framework. Systematic Reviews 2013;2:71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. BMJ 2013;346:f1049. [DOI] [PubMed] [Google Scholar]
- James Lind Alliance Priority Setting Partnerships. Neuro‐oncology Top 10. www.jla.nihr.ac.uk/priority‐setting‐partnerships/neuro‐oncology/top‐10‐priorities/ (accessed 29 May 2018).
- Louis DN, Perry A, Reifenberger G, Deimling A, Figarella‐Branger D, Cavenee W, et al. The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary. Acta Neuropathologica 2016;131(6):803‐20. [DOI] [PubMed] [Google Scholar]
- Malmström A, Grønberg BH, Marosi C, Stupp R, Frappaz D, Schultz H, et al. Temozolomide versus standard 6‐week radiotherapy versus hypofractionated radiotherapy in patients older than 60 years with glioblastoma: the Nordic randomised, phase 3 trial. Lancet Oncology 2012;13(9):916‐26. [DOI] [PubMed] [Google Scholar]
- Moons KGM, Groot JAH, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLOS Medicine 2014;11(10):e1001744. [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Institute for Health and Care Excellence. Brain tumours (primary) and brain metastases in adults. NICE guideline [NG99]. Published date: July 2018. www.nice.org.uk/guidance/ng99 (accessed 11 July 2018).
- Olesen J, Gustavsson A, Svensson M, Wittchen HU, Jönsson B, CDBE2010 study group, European Brain Council. The economic cost of brain disorders in Europe. European Journal of Neurology 2012;19(1):155‐62. [DOI] [PubMed] [Google Scholar]
- Ostrom QT, Bauchet L, Davis FG, Deltour I, Fisher JL, Langer CE, et al. The epidemiology of glioma in adults: a "state of the science" review. Neuro‐Oncology 2014;16(7):896‐913. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Parmar MK, Torri V, Stewart L. Extracting summary statistics to perform meta‐analyses of the published literature for survival endpoints. Statistics in Medicine 1998;17(24):2815‐34. [DOI] [PubMed] [Google Scholar]
- Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS‐I: a tool for assessing risk of bias in non‐randomized studies of interventions. BMJ 2016;355:i4919. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stupp R, Mason WP, Bent MJ, Weller M, Fisher B, Taphoorn MJB, et al. Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. New England Journal of Medicine 5005;352:987‐96. [DOI] [PubMed] [Google Scholar]
- Sun MZ, Oh T, Ivan ME, Clark AJ, Safaee M, Sayegh ET, et al. Survival impact of time to initiation of chemoradiotherapy after resection of newly diagnosed glioblastoma. Journal of Neurosurgery 2015;122(5):1144‐50. [DOI] [PubMed] [Google Scholar]
- Tierney JF, Stewart LA, Ghersi D, Burdett S, Sydes MR. Practical methods for incorporating summary time‐to‐event data into meta‐analysis. Trials 2007;8:16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weller M, Bent M, Tonn JC, Stupp R, Preusser M, Cohen‐Jonathan‐Moyal E, et al. European Association for Neuro‐Oncology (EANO) guideline on the diagnosis and treatment of adult astrocytic and oligodendroglial gliomas. Lancet Oncology 2017;18(6):e315‐29. [DOI] [PubMed] [Google Scholar]
- Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks, JJ, Reitsma JB, et al. QUADAS‐2: a revised tool for the quality assessment of diagnostic accuracy studies. Annals of Internal Medicine 2011;155(8):529‐36. [DOI] [PubMed] [Google Scholar]
- Whiting P, Savović J, Higgins JP, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. Journal of Clinical Epidemiology 2016;69:225‐34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wick W, Platten M, Meisner C, Felsberg J, Tabatabai G, Simon M, et al. Temozolomide chemotherapy alone versus radiotherapy alone for malignant astrocytoma in the elderly: the NOA‐08 randomised, phase 3 trial. Lancet Oncology 2012;13(7):707‐15. [DOI] [PubMed] [Google Scholar]
- Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Annals of Internal Medicine 2019;170(1):51‐8. [DOI] [PubMed] [Google Scholar]
- Zhang K, Wang XQ, Zhou B, Zhang L. The prognostic value of MGMT promoter methylation in Glioblastoma multiforme: a meta‐analysis. Familial Cancer 2013;12(3):449‐58. [DOI] [PubMed] [Google Scholar]
- Zhao H, Wang S, Song C, Zha Y, Li L. The prognostic value of MGMT promoter status by pyrosequencing assay for glioblastoma patients' survival: a meta‐analysis. World Journal of Surgical Oncology 2016;14:261. [DOI] [PMC free article] [PubMed] [Google Scholar]