Skip to main content
NCBI home page
The Cochrane Database of Systematic Reviews logoLink to The Cochrane Database of Systematic Reviews
. 2019 Sep 17;2019(9):CD013431. doi: 10.1002/14651858.CD013431

Immune checkpoint inhibitors for unresectable hepatocellular carcinoma

Omar Abdel‐Rahman 1,✉, Zeinab Elsayed 2
PMCID: PMC6748353

Abstract

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

To assess the benefits and harms of immune checkpoint inhibitors versus placebo, no treatment, or other systemic or locoregional therapies for people with unresectable hepatocellular carcinoma.

Background

Description of the condition

Liver cancer is a global health problem and a major cause of cancer‐related mortality and morbidity. Unfotunately, the incidence of liver cancer has been increasing over the past two decades (GBD Liver Cancer Collaboration 2017). Hepatocellular carcinoma is the most common form of liver cancer. The most common risk factors for hepatocellular carcinoma include hepatitis B, hepatitis C, alcoholic liver disease, and non‐alcoholic fatty liver disease (Sanyal 2010; Schiefelbein 2012).

Following the geographical distribution of these risk factors, there is a considerable geographical variability with regards to the incidence of hepatocellular carcinoma (Masuzaki 2009). While hepatitis B plays a principal role in the risk of developing hepatocellular carcinoma within Asian and Western Pacific regions, hepatitis C and alcoholism are more important in the development of hepatocellular carcinoma in the Middle East and Europe (Kwon 2012). Other factors which might increase the risk of developing hepatocellular carcinoma include aflatoxin exposure, smoking, and rare genetic diseases like Wilson's disease and haemochromatosis (Abdel‐Rahman 2017).

Contrary to the majority of other solid tumours, most cases of hepatocellular carcinoma can be diagnosed through non‐invasive diagnostic methods without the need for a biopsy. According to guidelines from the European Association for the Study of the Liver (EASL) and the American Association for the Study of Liver Diseases (AASLD) (EASL 2018; Heimbach 2018), diagnosis of hepatocellular carcinoma can be established if well‐defined radiological criteria are fulfilled. Biopsy should only be used if diagnosis cannot be established radiologically.

While the American Joint Committee on Cancer staging system is the most commonly used staging system for the majority of solid tumours, it is less frequently used in the assessment of people with hepatocellular carcinoma (Abdel‐Rahman 2018). Other staging systems which incorporate patient‐related factors (including severity of chronic liver disease and performance status) are more commonly used. Examples of these staging systems include the Barcelona Clinic Liver Cancer system and Okuda system (Okuda 1985; Levy 2002). Of these systems, the Barcelona Clinic Liver Cancer system is, by far, the most widely used among physicians dealing with hepatocellular carcinoma worldwide as it combines patient‐related factors (Eastern Cooperative Oncology Group performance score), tumour‐related factors, and liver disease‐related factors (Child‐Pugh score). The use of the Barcelona Clinic Liver Cancer staging system has been endorsed by many international organisations, including the AASLD and EASL (EASL 2018; Heimbach 2018).

Treatment options for hepatocellular carcinoma depend on the resectability status of the tumour and its host. For people with potentially resectable disease, options include liver resection, liver transplantation, radiofrequency ablation, and microwave ablation. Only 20% of people with hepatocellular carcinoma might be eligible for potentially curative treatment options (including resection and transplantation). Unfortunately, after liver resection, disease recurrence can still be shown in 50% to 70% of instances within five years (Berzigotti 2015). For people with locally advanced disease, transarterial chemoembolisation and radioembolisation are the traditional interventions offered (Abdel‐Rahman 2013), although evidence‐based support for these is questionable (Oliveri 2011; Forner 2012). Systemic therapy is used for people with advanced disease. Sorafenib (and more recently lenvatinib) is considered the standard first‐line systemic treatment for people with advanced hepatocellular carcinoma. This consideration is based on the results from a number of landmark randomised trials. In the SHARP trial, sorafenib was compared with placebo among people with advanced hepatocellular carcinoma, and it improved overall survival, with a hazard ratio of 0.69 (95% confidence interval (CI) 0.55 to 0.87) (Llovet 2008). More recently, lenvatinib has been shown to be non‐inferior to sorafenib (with regards to overall survival) in another randomised trial, with a hazard ratio of 0.92 (95% CI 0.79 to 1.06) (Kudo 2018).

Description of the intervention

Immune checkpoint inhibitors represent a class of anticancer medications which are administered intravenously and are usually given every two to three weeks (Mahoney 2015). There are two major categories of these agents: cytotoxic T‐lymphocyte associated protein‐4 (CTLA‐4) inhibitors and programmed death‐ligand 1 (PD‐L1) inhibitors. Examples of the former group include ipilimumab and tremelimumab, while examples of the latter group include pembrolizumab, nivolumab, atezolizumab, avelumab, and durvalumab (Buchbinder 2015). While CTLA‐4 inhibitors have shown activity in the management of advanced malignant melanoma, PD‐(L)1 inhibitors have proved their efficacy in the treatment of a number of hard‐to‐treat solid tumours including malignant melanoma, non‐small cell lung cancer, and renal cell carcinoma (Mier 2015). Previous studies in other solid tumours have evaluated the use of these agents as monotherapy, immunotherapy combination (i.e. CTLA‐4 inhibitor plus PD‐(L)1 inhibitor), and chemo‐immunotherapy combination (i.e. combination of an immune checkpoint inhibitor plus chemotherapy) (Abdel‐Rahman 2016). Additional ongoing studies in a number of solid tumours are evaluating immune checkpoint inhibitors in combination with external beam radiotherapy.

How the intervention might work

Following T‐cell activation, a number of inhibitory receptors (including CTLA‐4 and PD‐1 proteins) work to limit the over‐activation of T cells. While CTLA‐4 receptor works primarily in the priming phase, PD‐1 works essentially in the effector phase of T‐cell response (Granier 2017). In some chronic diseases (including cancer), prolonged exposure of T cells to cancer antigens leads to a phenomenon called "exhaustion" which can be defined simply as the decline of the effector function of T cells (Wherry 2003). Binding of inhibitors of CTLA‐4 or PD‐1, or both, to their respective receptors would remove the inhibitory effect of these receptors. This has been observed to restore the anti‐cancer effects of T cells, and thus reverse the phenomenon of exhaustion (Ahn 2018).

The reactivation of T cells by immune checkpoint inhibitors might bring about a host of unwanted adverse effects. This is related to the fact that stimulated T cells might attack some normal tissues, just as they would attack cancer cells. This could lead to a number of immune‐related adverse events which might include endocrine adverse events, pneumonitis, skin rash, colitis, neurological and musculoskeletal toxicities (among many other adverse effects) (Abdel‐Rahman 2017a; Eltobgy 2017; Xu 2018).

Why it is important to do this review

There is increasing interest in the use of immune checkpoint inhibitors for the management of solid tumours, including hepatocellular carcinoma (both in localised and advanced stages). This interest has been driven by the overall poor prognosis of the disease as well as the modest outcomes of available systemic and local therapy options. Early‐phase studies have suggested a reasonable activity of these agents in the management of advanced disease (El‐Khoueiry 2017); however, many international authorities and regulatory bodies are reluctant to adopt these agents within the treatment algorithm of hepatocellular carcinoma. Given the very high costs of immune checkpoint inhibitors, and an apparent lack of meta‐analyses or systematic reviews on these agents for unresectable hepatocellular carcinoma, it is necessary to undertake a thorough assessment of the benefits and harms of immune checkpoint inhibitors in the context of hepatocellular carcinoma management.

Objectives

To assess the benefits and harms of immune checkpoint inhibitors versus placebo, no treatment, or other systemic or locoregional therapies for people with unresectable hepatocellular carcinoma.

Methods

Criteria for considering studies for this review

Types of studies

We will include all randomised clinical trials comparing immune checkpoint inhibitors versus placebo, no treatment, or systemic or locoregional therapies (whether alone or in combination) for unresectable hepatocellular carcinoma. We will not place limitations on language, format, or year of publication.

Types of participants

We will include adults (aged 18 and over) with histologically or radiologically diagnosed unresectable hepatocellular carcinoma. Radiological diagnosis of hepatocellular carcinoma should have been done using validated radiological criteria (e.g. AASLD criteria).

Types of interventions

Experimental intervention: immune checkpoint inhibitors.

Control intervention: placebo, no treatment, or systemic or locoregional therapies.

We will allow cointerventions if they were administered equally to all trial intervention groups.

Types of outcome measures

Primary outcomes

  • All‐cause mortality up to one year.

  • Health‐related quality of life at 12 weeks (as reported by the participants and as assessed by standard grading systems) (e.g. FACT‐Hep 2007).

  • Proportion of participants with one or more serious adverse event, as defined by the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) Guideline for Good Clinical Practice as any untoward medical occurrence that at any dose resulted in death, was life‐threatening, required hospitalisation or prolongation of existing hospitalisation, or resulted in persistent or significant disability or incapacity, or was a congenital anomaly/birth defect, or any medical event that might have jeopardised the person, or required intervention to prevent it (ICH‐GCP 1997).

Secondary outcomes

  • Time to progression of the tumour. We will analyse this outcome according to time‐to‐event data analysis guidelines described below (Measures of treatment effect).

  • Tumour response assessments (as recommended by the response evaluation in solid tumours criteria) (Eisenhauer 2009).

    • Proportion of people with complete response: disappearance of all target lesions. Any pathological lymph nodes (whether target or non‐target) must have reduction in short axis to less than 10 mm.

    • Proportion of people with partial response: at least a 30% decrease in the sum of diameters of target lesions, taking as reference the baseline sum diameters.

    • Proportion of people with progressive disease: at least a 20% increase in the sum of diameters of target lesions, taking as reference the smallest sum on study (this includes the baseline sum if that was the smallest on study). In addition to the relative increase of 20%, the sum must also have demonstrated an absolute increase of at least 5 mm (note: the appearance of one or more new lesions is also considered as progression).

    • Proportion of people with stable disease: neither sufficient shrinkage to qualify for partial response nor sufficient increase to qualify for progressive disease, taking as reference the smallest sum diameters while on study.

    • In addition, we will consider tumour response assessments according to the immune response evaluation criteria in solid tumours (Seymour 2017).

  • Proportion of participants with one or more non‐serious adverse event: we will consider as non‐serious events, any medical occurrences not necessarily having a causal relationship with the treatment that do not fulfil the criteria of serious adverse events clarified above (ICH‐GCP 1997)

Exploratory outcomes

  • Individual serious adverse events.

  • Individual adverse events considered not to be serious.

We will not use outcomes, reported or mentioned in the trials, as criteria for inclusion of studies in the review.

Search methods for identification of studies

Electronic searches

We will search the Cochrane Hepato‐Biliary Group (CHBG) Controlled Trials Register (to be searched internally by the CHBG Information Specialist via the Cochrane Register of Studies Web), the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library, MEDLINE Ovid, Embase Ovid, LILACS, Science Citation Index Expanded, and Conference Proceedings Citation Index – Science (Royle 2003). We will search all the listed databases from their date of inception onwards. Appendix 1 gives the preliminary search strategies with the expected time spans of the searches.

Searching other resources

We will check reference lists of primary original studies and review articles manually for further related articles (cross‐references). We will search online trial registries such as ClinicalTrials.gov (clinicaltrials.gov/), European Medicines Agency (EMA) (www.ema.europa.eu/ema/), WHO International Clinical Trial Registry Platform (www.who.int/ictrp), the Food and Drug Administration (FDA) (www.fda.gov), as well as pharmaceutical company sources for ongoing or unpublished trials. We will also examine the lists of references of identified studies in order to identify additional studies, as well as contact the main authors of studies and content experts for further unpublished or ongoing studies. We will also search for grey literature in the System for Information on Grey Literature in Europe “OpenGrey” (www.opengrey.eu/).

Data collection and analysis

We will perform the review according to the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011a). We will perform data analyses using the Cochrane statistical software, Review Manager 5.3 (Review Manager 2014). We will resolve any discordance by consensus. If we cannot reach consensus and no other person joins us during the review preparation, we will ask the contact editor of the review to arbitrate.

Selection of studies

The review authors, independently of each other, will identify the trials for inclusion.

We will exclude duplicate records based on review of titles. We will review abstracts of the remaining articles. We will exclude studies using duplicate patient data sets. We will go through the full text of the remaining articles for relevancy to the review. We will list the excluded studies and the reasons for exclusion.

If, during the selection of trials, we identify observational studies (e.g. quasi‐randomised studies, cohort studies, or case reports) that report adverse events during the study period, we will include these studies for a review of the reported adverse events only. We will not specifically search for observational studies for inclusion in this review, which is a limitation. We are conscious that by not looking for all observational studies on adverse events, we allow the risks of putting more emphasis on potential benefits than on potential harms, and of overlooking uncommon and late adverse events (Storebø 2018). We will not analyse the extracted data on harms from non‐randomised clinical studies together with the data on harms from randomised clinical trials included in the review; neither we will assess the risk of bias in these studies. However, at the end of the 'Results' section we will refer to the extracted narrative data on harms (with a link to the table containing this information), or we may present a narrative analysis.

Data extraction and management

We will extract the data individually. We will extract details of study population, interventions, and outcomes using a piloted, standardised data extraction form. This form will include the following items:

  • publication year;

  • country;

  • year the trial was conducted;

  • inclusion and exclusion criteria;

  • whether the investigators performed sample size calculation;

  • population characteristics such as age and sex ratio;

  • baseline characteristics including Child‐Pugh score, Eastern Cooperative Oncology Group Performance Status, Barcelona Clinic Liver Cancer stage, proportion of participants positive for hepatitis B and hepatitis C virus;

  • experimental and control arms of the trial;

  • outcomes (see Types of outcome measures);

  • risk of bias (see Assessment of risk of bias in included studies);

  • whether the study performed an intention‐to‐treat analysis;

  • events reported in observational studies. We will present the data on harm from observational studies (quasi‐randomised and cohorts), case reports, or letters to the editors — if retrieved during our searches for randomised trials — only in a narrative format.

If information is lacking, we will attempt to contact study authors to obtain it.

We will discuss any disagreements in data extraction and management with each other. If we cannot reach consensus, we will ask the contact editor of our protocol to arbitrate.

Assessment of risk of bias in included studies

The review authors, independently of each other, will assess the risk of bias of each potentially included trial according to the recommendations in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011a). We will use the following definitions in the assessment of risk of bias (Schulz 1995; Moher 1998; Kjaergard 2001; Wood 2008; Savović 2012a; Savović 2012b; Savović 2018).

Allocation sequence generation

  • Low risk of bias: sequence generation was achieved using computer random number generation or a random number table. Drawing lots, tossing a coin, shuffling cards, and throwing dice were adequate if performed by an independent person not otherwise involved in the trial.

  • Unclear risk of bias: the method of sequence generation was not specified.

  • High risk of bias: the sequence generation method was not random. We will only include such studies for assessment of harms.

Allocation concealment

  • Low risk of bias: the participant allocations could not have been foreseen in advance of, or during, enrolment. Allocation was controlled by a central and independent randomisation unit. The allocation sequence was unknown to the investigators (e.g. if the allocation sequence was hidden in sequentially numbered, opaque, and sealed envelopes).

  • Unclear risk of bias: the method used to conceal the allocation was not described so that intervention allocations may have been foreseen in advance of, or during, enrolment.

  • High risk of bias: the allocation sequence was likely to be known to the investigators who assigned the participants. We will only include such studies for assessment of harms.

Blinding of participants and personnel

  • Low risk of bias: it was mentioned that both participants and personnel providing the interventions were blinded, and the method of blinding was described, so that knowledge of allocation was prevented during the trial.

  • Unclear risk of bias: it was not mentioned if the trial was blinded, or the trial was described as blinded, but the method or extent of blinding was not described, so that knowledge of allocation was possible during the trial.

  • High risk of bias: the trial was not blinded, so that the allocation was known during the trial.

Blinded outcome assessment

  • Low risk of bias: outcome assessment was carried out blinded for all relevant outcomes, and the method of blinding was described, so that knowledge of allocation was prevented.

  • Unclear risk of bias: blinding of outcome assessment was not described, or the outcome assessment was described as blinded, but the method of blinding was not described, so that knowledge of allocation was possible.

  • High risk of bias: outcome assessment was not blinded, so that the allocation was known to outcome assessors.

Incomplete outcome data

  • Low risk of bias: missing data were unlikely to make treatment effects depart from plausible values. Sufficient methods, such as multiple imputation, were employed to handle missing data.

  • Unclear risk of bias: there was insufficient information to assess whether missing data, in combination with the method used to handle missing data, were likely to induce bias on the results.

  • High risk of bias: the results were likely to be biased due to missing data.

Selective outcome reporting

  • Low risk of bias: the trial reported the following pre‐defined primary outcomes: all‐cause mortality, serious adverse events, and time to progression of the tumour. If the original trial protocol was available, the outcomes reported in the study report should be those called for in that protocol. If the trial protocol was obtained from a trial registry (e.g. www.clinicaltrials.gov), the outcomes sought should be those enumerated in the original protocol if the trial protocol was registered before or at the time that the trial was begun. If the trial protocol was registered after the trial was begun, those outcomes would not be considered reliable.

  • Unclear risk of bias: the study authors did not report all predefined outcomes fully, or it was unclear whether the study authors recorded data on these outcomes or not.

  • High risk of bias: the study authors did not report one or more pre‐defined outcomes.

Other bias

  • Low risk of bias: the trial appeared to be free of other factors that could put it at risk of bias.

  • Unclear risk of bias: the trial may or may not have been free of other factors that could put it at risk of bias.

  • High risk of bias: there were other factors in the trial that could put it at risk of bias.

Overall risk of bias

We will assess the overall risk of bias for each trial as follows.

  • Low risk of bias: if all the above 'Risk of bias' sources are assessed as being at low risk of bias.

  • High risk of bias: if one or more of the above 'Risk of bias' sources are assessed as being at unclear or high risk of bias.

We will assess blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective outcome reporting for each outcome. Thus, we will be able to assess the bias risk for each outcome in addition to each trial.

We will discuss any disagreements with each other. If we cannot reach consensus, we will ask the contact editor of our protocol to arbitrate.

Measures of treatment effect

For dichotomous variables, we will calculate relative risks (RRs) with 95% confidence intervals (CIs). For continuous variables, we will calculate the mean difference (MD) or standardised mean difference (SMD) with 95% CIs. Time‐to‐event data analyses are usually based on hazard ratios (HRs); so when these data are provided, we will use HRs. If HRs are not provided, we plan to use the method suggested in Parmar 1998 to extract HRs from the published data. In case the results of health‐related quality of life are reported through different scales in different studies, we will report the results of each study independently, and the results will not be combined together.

Unit of analysis issues

The unit of analysis is the participant undergoing treatment for unresectable hepatocellular carcinoma according to the intervention group to which the participant is randomly assigned. In the case of cross‐over trials, we will use the outcome data after the period of first intervention because the assigned treatments could have residual effects. Due to the clinical situation, we do not expect to find cluster‐randomised trials. In case of trials with multiple intervention groups, we will collect data for all trial intervention groups that meet our inclusion criteria. We may have to divide the control group into two to avoid double‐counting in case this was a common comparator.

Dealing with missing data

If we are unable to extract data from the text, or a statistic is missing or unclear, we will contact the authors of the original article to request the necessary information. If we receive no information from the authors, then we will report this statistic as missing. We will then base our analysis on the number of participants at the time of last follow‐up. We will use an intention‐to‐treat analysis for the outcomes all‐cause mortality up to one year, and proportion of participants with one or more serious adverse events. We will include participants with incomplete or missing data in sensitivity analyses by imputing them according to the following two scenarios (Hollis 1999).

  • Extreme‐case analysis favouring the experimental intervention (best‐worst case scenario: none of the dropouts/participants lost from the experimental arm, but all of the dropouts/participants lost from the control arm, will be assumed to have experienced the outcome, including all randomised participants in the denominator.

  • Extreme‐case analysis favouring the control (worst‐best case scenario): all dropouts/participants lost from the experimental arm, but none from the control arm, will be assumed to have experienced the outcome, including all randomised participants in the denominator.

In addition, we will perform an as‐treated/per‐protocol analysis (Higgins 2011b). We will only use the data available for the outcomes time to progression of the tumour, and health‐related quality of life.

Assessment of heterogeneity

We will use the Chi2 test to explore between‐trial statistical heterogeneity (P value less than 0.1). In addition, we will quantify the degree of heterogeneity observed in the results using the I2 statistic, which can be interpreted as the percentage of variation observed between the trials that is attributable to between‐trial differences rather than sampling error (chance). We will interpret I2 as suggested in Higgins 2011a: 0% to 40%: might not be important; 30% to 60%: might represent moderate heterogeneity; 50% to 90%: might represent substantial heterogeneity; 75% to 100%: considerable heterogeneity. In case of considerable heterogeneity, we will explore the possible sources by means of sensitivity analyses (Higgins 2011a).

Assessment of reporting biases

We will carry out a comprehensive search in order to minimise publication bias. If we identify a sufficient number of trials for inclusion (at least 10 trials), we will use a funnel plot to explore such bias (Egger 1997; Macaskill 2001).

Data synthesis

If we judge the trials to be sufficiently similar in terms of participants, interventions, comparators, and outcome assessment, we will conduct a meta‐analysis. We will use a random‐effects analysis because we expect that the included trials will be heterogenous.

We will produce summary estimates of the treatment effect using Review Manager 5 (Review Manager 2014) and will present data using forest plots, where possible. For time‐to‐event data, we will plot and meta‐analyse estimates of HRs and 95% CIs as presented in the study reports, using the generic inverse variance method in Review Manager 5 (Review Manager 2014).

Subgroup analysis and investigation of heterogeneity

We plan to perform the following subgroup analyses, in order to investigate the potential clinical relevance of these subgroups in defining the outcomes of participants with hepatocellular carcinoma treated with immune checkpoint inhibitors versus comparator arms.

  • Trials at low risk of bias compared to trials at high risk of bias (because trials at high risk of bias tend to overestimate intervention effects).

  • Type of immune checkpoint inhibitor (because this is a well‐established prognostic indicator for people having hepatocellular carcinoma).

  • Trials including participants with Child‐Pugh A liver cirrhosis (defined as a Child‐Pugh score of five to six) compared to trials including participants with Child‐Pugh B liver cirrhosis (defined as a Child‐Pugh score of seven to nine) (because this is a well‐established prognostic indicator for people having hepatocellular carcinoma) (Pugh 1973).

  • Eastern Cooperative Oncology Group score of zero compared to a score of one or two (because this is a well‐established prognostic indicator for people having hepatocellular carcinoma).

  • PD‐L1 level of expression (among patients receiving PD‐L1 inhibitors) (because this is a well‐established prognostic indicator for people having hepatocellular carcinoma).

Sensitivity analysis

We may perform sensitivity analyses if clinical and methodological variations are identified, such as tumour response assessments, adequacy of allocation concealment, incomplete reporting of first primary outcome (HRs for death, estimated from the text or acquired directly from study authors). In addition, to assess the robustness of the results obtained with the random‐effects meta‐analysis model, we will also conduct a sensitivity analysis using a fixed‐effect approach. In case of divergence of the results obtained with the two models, we will present both results; otherwise, we will report only results obtained with the random‐effects model.

Trial Sequential Analysis

We will examine apparently significant beneficial and harmful intervention effects, as well as neutral effects, with Trial Sequential Analyses in order to evaluate if these apparent effects could be caused by random error ('play of chance') (Brok 2008; Wetterslev 2008; Brok 2009; Thorlund 2009; Wetterslev 2009; Thorlund 2010; Thorlund 2011; TSA 2011; Wetterslev 2017).

We will apply Trial Sequential Analysis as cumulative meta‐analyses are at risk of producing random errors due to sparse data and repetitive testing of the accumulating data (Wetterslev 2008; Wetterslev 2009). To control random errors, we will calculate the required information size (i.e. the number of participants needed in a meta‐analysis to detect or reject a certain intervention effect). The required information size calculation should also account for the diversity present in the meta‐analysis (Wetterslev 2009). In our meta‐analysis, the diversity‐adjusted required information size will be based on the event proportion in the control group; assumption of a plausible RR reduction of 20% or the RR reduction observed in the included trials with low risk of bias; a risk of type I error of 2.5% due to three primary outcomes and 2.5% due to the three secondary outcomes; a risk of type II error of 20%; and the diversity of the meta‐analysis. The underlying assumption of Trial Sequential Analysis is that testing for significance may be performed each time a new trial is added to the meta‐analysis. We will add the trials according to the year of publication, and, if more than one trial was published in a year, we will add trials alphabetically according to the last name of the first author. On the basis of the diversity‐adjusted required information size, trial sequential monitoring boundaries will be constructed (Thorlund 2011). These boundaries will determine the statistical inference one may draw regarding the cumulative meta‐analysis that has not reached the required information size. If the cumulative Z‐curve crosses the trial sequential monitoring boundary for benefit or harm before the diversity‐adjusted required information size is reached, firm evidence may perhaps be established and further trials may be superfluous. In contrast, if the boundary is not surpassed, it is most probably necessary to continue doing trials in order to detect or reject a certain intervention effect. This could be determined by assessing if the cumulative Z‐curve crossed the trial sequential monitoring boundaries for futility.

We will use Trial Sequential Analysis (TSA 2011; Castellini 2018) to assess imprecision for the following outcomes: all‐cause mortality at one year; health‐related quality of life; proportion of participants with one or more serious adverse events; time to progression of the tumour; tumour response assessments; and proportion of participants with one or more non‐serious adverse events, and we will compare the result with the GRADE assessment of imprecision (GRADEpro GDT).

'Summary of findings' tables

We will create 'Summary of findings' tables using GRADEpro GDT software (GRADEpro GDT), in which we will present the evidence for the outcomes all‐cause mortality at one year; health‐related quality of life; proportion of participants with one or more serious adverse event; time to progression of the tumour; tumour response assessments; and proportion of participants with one or more non‐serious adverse event. If no data are reported on any of these outcomes, we will still present them in the table. We will use the definitions outlined in Table 1. We will evaluate the certainty of the evidence for outcomes reported in the review, considering within‐study risk of bias; indirectness of evidence (population, intervention, control, outcomes); unexplained heterogeneity or inconsistency of results (including problems with subgroup analyses); imprecision of results (wide CIs); and probability of publication bias (Balshem 2011; Guyatt 2011a; Guyatt 2011b; Guyatt 2011c; Guyatt 2011d; Guyatt 2011e; Guyatt 2011f; Guyatt 2011g; Guyatt 2011h; Guyatt 2013a; Guyatt 2013b; Guyatt 2013c; Guyatt 2013d; Mustafa 2013; Guyatt 2017).

Table 1.

Explanations of terms used in the 'Summary of findings' table

Outcomes The tables provide the findings for the most important outcomes for someone making a decision. These include potential benefits and harms, and whether the included studies provide data for these outcomes or not. Additional findings may be reported elsewhere in the review.
Assumed control group risk Assumed control‐group risks can be based either on the control‐group risks reported in the included studies or on epidemiological data from elsewhere. When only one control‐group risk is provided, it is normally the median control‐group risk across the studies that provided data for that outcome. Risk is the probability of an outcome occurring. The control‐group risk is the risk of an outcome occurring in the comparison group (without the intervention).
Corresponding intervention group risk Risk is the probability of an outcome occurring. The intervention‐group risk is the risk of an outcome occurring in the group receiving the intervention.
Relative effect Relative effect or risk ratio (RR) Relative effects are ratios. Here the relative effect is expressed as a risk ratio. Risk is the probability of an outcome occurring. A RR is the ratio between the risk in the intervention group and the risk in the control group. If the risk in the control group is 10% (100 per 1000) and the risk in the intervention group is 1% (10 per 1000), the RR is 10/100 or 0.10. If the RR is exactly 1.0, this means there is no difference between the occurrence of the outcome in the intervention and the control group. It is unusual for the RR to be exactly 1.0, and understanding what it means if it is above or below this value depends on whether the outcome being counted is judged to be good or bad. If the RR is greater than 1.0, the intervention increases the risk of the outcome. If it is a good outcome (e.g. the birth of a healthy baby), an RR of more than 1.0 indicates a desirable effect for the intervention. Whereas, if the outcome is bad (e.g. death), an RR of more than 1.0 would indicate an undesirable effect. If the RR is less than 1.0, the intervention decreases the risk of the outcome. This indicates a desirable effect if it is a bad outcome (e.g. death), and an undesirable effect if it is a good outcome (e.g. birth of a healthy baby).
What is the difference between absolute and relative effects?
The effect of an intervention can be described by comparing the risk of the intervention group with the risk of the control group. Such a comparison can be made in different ways. One way to compare two risks is to calculate the difference between the risks. This is the absolute effect. Consider the risk for blindness in a person with diabetes over a five‐year period. If the risk for blindness is found to be 20 in 1000 (2%) in a group of people treated conventionally and 10 in 1000 (1%) in people treated with a new drug, the absolute effect is derived by subtracting the intervention group risk from the control group risk: 2%/1% = 1%. Expressed in this way, it can be said that the new drug reduces the five‐year risk for blindness by 1% (absolute effect is 10 fewer per 1000). Another way to compare risks is to calculate the ratio of the two risks. Given the data above, the relative effect is derived by dividing the two risks, with the intervention risk being divided by the control risk: 1% ÷ 2% = ½ (0.50). Expressed in this way, as the 'relative effect', the five‐year risk for blindness with the new drug is half the risk with the conventional drug. Here the table presents risks as x per 1000 (or 100, etc.), instead of per cent, as this tends to be easier to understand. Whenever possible, the table presents the relative effect as the RR. Usually the absolute effect is different for groups that are at high and low risk, whereas the relative effect often is the same. Therefore, when it is relevant, we have reported indicative risks for groups at different levels of risk. Two or three indicative control‐group risks and the corresponding intervention‐group risks are presented when there are important differences across different populations.
Mean difference The mean difference (MD) is the average difference between the intervention group and the control group across studies. Here a weighted MD is used, which means the results of some of the studies make a greater contribution to the average than others. Studies with more precise estimates for their results (narrower confidence intervals) are given more weight. This way of measuring effect is used when combining or comparing data for continuous outcomes, such as weight, blood pressure, or pain measured on a scale. When different scales are used to measure the same outcome, e.g. different pain scales, a standardised mean difference (SMD) may be provided. This is a weighted MD standardised across studies, giving the average difference in standard deviations for the measures of that outcome.
Confidence interval A confidence interval (CI) is a range around an estimate that conveys how precise the estimate is; in this example the result is the estimate of the intervention group risk. The CI is a guide to how sure we can be about the quantity we are interested in (here the true absolute effect). The narrower the range between the two numbers, the more confident we can be about what the true value is; the wider the range, the less sure we can be. The width of the CI reflects the extent to which chance may be responsible for the observed estimate (with a wider interval reflecting more chance).
95% confidence interval As explained above, the CI indicates the extent to which chance may be responsible for the observed numbers. In the simplest terms, a 95% CI means that we can be 95% confident that the true size of effect is between the lower and upper confidence limit (e.g. 0 and 3 in the blindness drugs example mentioned above). Conversely, there is a 5% chance that the true effect is outside of this range.
Not statistically significant Statistically significant means that a result is unlikely to have occurred by chance. The usual threshold for this judgement is that the results, or more extreme results, would occur by chance with a probability of less than 0.05 if the null hypothesis (no effect) was true. When results are not statistically significant, as in this example, this is stated to alert users to the possibility that the results may have occurred by chance.
Number of participants (studies) The table provides the total number of participants across studies and the number of studies that provided data for that outcome. This indicates how much evidence there is for the outcome.
Certainty of the evidence The certainty of the evidence is a judgement about the extent to which we can be confident that the estimates of effect are correct. These judgements are made using the GRADE system, and are provided for each outcome. The judgements are based on the type of study design (randomised trials versus observational studies), the risk of bias, the consistency of the results across studies, and the precision of the overall estimate across studies. For each outcome, the evidence is rated using the following definitions:

  • high certainty: further research is very unlikely to change our confidence in the estimate of effect;

  • moderate certainty: further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate;

  • low certainty: further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate;

  • very low certainty: we are very uncertain about the estimate.
‐ A '‐' symbol indicates that the information is not relevant.
Open in a new tab

vs. = versus

We will define the levels of certainty of evidence as high, moderate, low, or very low, as follows.

  • High certainty: we are very confident that the true effect lies close to that of the estimate of the effect.

  • Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different.

  • Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect.

  • Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.

Acknowledgements

We thank the Cochrane Hepato‐Biliary Group and their supporting editorial team.

Peer reviewers: Daria Varganova, Russia; Giovanni Casazza, Italy. Contact editor: Brian Davidson, UK. Sign‐off editor: Christian Gluud, Denmark. Cochrane Abdomen and Endocrine Network editor of the review: Rachel Richardson, UK.

Cochrane Review Group funding acknowledgement: the Danish State is the largest single funder of the Cochrane Hepato‐Biliary Group through its investment in The Copenhagen Trial Unit, Centre for Clinical Intervention Research, Rigshospitalet, Copenhagen University Hospital, Denmark. Disclaimer: the views and opinions expressed in this review are those of the authors and do not necessarily reflect those of the Danish State or The Copenhagen Trial Unit.

Appendices

Appendix 1. Search strategies

Database Time span Search strategy
Cochrane Hepato‐Biliary Group Controlled Trials Register Date will be given at review stage. (((immune checkpoint or cytotoxic t‐lymphocyte associated protein‐4 or CTLA‐4 or programme death or PD) and inhibitor*) or anticancer medication* or ipilimumab or tremelimumab or pembrolizumab or nivolumab or atezolizumab or avelumab or durvalumab) AND (((liver or hepato*) and (carcinom* or cancer* or neoplasm* or malign* or tumor*)) or HCC)
Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library Latest issue #1 MeSH descriptor: [Antibodies, Monoclonal, Humanized] explode all trees
#2 (((immune checkpoint or cytotoxic t‐lymphocyte associated protein‐4 or CTLA‐4 or programme death or PD) and inhibitor*) or anticancer medication* or ipilimumab or tremelimumab or pembrolizumab or nivolumab or atezolizumab or avelumab or durvalumab)
#3 #1 or #2
#4 MeSH descriptor: [Carcinoma, Hepatocellular] explode all trees
#5 MeSH descriptor: [Liver Neoplasms] explode all trees
#6 (((liver or hepato*) and (carcinom* or cancer* or neoplasm* or malign* or tumor*)) or HCC)
#7 #4 or #5 or #6
#8 #3 and #7
MEDLINE Ovid 1946 to the date of the search 1. exp Antibodies, Monoclonal, Humanized/
2. (((immune checkpoint or cytotoxic t‐lymphocyte associated protein‐4 or CTLA‐4 or programme death or PD) and inhibitor*) or anticancer medication* or ipilimumab or tremelimumab or pembrolizumab or nivolumab or atezolizumab or avelumab or durvalumab).mp. [mp=title, abstract, original title, name of substance word, subject heading word, floating sub‐heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms]
3. 1 or 2
4. exp Carcinoma, Hepatocellular/
5. exp Liver Neoplasms/
6. (((liver or hepato*) and (carcinom* or cancer* or neoplasm* or malign* or tumor*)) or HCC).mp. [mp=title, abstract, original title, name of substance word, subject heading word, floating sub‐heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms]
7. 4 or 5 or 6
8. 3 and 7
9. (random* or blind* or placebo* or meta‐analys*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, floating sub‐heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms]
10. 8 and 9
Embase Ovid 1974 to the date of the search 1. exp immunological antineoplastic agent/
2. (((immune checkpoint or cytotoxic t‐lymphocyte associated protein‐4 or CTLA‐4 or programme death or PD) and inhibitor*) or anticancer medication* or ipilimumab or tremelimumab or pembrolizumab or nivolumab or atezolizumab or avelumab or durvalumab).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading word, candidate term word]
3. 1 or 2
4. exp liver cell carcinoma/
5. exp liver tumor/
6. (((liver or hepato*) and (carcinom* or cancer* or neoplasm* or malign* or tumor*)) or HCC).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading word, candidate term word]
7. 4 or 5 or 6
8. 3 and 7
9. (random* or blind* or placebo* or meta‐analys*).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading word, candidate term word]
10. 8 and 9
11. limit 10 to (adult <18 to 64 years> or aged <65+ years>)
LILACS (Bireme) 1982 to the date of the search (((immune checkpoint or cytotoxic t‐lymphocyte associated protein‐4 or CTLA‐4 or programme death or PD) and inhibitor$) or anticancer medication$ or ipilimumab or tremelimumab or pembrolizumab or nivolumab or atezolizumab or avelumab or durvalumab) [Words] and (((liver or hepato$) and (carcinom$ or cancer$ or neoplasm$ or malign$ or tumor$)) or HCC) [Words]
Science Citation Index Expanded (Web of Science) 1900 to the date of the search #5 #4 AND #3
#4 TS=(random* or blind* or placebo* or meta‐analys*)
#3 #2 AND #1
#2 TS=(((liver or hepato*) and (carcinom* or cancer* or neoplasm* or malign* or tumor*)) or HCC)
#1 TS=(((immune checkpoint or cytotoxic t‐lymphocyte associated protein‐4 or CTLA‐4 or programme death or PD) and inhibitor*) or anticancer medication* or ipilimumab or tremelimumab or pembrolizumab or nivolumab or atezolizumab or avelumab or durvalumab)
Conference Proceedings Citation Index – Science (Web of Science) 1990 to the date of the search #5 #4 AND #3
#4 TS=(random* or blind* or placebo* or meta‐analys*)
#3 #2 AND #1
#2 TS=(((liver or hepato*) and (carcinom* or cancer* or neoplasm* or malign* or tumor*)) or HCC)
#1 TS=(((immune checkpoint or cytotoxic t‐lymphocyte associated protein‐4 or CTLA‐4 or programme death or PD) and inhibitor*) or anticancer medication* or ipilimumab or tremelimumab or pembrolizumab or nivolumab or atezolizumab or avelumab or durvalumab)
Open in a new tab

Contributions of authors

Omar Abdel‐Rahman wrote the protocol. Zeinab Elsayed revised the protocol. Both review authors accepted the final protocol for publication.

Sources of support

Internal sources

  • None, Other.

External sources

  • None, Other.

Declarations of interest

Omar Abdel‐Rahman: none known. Zeinab Elsayed: none known.

New

References

Additional references

  1. Abdel‐Rahman O, Elsayed Z. Combination trans arterial chemoembolization (TACE) plus sorafenib for the management of unresectable hepatocellular carcinoma: a systematic review of the literature. Digestive Disease and Science 2013;58(12):3389‐96. [DOI: 10.1007/s10620-0132872-x] [DOI] [PubMed] [Google Scholar]
  2. Abdel‐Rahman O. Combination or single‐agent ipilimumab as immunotherapy of advanced melanoma: a critical review. Melanoma Management 2016;3(3):231‐43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Abdel‐Rahman O, Helbling D, Schob O, Eltobgy M, Mohamed H, Schmidt J, et al. Cigarette smoking as a risk factor for the development of and mortality from hepatocellular carcinoma: an updated systematic review of 81 epidemiological studies. Journal of Evidence‐Based Medicine 2017;10(4):245‐54. [DOI] [PubMed] [Google Scholar]
  4. Abdel‐Rahman O, Oweira H, Petrausch U, Helbling D, Schmidt J, Mannhart M, et al. Immune‐related ocular toxicities in solid tumor patients treated with immune checkpoint inhibitors: a systematic review. Expert Review of Anticancer Therapy 2017;17(4):387‐94. [DOI] [PubMed] [Google Scholar]
  5. Abdel‐Rahman O. Assessment of the discriminating value of the 8th AJCC stage grouping for hepatocellular carcinoma. HPB: the Official Journal of the International Hepato Pancreato Biliary Association 2018;20(1):41‐8. [DOI] [PubMed] [Google Scholar]
  6. Ahn E, Araki K, Hashimoto M, Li W, Riley JL, Cheung J, et al. Role of PD‐1 during effector CD8 T cell differentiation. Proceedings of the National Academy of Sciences of the United States of America 2018;115(18):4749‐54. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines 3: rating the quality of evidence ‐ introduction. Journal of Clinical Epidemiology 2011;64:401‐6. [DOI] [PubMed] [Google Scholar]
  8. Berzigotti A, Reig M, Abraldes JG, Bosch J, Bruix J. Portal hypertension and the outcome of surgery for hepatocellular carcinoma in compensated cirrhosis: a systematic review and meta‐analysis. Hepatology 2015;61(2):526‐36. [DOI] [PubMed] [Google Scholar]
  9. Brok J, Thorlund K, Gluud C, Wetterslev J. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta‐analyses. Journal of Clinical Epidemiology 2008;61:763‐9. [DOI] [PubMed] [Google Scholar]
  10. Brok J, Thorlund K, Wetterslev J, Gluud C. Apparently conclusive meta‐analyses may be inconclusive ‐ trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta‐analyses. International Journal of Epidemiology 2009;38(1):287‐98. [DOI] [PubMed] [Google Scholar]
  11. Buchbinder EI, McDermott DF. Cytotoxic T‐lymphocyte antigen‐4 blockade in melanoma. Clinical Therapeutics 2015;37(4):755‐63. [DOI] [PubMed] [Google Scholar]
  12. Castellini G, Bruschettini M, Gianola S, Gluud C, Moja L. Assessing imprecision in Cochrane systematic reviews: a comparison of GRADE and Trial Sequential Analysis. Systematic Reviews 2018;7(1):110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. EASL. EASL Clinical Practice Guidelines: management of hepatocellular carcinoma. Journal of Hepatology 2018;69(1):182‐236. [DOI] [PubMed] [Google Scholar]
  14. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta‐analysis detected by a simple, graphical test. BMJ (Clinical Research Ed.) 1997;315:629‐34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Eisenhauer EA, Therasse P, Bogaerts J, Schwartz LH, Sargent D, Ford R, et al. New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). European Journal of Cancer 2009;45(2):228‐47. [DOI] [PubMed] [Google Scholar]
  16. El‐Khoueiry AB, Sangro B, Yau T, Crocenzi TS, Kudo M, Hsu C, et al. Nivolumab in patients with advanced hepatocellular carcinoma (CheckMate 040): an open‐label, non‐comparative, phase 1/2 dose escalation and expansion trial. Lancet 2017;389(10088):2492‐502. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Eltobgy M, Oweira H, Petrausch U, Helbling D, Schmidt J, Mehrabi A, et al. Immune‐related neurological toxicities among solid tumor patients treated with immune checkpoint inhibitors: a systematic review. Expert Review of Neurotherapeutics 2017;17(7):725‐36. [DOI] [PubMed] [Google Scholar]
  18. FACT‐Hep: for patients with hepatobiliary cancer (liver, bile duct and pancreas). www.facit.org/FACITOrg/Questionnaires (accessed 9 November 2018).
  19. Forner A, Llovet JM, Bruix J. Chemoembolization for intermediate HCC: is there proof of survival benefit?. Journal of Hepatology 2012;56(4):984‐6. [DOI] [PubMed] [Google Scholar]
  20. Global Burden of Disease Liver Cancer Collaboration. The burden of primary liver cancer and underlying etiologies from 1990 to 2015 at the global, regional, and national level. Results from The Global Burden of Disease Study 2015. JAMA Oncology 2017;3(12):1683‐91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. McMaster University (developed by Evidence Prime). GRADEpro GDT. Version accessed 4 July 2019. Hamilton (ON): McMaster University (developed by Evidence Prime), 2015.
  22. Granier C, Guillebon E, Blanc C, Roussel H, Badoual C, Colin E, et al. Mechanisms of action and rationale for the use of checkpoint inhibitors in cancer. ESMO Open 2017;2(2):e000213‐e. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence‐‐publication bias. Journal of Clinical Epidemiology 2011;64(12):1277‐82. [PUBMED: 21802904] [DOI] [PubMed] [Google Scholar]
  24. Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso‐Coello P, Rind D, et al. GRADE guidelines: 6. Rating the quality of evidence‐‐imprecision. Journal of Clinical Epidemiology 2011;64(12):1283‐93. [PUBMED: 21839614] [DOI] [PubMed] [Google Scholar]
  25. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 7. Rating the quality of evidence‐‐inconsistency. Journal of Clinical Epidemiology 2011;64(12):1294‐302. [PUBMED: 21803546] [DOI] [PubMed] [Google Scholar]
  26. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidence‐‐indirectness. Journal of Clinical Epidemiology 2011;64(12):1303‐10. [PUBMED: 21802903] [DOI] [PubMed] [Google Scholar]
  27. Guyatt GH, Oxman AD, Sultan S, Glasziou P, Akl EA, Alonso‐Coello P, et al. GRADE guidelines: 9. Rating up the quality of evidence. Journal of Clinical Epidemiology 2011;64(12):1311‐6. [PUBMED: 21802902] [DOI] [PubMed] [Google Scholar]
  28. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction‐GRADE evidence profiles and summary of findings tables. Journal of Clinical Epidemiology 2011;64(4):383‐94. [PUBMED: 21195583] [DOI] [PubMed] [Google Scholar]
  29. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. Journal of Clinical Epidemiology 2011;64(4):395‐400. [PUBMED: 21194891] [DOI] [PubMed] [Google Scholar]
  30. Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso‐Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence‐‐study limitations (risk of bias). Journal of Clinical Epidemiology 2011;64(4):407‐15. [PUBMED: 21247734] [DOI] [PubMed] [Google Scholar]
  31. Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso‐Coello P, et al. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. Journal of Clinical Epidemiology 2013;66(2):151‐7. [DOI] [PubMed] [Google Scholar]
  32. Guyatt G, Andrews J, Oxman AD, Alderson P, Dahm P, Falck‐Ytter Y, et al. GRADE guidelines: 15. Going from evidence to recommendations: the significance and presentation of recommendations. Journal of Clinical Epidemiology 2013;66(7):719‐25. [DOI] [PubMed] [Google Scholar]
  33. Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, et al. GRADE guidelines: 13. Preparing summary of findings tables ‐ continuous outcomes. Journal of Clinical Epidemiology 2013;66(2):173‐83. [DOI] [PubMed] [Google Scholar]
  34. Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, et al. GRADE guidelines: 12. Preparing summary of findings tables ‐ binary outcomes. Journal of Clinical Epidemiology 2013;66(2):158‐72. [DOI] [PubMed] [Google Scholar]
  35. Guyatt GH, Ebrahim S, Alonso‐Coello P, Johnston BC, Mathioudakis AG, Briel M, et al. GRADE guidelines 17: assessing the risk of bias associated with missing participant outcome data in a body of evidence. Journal of Clinical Epidemiology 2017;87:14‐22. [PUBMED: 28529188] [DOI] [PubMed] [Google Scholar]
  36. Heimbach JK, Kulik LM, Finn RS, Sirlin CB, Abecassis MM, Roberts LR, et al. AASLD guidelines for the treatment of hepatocellular carcinoma. Hepatology 2018;67(1):358‐80. [DOI] [PubMed] [Google Scholar]
  37. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from training.cochrane.org/handbook.
  38. Higgins JPT, Green S (editors). Chapter 16: Special topics in statistics. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from training.cochrane.org/handbook.
  39. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ (Clinical Research Ed.) 1999;319:670‐4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. International Conference on Harmonisation Expert Working Group. International conference on harmonisation of technical requirements for registration of pharmaceuticals for human use. ICH harmonised tripartite guideline. Guideline for good clinical practice CFR & ICH Guidelines. Vol. 1, Philadelphia (PA): Barnett International/PAREXEL, 1997. [Google Scholar]
  41. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomised trials in meta‐analyses. Annals of Internal Medicine 2001;135(11):982‐9. [DOI] [PubMed] [Google Scholar]
  42. Kudo M, Finn RS, Qin S, Han KH, Ikeda K, Piscaglia F, et al. Lenvatinib versus sorafenib in first‐line treatment of patients with unresectable hepatocellular carcinoma: a randomised phase 3 non‐inferiority trial. Lancet 2018;391(10126):1163‐73. [DOI] [PubMed] [Google Scholar]
  43. Kwon O, Jung Y, Bae K. Anti‐hepatitis B core positivity as a risk factor for hepatocellular carcinoma in alcoholic cirrhosis: a case‐control study. Alcohol 2012;46:537‐41. [DOI] [PubMed] [Google Scholar]
  44. Levy I, Sherman M. Staging of hepatocellular carcinoma: assessment of the CLIP, Okuda, and Child‐Pugh staging systems in a cohort of 257 patients in Toronto. Gut 2002;50(6):881‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Llovet JM, Ricci S, Mazzaferro V, Hilgard P, Gane E, Blanc JF, et al. Sorafenib in advanced hepatocellular carcinoma. New England Journal of Medicine 2008;359(4):378‐90. [DOI] [PubMed] [Google Scholar]
  46. Macaskill P, Walter SD, Irwig L. A comparison of methods to detect publication bias in meta‐analysis. Statistics in Medicine 2001;20:641‐54. [DOI] [PubMed] [Google Scholar]
  47. Mahoney KM, Freeman GJ, McDermott DF. The next immune‐checkpoint inhibitors: PD‐1/PD‐L1 blockade in melanoma. Clinical Therapeutics 2015;37(4):764‐82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Masuzaki R, Yoshida H, Omata M. Hepatitis C and hepatocellular carcinoma. Hepatocellular Carcinoma Diagnosis and Treatment. New York: Humana Press, 2009:45‐50. [Google Scholar]
  49. Mier JW. Immune checkpoint inhibitors in the treatment of metastatic melanoma. Clinical Therapeutics 2015;37(4):753‐4. [DOI] [PubMed] [Google Scholar]
  50. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta‐analyses?. Lancet 1998;352(9128):609‐13. [DOI] [PubMed] [Google Scholar]
  51. Mustafa RA, Santesso N, Brozek J, Akl EA, Walter SD, Norman G, et al. The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses. Journal of Clinical Epidemiology 2013;66(7):736‐42; quiz 742.e1‐5. [PUBMED: 23623694] [DOI] [PubMed] [Google Scholar]
  52. Okuda K, Ohtsuki T, Obata H, Tomimatsu M, Okazaki N, Hasegawa H, et al. Natural history of hepatocellular carcinoma and prognosis in relation to treatment: study of 850 patients. Cancer 1985;56:918‐28. [DOI] [PubMed] [Google Scholar]
  53. Oliveri RS, Wetterslev J, Gluud C. Transarterial (chemo)embolisation for unresectable hepatocellular carcinoma. Cochrane Database of Systematic Reviews 2011, Issue 3. [DOI: 10.1002/14651858.CD004787.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Parmar MK, Torri V, Stewart L. Extracting summary statistics to perform meta‐analyses of the published literature for survival endpoints. Statistics in Medicine 1998;17(24):2815‐34. [DOI] [PubMed] [Google Scholar]
  55. Pugh RNH, Murray‐Lyon IM, Dawson JL, Pietroni MC, Williams R. Transection of the oesophagus for bleeding oesophageal varices. BJS 1973;60(8):646‐9. [DOI] [PubMed] [Google Scholar]
  56. The Nordic Cochrane Centre, Cochrane. Review Manager (RevMan). Version 5.3. Copenhagen: The Nordic Cochrane Centre, Cochrane, 2014.
  57. Royle P, Milne R. Literature searching for randomised controlled trials used in Cochrane reviews: rapid versus exhaustive searches. International Journal of Technology Assessment in Health Care 2003;19(4):591‐603. [DOI] [PubMed] [Google Scholar]
  58. Sanyal AJ, Yoon SK, Lencioni R. The etiology of hepatocellular carcinoma and consequences for treatment. Oncologist 2010;15(Suppl 4):14‐22. [DOI] [PubMed] [Google Scholar]
  59. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Annals of Internal Medicine 2012;157(6):429‐38. [DOI] [PubMed] [Google Scholar]
  60. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Health Technology Assessment 2012;16(35):1‐82. [DOI] [PubMed] [Google Scholar]
  61. Savović J, Turner RM, Mawdsley D, Jones HE, Beynon R, Higgins JPT, et al. Association between risk‐of‐bias assessments and results of randomized trials in Cochrane Reviews: The ROBES Meta‐Epidemiologic Study. American Journal of Epidemiology 2018;187(5):1113‐22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Schiefelbein E, Zekri AR, Newton DW, Soliman GA, Banerjee M, Hung ChW, et al. Hepatitis C virus and other risk factors in hepatocellular carcinoma. Acta Virologica 2012;56(3):235‐40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273(5):408‐12. [DOI] [PubMed] [Google Scholar]
  64. Seymour L, Bogaerts J, Perrone A, Ford R, Schwartz LH, Mandrekar S, et al. iRECIST: guidelines for response criteria for use in trials testing immunotherapeutics. Lancet Oncology 2017;18(3):e143‐e52. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Storebø OJ, Pedersen N, Ramstad E, Kielsholm ML, Nielsen SS, Krogh HB, et al. Methylphenidate for attention deficit hyperactivity disorder (ADHD) in children and adolescents – assessment of adverse events in non‐randomised studies. Cochrane Database of Systematic Reviews 2018, Issue 5. [DOI: 10.1002/14651858.CD012069.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JP, Thabane L, et al. Can trial sequential monitoring boundaries reduce spurious inferences from meta‐analyses. International Journal of Epidemiology 2009;38(1):276‐86. [DOI] [PubMed] [Google Scholar]
  67. Thorlund K, Anema A, Mills E. Interpreting meta‐analysis according to the adequacy of sample size. An example using isoniazid chemoprophylaxis for tuberculosis in purified protein derivative negative HIV‐infected individuals. Clinical Epidemiology 2010;2:57‐66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Thorlund K, Engstrøm J, Wetterslev J, Brok J, Imberger G, Gluud C. User manual for Trial Sequential Analysis (TSA). ctu.dk/tsa/files/tsa_manual.pdf (accessed 23 July 2019).
  69. Copenhagen Trial Unit. TSA ‐ Trial Sequential Analysis. Version 0.9.5.10 Beta. Copenhagen: Copenhagen Trial Unit, 2011.
  70. Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta‐analysis. Journal of Clinical Epidemiology 2008;61(1):64‐75. [DOI] [PubMed] [Google Scholar]
  71. Wetterslev J, Thorlund K, Brok J, Gluud C. Estimating required information size by quantifying diversity in a random‐effects meta‐analysis. BMC Medical Research Methodology 2009;9:86. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Wetterslev J, Jakobsen JC, Gluud C. Trial Sequential Analysis in systematic reviews with meta‐analysis. BMC Medical Research Methodology 2017;17(1):39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Wherry EJ, Teichgraber V, Becker TC, Masopust D, Kaech SM, Antia R, et al. Lineage relationship and protective immunity of memory CD8 T cell subsets. Nature Immunology 2003;4(3):225‐34. [DOI] [PubMed] [Google Scholar]
  74. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman GD, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta‐epidemiological study. BMJ (Clinical Research Ed.) 2008;336:601‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Xu C, Chen YP, Du XJ, Liu JQ, Huang CL, Chen L, et al. Comparative safety of immune checkpoint inhibitors in cancer: systematic review and network meta‐analysis. BMJ (Clinical Research Ed.) 2018;363:k4226. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from The Cochrane Database of Systematic Reviews are provided here courtesy of Wiley

RESOURCES