Letter in reply: network meta-analysis for indirect comparison of lanadelumab and for the treatment of hereditary angioedema

We appreciate the comments from the authors of the Letter to the Editor regarding our recent publication “Network meta-analysis for indirect comparison of lanadelumab and berotralstat for the treatment of hereditary angioedema” [1] and welcome the opportunity to respond to the points raised.

Issue #1

The first point the authors' raise relates to the inclusion of all relevant clinical trials. Our analysis specifically focused on treatments in doses that were subsequently approved for long-term prophylaxis in patients (LTPs). Treatments that did not include LTP-approved doses or were not approved for reimbursement at the time the NMA was performed were excluded. Therefore, only clinical trials reporting on these interventions were included.

If the aim of research is clearly defined, there is no need for a large network including all interventions in the specific medical area. It is also a good practice to restrict to smaller networks of a limited number of interventions which are of interest to draw conclusions on comparative efficacy. We clearly described the rationale for focusing on these limited number of studies to evaluate comparative efficacy of treatments in doses that were subsequently approved for use in LTP.

Banerji 2017 [2] is a phase Ib study and therefore not of interest for the base case analysis. It is common practice to exclusively focus on phase III studies in evidence synthesis. It was however included in a sensitivity analysis, described in the systematic literature review section. The results of the sensitivity analysis were in line with the base case results excluding this study.

APeX-1 [3] is a dose-finding study on berotralstat and therefore not of interest for this research. Only relevant dosages of berotralstat which have been scientifically approved are relevant for the NMA.

APeX-J [4] is a study conducted in a Japanese population and therefore not directly comparable to the other studies included in the network due to differences in ethnicity.

Issue #2

The second point raised relates to between-study heterogeneity. Prior to starting the analyses, a feasibility assessment was conducted. The results were presented to and discussed with a medical expert in the field. In addition, the literature was investigated for potential confounders (the covariates age, weight, BMI, sex, race, HAE type, baseline attack rate and prior use of prophylaxis). Table 2 in the publication shows a thorough comparison of medically relevant covariates over the studies [1]. Small sample size is always a limitation in rare disease and this has been described in a transparent way. Duration of follow-up was different; however, since rate ratios which were the output of Poisson regression models in HAE attack rate have been used as an input into the ITC, at least for the primary outcome of interest these were adjusted for. The comparability of crossover and parallel arm design has been described in the discussion and data showed no impact of a carry-over effect in this indication. The outcome definitions have been deemed comparable.

The five population baseline characteristics were selected following a discussion with an expert clinician in the field and are therefore deemed relevant from a medical perspective, which is confirmed in other publications:

There is limited literature reporting on prognostic factors in HAE. A search in Pubmed (https://pubmed.ncbi.nlm.nih.gov/) identified two studies reporting on prognostic factors (Banerji et al. [5], Sundler Björkman et al. [6]), as well as three studies reporting on confounders (Longhurst et al. [7], Giacomini et al. [8], Wang et al. [9]). A study protocol published by the Paul Ehrlich Institute (by CENTOGENE AG 2020 [10]) reported on which variables were adjusted for in the statistical analysis.

Prognostic factors

In Banerji et al. (Table 3 [5]), age, sex, race, HAE type, presence of family history, and comorbidity categories are listed with respect to differences and similarities in the number of HAE attacks over the last 6 months. The table shows that younger ages ≤30 years more often have a moderate number of attacks of 1–3, whereas those aged 31–40 more often have a high number of attacks of ≥13. The age group including 41–64-year-olds seem to have a better prognosis with frequently reporting zero events during the last 6 months, and ≥65-year-olds most frequently report on a moderate number of attacks of 1–3. Caucasians most frequently report on 1–3 attacks, whereas non-Whites most frequently report on ≥13 attacks. HAE type I patients seem to have slightly more attacks on average than those suffering from type II; the same can be found in those with a positive family history of HAE. The most common comorbidity is anxiety, which also results on a high number of 7–12 attacks in most patients. These statistics are, however, only of descriptive nature, and no hypothesis testing has been conducted, so it is unknown if any significant differences would be found. Therefore, these are just hints for possible prognostic factors, yet there is no conclusive evidence.

Confounders

In Longhurst et al. [7], differences in route of administration and in baseline attack frequency are mentioned as confounders. The CENTOGENE AG study protocol [10] describes using statistical analysis such as Poisson regression adjusting for differences in follow-up duration, and potential confounders age, sex and number of attacks per year on standard therapy. Giacomini et al. [8] mention disease severity as a potential confounding factor. Wang et al. [9] conducted a multivariate linear regression adjusting for age, sex, smoking habit, alcohol consumption and long-term prophylaxis as potential confounding factors.

Differences in eligibility criteria between the studies (such as the number of investigator-confirmed HAE attacks) are described in detail in the section on systematic literature review. Differences in number of attacks within run-in periods do not prevent pooling data; it is just necessary to describe these differences in a transparent way, which has been conducted. The same applies to outcome definitions and trial designs. Please review Supplementary Table 2 from Watt et al. [1] as it lists the trial design of the studies in the evidence network in detail. We are transparent about the fact that the CHANGE study is a crossover study; however, crossover and parallel arm studies can be pooled for the purpose of evidence synthesis, especially if data of the comparator of interest are only available from a crossover study. The carryover effect of the crossover design is usually not deemed relevant if the wash-out period between the trial periods is deemed sufficiently long.

Study design & reporting

Study setting

It is standard practice to compare innovative interventions to the standard of care. The standard of care is often an established treatment, in the case of HAE this is Cinryze. It is not a limitation if this established treatment is older or has been applied already a decade ago; this is often the case for standard of care.

Minimum baseline attack frequency

We are transparent about these differences and have highlighted these in Table 3 [5], comparing the baseline characteristics over the studies.

Concomitant HAE therapy

As described in the systematic literature review section on page 5, on-demand treatment was permitted in all three studies. In HELP, intravenous C1-INH, icatibant or ecallantide were allowed, in APeX-2, icatibant, plasma-derived C1-INH, ecallantide or recombinant C1-INH were allowed. In CHANGE, open-label C1-INH was used for on-demand treatment. Concomitant prophylactic treatment is not necessary for patients under lanadelumab and berotralstat treatment; this does not mean, however, that these studies are not comparable to studies on Cinryze where additional prophylactic treatment is often necessary.

HAE attack reporting & analysis

Additional sensitivity analyses on investigator confirmed versus patient reported data were conducted and the results were in line with the base case analysis. The results are not shown in the publication.

Outcome data availability

Even though exposure time was not directly reported in the CHANGE and HELP studies, rates and rate ratios were reported which were outcomes from the Poisson regression models adjusted for relevant covariates and with exposure time as an offset variable in the model.

Network connectivity

The authors' statement here is debatable. Even if some patients used concomitant LTP use, this does not have an impact on the placebo effect since patients in the Cinryze arm had the same options on using concomitant LTP and therefore this was averaged out. Route of placebo administration was not relevant since placebo was applied in the same form as the active treatment to allow the blinding. Active treatments were also compared even though sometimes the routes of administration differed.

Baseline characteristics

Baseline HAE attack rates

The CHANGE study reports baseline attack rate as 3.8 (Table 2 in [1]). Differences in baseline attack rate are still within a medically tolerable range which was confirmed with an expert clinician.

Prior long-term prophylaxis use

The androgens in the CHANGE study have been used at baseline, whereas the prophylaxis in the HELP and APeX-2 studies had been used prior to study entry. Therefore, for the CHANGE study, prior prophylaxis is shown as ‘not reported’ in Table 2 [1] on the baseline characteristics. Medication given at baseline is not the same thing as medication applied prior to study entry.

The authors' comment that the mentioned sources of between-study heterogeneity raise concerns about the validity of the presented NMA and the feasibility of conducting robust evidence synthesis with unbiased estimates. They comment that they identify strong indications of the transitivity assumption being violated, in which case NMA should not be conducted. We disagree, and have been transparent regarding differences in baseline characteristics, study design, outcome definitions, duration of follow-up and investigator assessment. We found the studies similar enough to allow pooling of data, which was also confirmed by an expert clinician in the field of hereditary angioedema. As with every indirect comparison, there are differences in pooled data, but being transparent about these and using adjusted data whenever needed (such as in the estimation of rate ratio on HAE attacks, for example, which were based on data adjusted for confounders and differences in duration of follow-up) still allows an indirect comparison of interventions of interest.

Issue #3

The third point raised relates to the use of accepted best practices regarding transparency in reporting. Assessment of the similarity assumption has been conducted in detail; comparison of baseline characteristics, study design, duration of follow-up, inclusion/exclusion criteria, outcome definitions, background treatments, etc. have been compared and the findings have been discussed.

Study selection also has been described in detail and studies have been excluded for reasons such as early phases (dose finding) as well as for including a Japanese population (not comparable to Caucasian populations).

The objective of the study was to indirectly compare interventions which were subsequently approved for LTP. An SLR does not merely serve the purpose to inform an NMA and can therefore have broader search criteria. Not all studies identified in an SLR need to be used to inform an NMA, as it depends on the research question.

A description of the process for selecting studies is stated including only LTP of approved doses, or approved for reimbursement for HAE LTP, at the time of the NMA. The focus of the manuscript is the findings of the ITC, a short summary of the SLR is provided to define the inclusion and exclusion criteria, the dates when searches were executed and an overview of the SLR goals. Although a PRISMA diagram is not included, the results of the SLR section do state the different stages and process of publication inclusion in the SLR.

Outcome selection mainly depends on data availability, and since only three studies informed this analysis, not all outcomes could be evaluated due to sparse data. Primary and secondary outcomes of interest were however included in the analysis, such as HAE attack rate, probability of achieving ≥90% reduction in the monthly HAE attack rate, and treatment efficacy at steady state.

Grading of an NMA based only on three studies was not deemed relevant since data sparsity would always result in a lower grading due to limited evidence.

Issue #4

The fourth point raised relates to the inclusion of a thorough discussion of the findings and limitations. Limitations are stated in the discussion section. The limitation of the absence of head-to-head studies is described, as well as the limitation of small sample size in rare disease, differences in duration of follow-up, and crossover versus parallel arm design.

Prior to conducting an NMA, a thorough feasibility assessment study was conducted to assess whether data were similar enough to allow the pooling. Our assessment of baseline characteristics, study exclusion and inclusion criteria, outcome definitions, and duration of follow-up showed that there were differences between studies (which usually is the case), and we were transparent in reporting these. We also consulted with an expert clinician as to whether these differences were too large to allow pooling of data. As a result of these evaluations, we decided to move forward with the indirect comparison and at the same time report on the limitations of dissimilarities in detail. In the NMA on HAE attack, adjusted data were used to inform the analysis (adjusting for differences in duration of follow-up as well as for potential confounders). This is a valid approach ensuring that studies are as comparable as possible and to still be able to draw valid conclusions from an ITC based on data which have dissimilarities.

Data inputs on probability of ≥90% reduction in HAE attacks were also based on the outputs of regression models which were adjusted for time points of occurrence of attacks and documentation of attacks.

Summary

Relating to the authors' final summary, we note that a risk of bias assessment was conducted, but not reported. All the included RCTs were good-quality studies in terms of low levels of selection bias, performance bias and attrition bias. In addition, study limitations were described in a transparent way in the discussion section.