Letter to the editor: network meta-analysis for indirect comparison of lanadelumab and berotralstat for the treatment of hereditary angioedema

We read with interest the publication titled “Network meta-analysis for indirect comparison of lanadelumab and berotralstat for the treatment of hereditary angioedema” [1]. Using frequentist network meta-analysis (NMA) methods, the authors compared lanadelumab (Takhzyro^®) 300 mg Q2W or 300 mg Q4W to berotralstat (Orladeyo^®) 150 mg or 110 mg in the prevention of hereditary angioedema (HAE) attacks. We believe that indirect treatment comparisons (ITCs), including NMAs, are a valuable tool to compare the efficacy and safety of alternative treatment options in the absence of evidence from direct comparisons. However, ITCs are associated with inherent limitations by their very nature, and it is imperative that authors provide a complete account of the limitations of an ITC to provide essential context to any conclusions based on their analysis. We have identified numerous limitations within the study by Watt et al. that we feel compelled to highlight, as these raise concerns about the validity of the authors' findings and conclusions.

Specifically, major concerns that we wish to note are:

• The authors did not include all relevant clinical trials in their analysis.

• The study did not thoroughly investigate between-study heterogeneity and, consequently, did not identify concerns about the comparability of included studies.

• The study did not conform to widely accepted best practices regarding transparency in reporting.

• The study omits a thorough discussion of its findings and limitations, and needs to be seen within the context of recent reports by Cochrane and several health technology assessment (HTA) agencies, which have all stated that high levels of heterogeneity preclude conclusions about the relative efficacy and safety of HAE long-term prophylaxis (LTP) therapies.

Issue #1

The authors did not include all relevant clinical trials. Even if only US FDA-approved dosing regimens of lanadelumab or berotralstat were considered, the authors should have additionally included studies by Banerji et al., 2017 [2], as well as the APeX-1 [3] and APeX-J [4] trials. Furthermore, other relevant LTP therapies were omitted altogether. As detailed in ‘Issue #3’ below, no rationale was provided for this omission. The resulting network of evidence is very sparse; consequently, the network structure does not allow for the evaluation of potential inconsistency. These shortcomings limit the validity and interpretability of the analysis.

Issue #2

The authors did not provide a complete assessment of study eligibility for evidence synthesis and provided only a cursory statement that the studies were deemed to have similarity with regard to five population baseline characteristics (i.e., age, gender, weight, BMI, baseline HAE attacks). In the absence of comprehensive information regarding eligibility criteria, characteristics of enrolled patient populations, outcome definitions and assessments and trial designs, it is impossible to determine whether or how a detailed and transparent assessment of the similarity or dissimilarity of included studies was performed. Such an analysis is an important prerequisite to establish whether a robust and valid evidence synthesis is feasible. There are substantial between-study heterogeneity and limitations in data availability across the included studies, none of which appear to have been considered or reported by Watt et al., including:

Study design & reporting

• Study setting: the CHANGE [5], HELP [6] and APeX-2 [7] trials cover a period of 15 years, in which clinical practice patterns and the treatment landscape have evolved substantially.

• Minimum baseline attack frequency: APeX-2 and HELP required that patients experience at least 1 attack/month in order to be eligible, while CHANGE required at least 2 attacks/month.

• Concomitant HAE therapy: no concomitant prophylactic medications were permitted in APeX-2 and HELP, whereas androgens and antifibrinolytic drugs were permitted as prophylaxis in CHANGE.

• HAE attack reporting and analysis: HAE attacks in CHANGE and APeX-2 were patient-reported within 24 h, as opposed to 72 h in HELP; there was no investigator confirmation of HAE attacks in CHANGE; differences are also observed in the adjustment of HAE attack rates (e.g., by baseline attack rate vs no adjustment).

• Outcome data availability: CHANGE and HELP did not report exposure time that would permit an analysis for ‘HAE attack rate’ as a Poisson outcome.

• Network connectivity: placebo is not an appropriate bridging comparator for the presented network given the permission of concomitant LTP use in the CHANGE trial but not in APeX-2 or HELP; in addition, the potential impact of differing routes of placebo administration (i.e., IV, oral, SC) is not discussed.

Baseline characteristics

• Baseline HAE attack rates: The CHANGE study did not report the baseline HAE attack rate, and substantial variation was observed between APeX-2 and HELP (ranging from 2.9 to 4.0).

• Prior long-term prophylaxis use: Prior LTP use varied substantially across CHANGE (9.1–18.2%, exclusively androgens), HELP (51.9% for 300 mg Q2W and 69% for 300 mg Q4W) and APeX-2 (75%).

Overall, the aforementioned sources of between-study heterogeneity raise concerns about the validity of the presented NMA and the feasibility of conducting robust evidence synthesis with unbiased estimates. We see strong indications of the transitivity assumption being violated, in which case NMA should not be conducted.

Issue #3

The authors did not provide a fully transparent and complete description of their study, contrary to well established best practices. Most fundamentally, the study objective is not entirely clear: purportedly the aim was to conduct an indirect comparison of lanadelumab and berotralstat. However, this is at odds with the publication defining the broader class of LTP therapies as a group of interventions of interest, and including plasma-derived C1 inhibitor (C1-INH) as an intervention in the NMA. This unclarity is compounded by the Population, Intervention, Comparator, Outcomes, Study Design (PICOS) statement provided in the supplementary material, which includes therapies that are not licensed for LTP in HAE. While the authors mention the list of ultimately included studies, the publication neither presents the full electronic search strategy that was used for the SLR, nor a PRISMA flow diagram or a description of the process for selecting studies and for extracting relevant data from included search records. The publication would also have benefited from an explicit rationale on the outcome selection, which does not include relevant and clinically important outcomes such as the rate of moderate/severe HAE attacks, duration of HAE attacks, six-point improvement from baseline in angioedema-quality of life (AE-QoL), discontinuations due to adverse events (AEs) and severe AEs. Another important omission is the assessment of the risk of bias. Finally, the PRISMA reporting guideline [8] was not followed and there was no attempt to grade the evidence [9,10].

Issue #4

The study results need to be interpreted within the context of the study limitations and other recent reviews which also investigated the comparative efficacy and safety of approved LTP therapies used for the prevention of HAE attacks. Some or all of these reports were published at a time when the manuscript by Watt et al. may already have been submitted. However, it is worth noting that Watt et al. stated definite conclusions about the relative efficacy of compared treatments whereas a number of other recent studies concluded that high levels of between-trial heterogeneity do not allow drawing any such conclusions. For instance, the Cochrane review “Interventions for the long-term prevention of hereditary angioedema attacks.” [11] states that the rarity of HAE and current evidence base “does not allow conclusions on the comparative efficacy of the various drugs for people with HAE”. Similarly, both lanadelumab (Takhzyro^®) and berotralstat (Orladeyo^®) have been evaluated through rigorous processes by several national HTA agencies who commented on the appropriateness of ITCs of HAE LTP therapies to inform decision-making. In its appraisal of the lanadelumab HTA submission in 2020, which included an indirect comparison with intravenous (IV) C1-INH, the Canadian Agency for Drugs and Technologies in Health (CADTH) stated in its clinical review report [12] that “…important limitations with the indirect treatment comparison prevent drawing any conclusions regarding comparative efficacy of lanadelumab and IV C1-INH.” Similarly, in its positive reimbursement recommendation for berotralstat in March 2023, CADTH stated that an ITC between lanadelumab and berotralstat “…is unlikely to produce robust estimates of comparative efficacy or safety…” [13]. In response to the HTA submission of the manufacturer of berotralstat in Germany, the Institute for Quality and Efficiency in HealthCare (IQWiG) concluded that a robust ITC of berotralstat versus C1-INH could not be conducted due to substantial differences in study designs, end point operationalization and comparator therapy [14]. In the US, the Institute for Clinical and Economic Review (ICER) concluded that an ITC of lanadelumab versus two C1-INH therapies was not feasible due to differences in study eligibility criteria (e.g., age and baseline attack rates), small study populations and differences in study design [15]. In the absence of a thorough assessment of such study differences (see ‘Issue #2’), as was presented in the Cochrane review and the assessment reports of several HTA agencies, and a discussion of the limitations arising from the points detailed in ‘Issue #1’ and ‘Issue #2’, it is not deemed appropriate to draw definite statements about comparative efficacy and safety of LTP therapies for the prevention of HAE attacks.

Summary

We have noted above numerous methodological limitations of the study by Watt et al. Notably, the study omitted relevant clinical trials as well as a transparent and complete assessment of the risk of bias of included trials and their comparability. As a result, the study did not detect or discuss the substantial extent of between-study heterogeneity in the sparse evidence base. This raises concerns about the validity of the NMA and the conclusions of the study. In the absence of a transparent and complete accounting for the aforementioned limitations of the study, we believe that the authors' conclusions are potentially spurious and unhelpful for guiding decisions regarding the choice of LTP therapy for the prevention of HAE attacks.