Model‐based Meta‐Analysis on the Efficacy of Pharmacological Treatments for Idiopathic Pulmonary Fibrosis

Abstract

Recently, the US Food and Drug Administration (FDA) approved the first two drugs (pirfenidone and nintedanib) indicated for the treatment of idiopathic pulmonary fibrosis (IPF). The purpose of this analysis was to leverage publicly available data to quantify comparative efficacy of compounds that are approved or in development. An analysis‐ready database was developed, and the analysis dataset is composed of summary‐level data from 43 arms in 20 trials, with treatment durations ranging from 8–104 weeks. A hierarchical multivariable regression model with nonparametric placebo estimation was used to fit the longitudinal profile of change from baseline of percent predicted forced vital capacity (%predicted FVC) data. Pirfenidone and nintedanib were the only drugs identified to have significant estimated positive treatment effects. Model simulations were performed to further evaluate the covariate and time course of treatment effects on longitudinal change from baseline %predicted FVC to inform future trial designs and support decision making.

Study Highlights.

WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?

☑ The recent approval of pirfenidone and nintedanib and the paucity of data from direct (head‐to‐head) comparisons among treatments for IPF prompt the need for indirect comparisons by leveraging data currently available.

WHAT QUESTION DID THIS STUDY ADDRESS?

☑ An MBMA was utilized to more fully leverage publicly available data by incorporating longitudinal information and to provide more accurate estimates of the true response and thereby a more valid comparison between treatments.

WHAT THIS STUDY ADDS TO OUR KNOWLEDGE

☑ The fixed‐effect model with nonparametric placebo estimation, with time‐dependent but saturable treatment effects and baseline %predicted FVC influencing the magnitude of the treatment effects was determined to be the best and most parsimonious model to describe change from baseline %predicted FVC for 15 treatment regimens.

HOW MIGHT THIS CHANGE DRUG DISCOVERY, DEVELOPMENT, AND/OR THERAPEUTICS?

☑ Because MBMA accounts for heterogeneity across trials, this framework can be applied to inform future trial designs with clinical trial simulations. Therefore, the presented framework will be adapted and re‐used to address questions related to the development of IPF drugs.

Idiopathic pulmonary fibrosis (IPF) is a fatal, chronic, progressive fibrosing interstitial pneumonia of unknown cause. The disease most commonly occurs in persons over the age of 50 years, more often in men than in women, and the majority of the patients have a history of cigarette smoking.1 Prevalence estimates of IPF ranged from 14–27.9 cases per 100,000 persons in the general population in the United States.2

In October 2014, the US Food and Drug Administration (FDA) approved the first two pharmacological therapies, pirfenidone and nintedanib, which demonstrate to slow the decline of lung function in IPF. In the face of the approval of the two new drugs, the treatment landscape for drug development in IPF is changing rapidly in terms of background therapy or treatment comparators, and patient enrollment in placebo‐controlled trials will be challenging going forward.

However, the paucity of data from direct (head‐to‐head) comparisons among treatments for IPF prompts the need for indirect comparisons by leveraging data currently available. Specifically, summary‐level information extracted from the literature can be combined into a meta‐analysis framework to compare treatment effects of different drugs across different patient populations. In recent years, network meta‐analysis (NMA) has been used to estimate the relative efficacy of IPF treatments.3, 4, 5, 6, 7 However, one major limitation of traditional NMAs is the complexity of integrating variable time courses and placebo effects, as in the case with IPF.8 Therefore, to more fully leverage publicly available data by incorporating longitudinal information across different time points, a model‐based meta‐analysis (MBMA) could be utilized to allow evaluation of the full time course of the response in terms of both its speed of onset, and maintenance and magnitude of drug effect, in order to provide more accurate estimates of the true response and thereby a more valid comparison between treatments.9 An MBMA for treatments in IPF had been published previously,10 however, in light of recent FDA approvals for pirfenidone and nintedanib and availability of additional data in the public domain from these two drugs and from other experimental therapies for IPF, an update of the MBMA and re‐evaluation of the model are warranted.

METHODS

Database construction

A systematic review of publicly available data from the PubMed database was conducted in September 2015, according to a prespecified database‐building protocol and based on the relevant identification, screening, and assessment steps described in the Cochrane Handbook for Systematic Review of Interventions and reporting items in the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) statement.11, 12 The search used the following keywords: (“idiopathic pulmonary fibrosis” (MeSH terms) OR “idiopathic” (all fields) AND “pulmonary” (all fields) AND “fibrosis” (all fields) OR “idiopathic pulmonary fibrosis” (all fields) AND “randomized controlled trial” (publication type) AND English (language)). Additional online data sources, including the FDA and ClinicalTrials.gov websites, were searched for unpublished trials and additional information on published trials.

Data extracted from each citation included but were not limited to: publication year, title, journal, author, trial, region, sponsor, trial year, trial design, and primary treatment. Patient demographics are captured, as well as randomized treatment information for each arm, such as dose, dosing frequency, and routes of administration. The notable clinical efficacy measures extracted included: forced vital capacity (FVC; in volume unit), normalized FVC expressed as percent of predicted normal (%predicted FVC; the lower %predicted FVC, the more severe of the disease in terms of pulmonary function), diffusing capacity of the lung for carbon monoxide, total lung capacity, time to IPF exacerbation, time to death or hospitalization, 6‐minute walk test distance, progression‐free survival, and forced expiration volume in 1 second. For continuous measurements, such as FVC and %predicted FVC, changes from baseline or percent change from baseline are extracted if reported.

The database was augmented using an R‐script that processes derivations for trials with missing SD, SE, or confidence interval (CI) from the nonmissing reported statistic values. The augmentation process also included standardizing endpoint units, covariate estimates and doses, and additional database variables were created to facilitate the MBMA analysis. Transformation was performed to generate the %predicted FVC from reported observed FVC in the three nintedanib trials using a standard equation.13 For continuous measurements, missing baseline value, postbaseline value, change from baseline, and percent change from baseline were calculated if any two of these values were reported. Furthermore, if <60% of trial arms had missing values of a prespecified covariate, the missing covariate values were imputed using multivariate linear regression, based on nonmissing clinically relevant trial variables.

Exploratory analysis

Summary tables were prepared to facilitate the systematic review of the database for the amount of data available for each treatment and efficacy end point across trials, the distribution of mean patient characteristics, and for features that might introduce bias in the analysis. Of the 17 efficacy end points available in the augmented database, one end point, namely change from baseline %predicted FVC (Δ%predicted FVC), was chosen for further exploratory analysis based on the extent of availability in the database and its acceptability as a primary efficacy end point for drug development in IPF.

Forest plots were utilized to assess and compare placebo responses of %predicted FVC among trials, to highlight effect outliers, and to investigate possible explanations for these differences.14 Additionally, longitudinal profiles of %predicted FVC as absolute values, as change from baseline, and as percent change from baseline were explored using time‐course plots.

To account for variability of responses between trials, covariates of interest were prespecified based on clinical consideration and physiological plausibility of the estimates and investigated for their ability to predict differences in SD values and treatment effects. The prespecified covariates were mean or median baseline %predicted FVC, disease duration, and age, as well as proportions of subjects who were men, white, and current or former smokers in each trial.

The database was further screened, during which duplicate end‐point data in the same trial were removed, with a preference to retain longitudinal data. When multiple summary statistic values were available for the same time points, mean values were chosen over the median value, and intent‐to‐treat populations were included whenever possible. Based on these selection criteria, an MBMA analysis dataset was developed.

Model development

Before modeling the treatment effect, imputation of missing SD values was explored using linear, log, exponential, and maximum effect (E_max) models. The model‐predicted SD values were then used in the MBMA model development for derivation of the study weights.

The longitudinal profile of Δ%predicted FVC was characterized by a hierarchical regression model using maximum likelihood estimation. If imbalance exists between the randomized groups at baseline, the change from baseline data would give the least biased estimate of a causal effect. A nonparametric approach was implemented to model the data from the placebo trial arms to allow the placebo estimate to vary unrestrained for each trial and each time point, so that only the overall estimation of a specific drug effect would impact the accuracy of the placebo estimate. The advantage of a nonparametric approach is that no arbitrary assumption had to be made on the distribution of the placebo response15 and that no bias is created due to placebo model misspecification. This is appropriate for summary‐level data from a therapeutic area that has highly variable placebo effects across clinical trials but more consistent treatment effects within trials.16

Multiple levels of heterogeneity were described as mixed‐effect variability terms, accommodating between‐study variability and within‐study variability. A residual variability term was used to account for the unexplained deviation from fixed effects.9 General model components included the following:

$${\Delta Y}_{ijt}={Eo}_{it}+{f(\theta }_{ki})+{\eta }_{ij}+{\varepsilon }_{ijt}$$ (1)

In Eq. 1, ΔY_ijt is the mean change from baseline response for the jth arm in the ith trial at time t, Eo_it is the placebo response (using the nonparametric approach) for the ith trial at time t, f(θ_ki) is the effect due to the kth treatment or covariate in the ith trial, η_ij is the random residual due to between arm variability, and ɛ_ijt is the random residual due to within arm variability, which is assumed to be normally distributed with a variance dependent on the sample size and observed SD of the trial arm. The model used the inverse of the estimated variance as weights, so that more weight was given to the trials with larger sample size and less variability in observations.

During model development, the treatment effects were initially incorporated to be constant over time and described by a scaling factor, E_max, which was estimated as a single parameter representing the maximal change in Δ%predicted FVC across all treatments. Next, under the assumption that treatments with a similar mechanism of action (i.e., in the same treatment class) share a common saturable relationship,17 a single E_max value was estimated for each drug class. This was followed by a more granular estimation of separate E_max for each treatment. Because there are 12 drug classes and 14 drugs represented in the analysis dataset, estimating the maximal effect of each treatment instead of each treatment class did not drastically increase the complexity of the model. For treatments with dose‐ranging data available, the potential dose‐response relationships were estimated by testing different functional forms.

The model development of the treatment effects in the model was further refined by investigating nonlinear time‐varying treatment effects. Alternatives to immediate attainment of the maximal treatment effect were evaluated, first by using a common functional form and estimating one set of time‐varying model parameters for all treatments; and then by testing different functional forms (e.g., E_max) and separate time‐varying model parameter values for various drugs and drug classes. This accommodated heterogeneity in the time of treatment response onset and time to reach maximal treatment effect among different drugs and drug classes.15

Summary‐level patient population characteristics were included in the model as covariates to help explain the variability among trials and treatment arms. Stepwise addition of the covariates was performed, without accounting for interaction between covariates, and only the statistically and clinically relevant covariates were retained in the final model.

Model development and complexity were driven by the data and guided by successful convergence of the minimization routine, comparison of statistical significance using Bayesian information criterion values against the reference model results within each successive step of the model development, and examination of parameter estimate precision values. All data exploration and model development, evaluation, and simulation were conducted using the R software version 3.1.2 (R Development Core Team, 2008), in particular, the “gnls” function in the “nlme” package version 3.1–128.

Model evaluation

The adequacy of model fits across trials was evaluated by visually inspecting goodness‐of‐fit plots, including time‐course and summary forest plots of observed distributions vs. predicted values. In addition, a sensitivity analysis was conducted to assess the impact of using predicted vs. a mixture of observed and predicted SD values for weighting purpose in the model.

Model application

Model simulations were performed using fixed and random effect estimates from the final MBMA model to predict the treatment effects at hypothetical dose and time points, as well as to further evaluate typical relationships between treatment and covariate effects on longitudinal Δ%predicted FVC.

RESULTS

Database overview

A total of 89 potentially relevant citations were retrieved from the initial search and, after review of the abstracts and full papers, 40 citations were chosen to be included in the IPF database, according to the inclusion/exclusion criteria specified in the database protocol. A flow chart of the number of citations screened and the reasons for exclusion are displayed in Figure 1.

Figure 1.

Flow chart of database and analysis dataset development. FDA, US Food and Drug Administration; IPF, idiopathic pulmonary fibrosis.

The final database was constructed from 40 citations from 32 trials, including 4 medical and statistical reviews from the FDA and 5 reporting results from ClinicalTrials.gov.

Exploratory analysis

A summary of the end points available in the final database showed that %predicted FVC is the second most reported efficacy end point, after mortality. Therefore, due to its prevalence in the database and relevance in phase II clinical trials, of which the drug development stage decisions that the analysis intended to support, Δ%predicted FVC was chosen for the MBMA. Further data selection was performed to construct an analysis dataset, during which additional trials were excluded based on their lack of availability of the end point of interest, of the preferred prespecified summary statistic type, and because of multiple references describing the same trial. The numbers and reasons that specific citations were not retained in the dataset are included in the flow diagram of analysis dataset development (Figure 1).

Following the data selection process, the analysis dataset consists of Δ%predicted FVC summary‐level data of 43 arms from 4,919 subjects in 20 trials (Table 1),18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35 17 of which were double‐blinded and placebo‐controlled, and none of the trials had stratified data within trial arms. The trials in the analysis dataset were conducted across Europe, North America, and Australia, but none in Asia, and large variabilities were observed in the sample size for each active treatment, as well as in the treatment durations.

Table 1.

Summary of treatments and clinical trials in the model‐based meta‐analysis dataset

Treatment	Class	Dose	Total no. of arms	Total no. of timepoints	Total no. of subjects	References
Ambrisentan	Endothelin receptor antagonist	10 mg	1	7	330	28
Azathioprine	Immunosuppressant	3 mg	1	1	14	30
Bosentan	Endothelin receptor antagonist	125 mg	1	1	74	22
Colchicine	Antigout	1 mg	3	7	39	18, 19, 33
Co‐trimoxazole	Antibiotic	960 mg	1	1	95	31
Etanercept	Tumor necrosis factor inhibitor	25 mg	1	5	46	29
Interferon‐gamma	Interferon	200 μg	4	8	607	18, 23, 32, 33
N‐acetylcysteine	Mucokinetic	1,800 mg/day	1	4	133	24
Nintedanib	Tyrosine kinase inhibitor	300 mg/day	3	21	723	34, 35
Pirfenidone	Antifibrotic	1,197, 2,403 mg/day	4	22	710	20, 25
Placebo	Placebo	Not applicable	17	63	2,035	20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 34, 35
Prednisone	Corticosteroid	40 mg	1	1	12	19
PRM‐151	Antifibrotic	1, 5, 10 mg/kg	3	3	15	27
Sildenafil	Phosphodiesterase inhibitor	20 mg	1	1	14	21
Warfarin	Anticoagulant	1.8 mg	1	3	72	26
Total			43	148	4,919

In the analysis dataset, the most commonly studied active treatments in the numbers of trials and subjects were interferon gamma, pirfenidone, and nintedanib, and the most common comparator was placebo with 2,035 subjects (Supplementary Figure S1). There were 8 trials with longitudinal Δ%predicted FVC data, and the active treatments with longitudinal data were: etanercept, warfarin, interferon gamma, ambrisentan, nintedanib, N‐acetylcysteine, pirfenidone, and colchicine. When comparing placebo arms across trials, a large variability in Δ%predicted FVC response was observed in the exploratory plots, and this characteristic was consistent with previous observations in clinical trials for IPF.16 In addition, there was a pattern of increasingly negative Δ%predicted FVC in the trials with longer treatment durations.

In the original database, the values for the prespecified covariate were missing in 0–36% of the trials and imputation was performed for the missing covariates, namely baseline %predicted FVC and smoking status, by leveraging possible clinically relevant information from nonmissing variables in the database, such as age, region, and publication year. Weight, height, and body mass index were not imputed because >60% of trial arms had missing values. The distributions of the covariate values across trials are presented in Supplementary Figure S2.

The covariates were also investigated for their potential to predict SD, in order to derive missing SD values, which corresponds to 38% of records in the augmented database. The final SD model included linear time and drug class as predictors, with the endothelin receptor antagonist treatment class, which consisted of data from ambrisentan, bosentan, and macitentan, having a higher predicted SD than other treatment classes. Observed and predicted SD values were compared and the correlation coefficient (r = 0.70) was sufficient for weighting purpose in MBMA and predicted SD values were used for MBMA model development.

Model development

The longitudinal Δ%predicted FVC data was fitted using a hierarchical effect model with a nonparametric placebo estimation. Model parameter estimates are shown in Table 2.

Table 2.

Parameter estimates of the final model

Parameter/effect	Estimate (95% CI)
Interferon‐gamma treatment	−0.618 (−2.46 to 1.23)
Colchicine treatment	−7.63 (−13 to −2.3)
Prednisone treatment	−11.9 (‐25.8 to 1.95)
Sildenafil treatment	2.67 (−7.15 to 12.5)
Bosentan treatment	1.75 (−4.37 to 7.86)
Ambrisentan treatment	−2.03 (−4.18 to 0.126)
N‐acetylcysteine treatment	−0.0568 (−2.01 to 1.9)
Warfarin treatment	−0.276 (−4.89 to 4.33)
Etanercept treatment	1.47 (−2.3 to 5.24)
Azathioprine treatment	5.89 (−4.14 to 15.9)
Co‐trimoxazole treatment	0.172 (−3.72 to 4.07)
Nintedanib treatment	3.31 (2.15 to 4.47)
Pirfenidone 1,197 mg/day treatment	2.42 (0.733 to 4.11)
Pirfenidone 2,403 mg/day treatment	3.87 (2.68 to 5.06)
PRM‐151 treatment	12.5 (−5.13 to 30.1)
Time‐varying effect (lambda)	−3.18 (−3.69 to −2.68)
Baseline FVC effect (embase)	1.48 (−2.7 to 5.66)
Between trial‐arm variability	1.07 × 10−6

CI, confidence interval; FVC, forced vital capacity.

Note: The treatment estimates are maximum effects in %predicted FVC; the time‐varying effect of the treatment is parameterized as 1‐exp(‐lambda*time), in which lambda is the shape parameter that determines the steepness of the curve; the baseline FVC effect on the treatment is parameterized as (arm level baseline %predicted FVC/74)^embase. The SE of the between trial arm variability was not estimated.

Treatment effects were estimated for all 14 active treatments in the analysis dataset. In the final model, the dose‐response relationship of pirfenidone was characterized using a step‐function, with the high dose (2,403 mg/day) having an estimated greater treatment effect than the low dose (1,197 mg/day) of pirfenidone. For PRM‐151, despite the fact that data from three dose strengths (1, 5, and 10 mg/kg) in a small trial (N = 15 in the active treatment arms) were included in the dataset, attempts to model a dose‐response relationship resulted in minimization failures during maximum likelihood estimation. This indicates that the models are overparameterized and the maximum treatment effect of PRM‐151 was, therefore, described by a single E_max parameter. Other treatments reported had only one dose regimen per treatment in the dataset; therefore, these treatments were modeled using a single E_max parameter to describe the maximum drug effect for each treatment. The final model provided reasonably precise estimates of the structural model parameters, especially for pirfenidone and nintedanib (≤34.9% relative SE).

The final model also included a time‐dependent treatment effect to estimate how rapid the maximum treatment response is achieved, and this time component of the drug effect is described by an empirical model function: 1‐exp(‐lambda*time), where lambda is the shape parameter that determines the steepness of the curve. Based on the estimated value (95% CI) of −3.18 (−3.69 to −2.68) for lambda, 90% of maximum response is achieved and approaching a plateau at ∼56 weeks, with the time to 50% maximum efficacy of 16.5 weeks after treatment initiation. Investigations of more complex models to describe the time component of the treatment effect for the overall and for specific individual treatments did not result in successful minimization of the model run, potentially due to limited longitudinal data; therefore, the same model function and a shared lambda term were used between all treatments.

Six prespecified covariates were investigated for their association with the treatment effects. The covariate search was conducted as part of the model development process, and only baseline %predicted FVC normalized to the approximate median value of 74% was included in the final model. The estimated covariate parameter value (95% CI) of 3.86 (‐0.631 to 8.35) means that the model predicts that the higher baseline %predicted FVC, the greater the impact of the treatment effect (i.e., subjects with higher baseline %predicted FVC are expected to show greater treatment effect and less disease progression with a positive treatment for IPF than subjects with lower baseline %predicted FVC).

Ranking of the treatments by the magnitude of the maximum effect (Figure 2) shows PRM‐151 is predicted to have the numerically highest maximum drug effect, although the point estimate is associated with a large 95% CI. Among the 15 treatment regimens evaluated, pirfenidone and nintedanib were the only drugs identified to have statistically significant positive treatment effects on Δ%predicted FVC, with model‐estimated 95% CIs not crossing the null.

Figure 2.

Model estimate and 95% confidence interval (CI) of saturable (maximum) effect for each active treatment. %PRED FVC, percent predicted forced vital capacity; IFN, interferon.

Model evaluation

Model diagnostic plots in the forms of longitudinal plots from representative pirfenidone and nintedanib trials (Figure 3) and from all the trials included in the analysis (Supplementary Material), as well as a forest plot (Figure 4) summarized by trial, showed good agreement between predicted and observed values with no systematic bias. As such, the model provided a reasonable fit to the observed data.

Figure 3.

Model fitted time‐course plots of %predicted forced vital capacity (FVC) change from baseline for representative pirfenidone and nintedanib trials. Note: Symbols and vertical bars are observed mean and SE of the time point, and the fitted line is the model prediction.

Figure 4.

Primary endpoint comparative forest plot of model‐predicted placebo‐corrected %predicted forced vital capacity (FVC) change from baseline, summarized by trial; only placebo‐controlled trials are shown. Note: Blue symbol and horizontal line are mean or median and 95% confidence interval (CI) of the primary endpoint of the trial, corresponding to the VALUE column; WEEK is the follow‐up duration of the trial, and N is the number of subjects included in the observed primary endpoint; vertical grey bar is the model‐prediction for the trial. The trials are categorized by the class of the active treatment, as indicated on the vertical labels on the right. IPF, idiopathic pulmonary fibrosis.

A sensitivity analysis was performed to assess the impact of missing SD and the validity of using predicted values by comparing the MBMA parameter estimates obtained from model runs using SD data from solely predicted values and from a mixture of predicted and observed SD values. The sensitivity analysis results showed that the parameter estimates were similar when a mixture of observed and predicted SD was used. Furthermore, even with some changes of the model parameters, the model predictions were virtually indistinguishable.

Model application

Hypothetical scenarios of interest were simulated by varying combinations of treatment effect, treatment duration, and covariate ranges. Simulated time‐courses of pirfenidone and nintedanib effects based on the observed medians (66% and 76%) of the stratified high and low baseline %predicted FVC groups shows that effect difference is small between the two baseline groups within each treatment (Figure 5).

Figure 5.

Predicted effect of baseline %predicted forced vital capacity (FVC) on pirfenidone and nintedanib treatment time‐course. Solid line represents the 50th percentile and the dotted lines represent the 95th prediction interval from 20,000 simulation replicates.

DISCUSSION

Using model‐based meta‐analysis principles, a hierarchical effect model was developed based on publicly available clinical trial data in this analysis to describe the longitudinal profiles of Δ%predicted FVC.

To better describe the longitudinal data, a time‐course effect model was used in the analysis. The functional form describing a steady increase to a maximal level is consistent with the time‐dependent treatment effect defined in a previous MBMA10 and the incorporation of the time‐course effect greatly improved the fitting. With diverse drug classes included in the analysis, different treatment durations might be needed to attain the maximal efficacy for each drug or drug class. However, the result of analysis suggested that a common time‐course effect model is sufficient to fit longitudinal data from eight different treatments and unique treatment classes.

Among the 14 distinct treatments included in the analysis, pirfenidone and nintedanib are of primary interest, as they are the only two approved IPF therapies to date. Therefore, the understanding of their comparative efficacy is crucial for designing future clinical trials in IPF. Furthermore, the combined sample sizes from the trials of each of these two treatments are greater than any of the other 12 treatments in the database. However, the analysis was not limited to only these two approved drugs because the placebo arms from other trials contributed to more precise identification of the time‐course of treatment effects for IPF.

The ranking of model‐predicted treatment effects indicated that PRM‐151 has the largest estimated drug effect (Figure 2), but given the small sample size (N = 15) and only one observed time point at 8 weeks from the PRM‐151 trial, the E_max estimate of this compound was not precise or reliable. The evaluation of the time‐course effect for the PRM‐151 treatment was, therefore, limited and the simulations results in a very wide CI. Additional longitudinal data for PRM‐151 will be needed to generate more reliable drug‐time course estimates to compare with other treatments.

Between the two drugs, pirfenidone and nintedanib, that showed definitive positive response in the analysis, the model estimates a numerically greater treatment impact from pirfenidone at the recommended dose of 2,403 mg/day over nintedanib 300 mg/day, although their 95% CIs overlap (Figure 2). This finding is consistent with results from NMA3, 6 and from a previous MBMA,10 which suggest that no significant difference between the two drugs was detected with respect to pulmonary function decline or survival benefit. In order to confirm these findings, a head‐to‐head comparison between pirfenidone and nintedanib is needed.

Model evaluation plots for representative pirfenidone and nintedanib trials (Figure 3) showed that the final MBMA model predicts the observed longitudinal data well, and, overall, the model was able to capture the observed values at the primary time point (Figure 4), despite the large differences in treatment duration. Among the 23 treatment arms from 13 treatments in the forest plot, the observed effects are fairly consistent across treatment arms for each of the treatments, in terms of whether or not the treatment has a generally positive, negative, or no treatment effect, and this finding agrees with previous observations in clinical trials for IPF.16 An obvious outlier is interferon gamma, of which two trials reported a large positive treatment effect, and two trials reported no treatment effect. The authors who reported the results from the INSPIRE trial,23 which showed no treatment effect and is the most recently published of the four and has the largest sample size of the interferon gamma trials, contrasted its results with the three previous interferon gamma trials that showed different outcomes, and they hypothesized that the difference in the treatment effects between the interferon gamma trials was due to large dropout rates, shorter treatment duration, or smaller sample size in the previous three trials. The results from the INSPIRE trial were also leveraged for the estimation of treatment effects of colchicine and prednisone. These two treatments lacked placebo‐controlled trial data in subjects with IPF but were indirectly compared to placebo through small active‐control trials18, 19, 33 with interferon gamma that showed worse efficacy than interferon gamma, which showed no treatment effect in the INSPIRE trial. Therefore, colchicine and prednisone were estimated to have the most negative treatment effects in the analysis with wide CIs.

In the current MBMA model, only the baseline %predicted FVC showed significant correlation with the treatment effect. Although baseline disease severity typically influences the E_max of the dose‐response relationship17 and, thus, would be expected to be included in the final model as a significant covariate, the inability of the model to identify other covariates could be attributed to a common limitation among meta‐analyses, stemming from the nature of summary‐level mean or median values.17, 36 Limited covariate distributions in the analysis dataset also reflect the similar entry criteria for many of the clinical trials included in the analysis. Additionally, some covariates could be correlated, making it difficult to isolate the independent effects of these covariates,8 especially in an analysis dataset that included only a small number of trials.

Using the database search algorithm stated in the Methods section, two additional randomized controlled trials published in 2016 were identified. As more data are published based on administration of pirfenidone and nintedanib as marketed products, continuous enhancement of the database in the future could help improve the understanding of the efficacy responses of these two new standard‐of‐care pharmacotherapies.

Supporting information

Supplementary Figure S1 Network plot of analysis dataset, indicating direct connections between treatments. Note: Size of the circle is proportional to the number of subjects in the trial, and the thickness of the line is proportional to the number of comparisons between trials.

Supplementary Figure S2 Distribution of covariate values among trial arms in the analysis dataset. Note: Age and disease duration are in years, baseline is in percent predicted forced vital capacity (FVC), and all other covariates are in percent of subjects in the trial arm. Upper and lower ends of the box represent 25th and 75th percentiles, the whiskers represent 1.5 times the interquartile range, and the symbols are outliers.

Supplementary Figure S3 Model fitted time‐course plots of %predicted forced vital capacity (FVC) change from baseline for trials included in the analysis. Note: Symbol and vertical bar are observed mean and SE of the time point, the solid line is the fitted model prediction, and the dotted line is the model estimated placebo response for trials without placebo control arm.

Supplementary Information.