Model‐Based Meta‐Analysis With MonolixSuite: A Tutorial for Longitudinal Categorical and Continuous Data

ABSTRACT

Model‐based meta‐analysis (MBMA) informs key drug development decisions by integrating data, published or unpublished, from multiple studies. Due to these various sources of information and the use of summary‐level data (e.g., mean responses over treatment arms or percent responders), MBMA models require careful implementation. This tutorial provides a comprehensive guide for conducting an MBMA with MonolixSuite, focusing on longitudinal continuous and categorical data. Two case studies are presented: the first examining naproxen in osteoarthritis and the second evaluating canakinumab compared to existing treatments in rheumatoid arthritis. The tutorial explains the process of model building and handling study heterogeneity in Monolix, including how to include between‐study variability and between‐treatment‐arm variability. It also shows how to apply appropriate weighting due to the use of summary‐level data. For model evaluation, the tutorial demonstrates the use of automatically generated diagnostic plots, statistical tests, and convergence assessment tools. Furthermore, it illustrates how to use the model in Simulx to support the decision‐making process, such as by simulating clinical trials. This step‐by‐step guidance offers practical insights for leveraging MBMA in model‐informed drug development.

1. What Is MBMA?

Model‐based meta‐analysis (MBMA) is a quantitative method that integrates aggregate data from multiple studies—such as mean responses across treatment arms—which can be combined with other data (e.g., internal clinical trial data), to support key drug development decisions [1]. In drug development, the importance of MBMA for decision‐making is widely acknowledged [2]. It enables leveraging prior knowledge from multiple sources to support strategic decision‐making, and facilitates external benchmarking and competitor assessment, allowing drug developers to position new treatments effectively within the existing therapeutic landscape [3, 4]. In the late stages of clinical drug development, MBMA helps inform target product profiles as well as the probability of pharmacological success (PoPS) [5] and supports late‐phase clinical trial design through optimized dose selection and calculation of the probability of clinical trial success [6]. Additionally, MBMA allows for indirect comparisons between treatments not tested in head‐to‐head trials and can contribute to disease progression model development to refine long‐term treatment planning [7]. It is especially valuable for assessing whether a new treatment may show added clinical benefit compared to the standard of care, and additionally to other investigative or approved treatment options [8].

MBMAs use summary study‐level data extracted from published literature, such as a pharmacodynamic response averaged by trial arm. An MBMA can be utilized for any therapeutic area, and is generally only limited by the amount of available data, which can be sparse in new development fields [2]. Published summary‐level data are usually longitudinal (i.e., repeated measures over time), though MBMAs for single time points (landmark data) have also been conducted [9].

According to a recent review of published MBMAs, although MBMA usage has been increasing over time, the methodological and reporting quality remains poor, with nearly all reviewed MBMA studies being rated “very low quality” according to the AMSTAR‐2 tool [2]. Data collection, methodology, and reporting should follow the guidelines from the Preferred Reporting Items for Systematic reviews and Meta‐Analyses (PRISMA) [10]. Similarly to all meta‐analyses, it is advisable to evaluate potential publication bias [11]. A simulation study showed that MBMAs are most reliable when including at least 10 studies and adequate covariate variation [12]. This tutorial demonstrates how to perform an MBMA using MonolixSuite and evaluate the model using automatically generated model diagnostic plots, statistical tests, and convergence assessment. Using simulations based on the developed models to inform key drug‐development decisions is also showcased.

1.1. How Does MBMA Work?

MBMA extends traditional meta‐analyses by incorporating additional aggregate trial data, such as dose–response and/or longitudinal data, using parametric models [3]. Akin to traditional meta‐analyses, it can include fixed effects alone, or both fixed and random effects. Including random effects in an MBMA allows accounting for between‐study variability (BSV) and between‐treatment‐arm variability (BTAV) in the treatment effect [9, 11].

In a fixed‐effects model, the underlying true effect is assumed to be identical across studies, implying there is no BSV in treatment effect other than that attributed to covariates. Therefore, deviations from the true effect in each study are presumed to arise only from random sampling error, and a study with a smaller sample size is more likely to produce an estimate further from the true effect. In a random‐effects model, on the other hand, each study is assumed to have its own true trial‐specific effect derived from a common distribution. In this case, differences in observed study effects are modeled based on covariate effects, BSV, and BTAV.

The estimated variability components in an MBMA may be viewed, in terms of practical model implementation, as analogous to the variability components included in a population pharmacokinetics/pharmacodynamics (PK/PD) analysis. Specifically, BSV in an MBMA is analogous to inter‐individual variability in a population analysis, and BTAV is analogous to inter‐occasion variability with BTAV being nested in BSV. Both MBMA and population analyses include a source of residual (unexplained) error, reflecting aspects of variability not accounted for by the model, including random sampling error, model misspecification, and measurement error. Similar to analyses based on individual‐level data, MBMA can evaluate the influence of covariates on BSV and BTAV, corresponding to a “meta‐regression” in traditional meta‐analyses or analogous to a covariate analysis in nonlinear mixed‐effects modeling [9, 11, 13]. However, care must be taken to avoid aggregation bias, that is, confounding group‐level and individual‐level effects [14].

A key difference between MBMA and individual‐level population analyses is the weighting of variability components. In MBMA, as in traditional meta‐analysis, the contribution of individual studies must be weighted [11]. The weighting can be based on the precision of the summary‐level data using the empirical standard error (SE) of reported mean effects [11]. Note that error bars in clinical data plots can represent either the SE of the reported mean effect, that is, the precision of the mean, or the standard deviation (SD), that is, the inter‐individual variability in the response. If SDs are reported, the SE can be derived as SE = SD/√N with sample size N. Alternatively, the weighting can be based on the sample size directly, in which case the variability (i.e., SD) is assumed to be the same across studies and time points. In an MBMA, the residual error is weighted, as is the BTAV, if estimated, but not the BSV. The rationale for this can be demonstrated with a linear individual‐level model across studies, with random effects for study and individual. Averaging responses per treatment group at defined times yields a study‐level model where variability is weighted by √N _ijk (number of individuals in treatment group k at time j of study i) [15]. For nonlinear random‐effects models, closed‐form solutions are not possible, though approximations maintain the partitioning of BSV, BTAV, and residual error. If feasible, empirical SE‐based weighting for the residual error is preferred over sample‐size‐based weighting, as it avoids assuming equal variability across studies, treatment arms, and time points [11].

The weighting of BTAV and residual error but not BSV can also be understood intuitively. The residual error in the likelihood balances the relative importance of the multiple summary‐level data points. Studies with larger sample sizes provide more precise average responses, so weighting the error model appropriately gives them greater influence. For BTAV, we assume that patient characteristics are similar across arms within a given study due to randomization. Even with identical treatments, the average response between arms will still differ due to inter‐individual variability and calculating the average over a limited number of individuals. Larger sample sizes reduce these discrepancies, making the average response across arms more consistent. In short, residual error and BTAV represent noise in average response due to finite sample sizes: both decrease as sample size increases, justifying their weighting. In contrast, BSV represents “true” differences in average responses across studies due to study‐level characteristics (such as study design, laboratory equipment, or inclusion criteria). These differences do not depend on the sample size; hence the BSV is not weighted. Importantly, when comparing responses between two study arms from different studies, both BSV and BTAV contribute: even if study‐level factors are identical and BSV is absent, variability persists due to BTAV.

1.2. What Does This Tutorial Cover?

This tutorial presents two different examples for how to perform an MBMA with continuous (Case Study 1) or categorical (Case Study 2) data using Monolix. For an introduction to using Monolix for pharmacokinetic modeling, see Traynard et al. [16], and for detailed step‐by‐step instructions, see the online documentation [17].

Monolix version 2024R1 was used for the analysis; R version 4.4.2 and packages dplyr (1.1.4), ggplot2 (3.5.1), cowplot (1.1.3), vpc (1.2.2), and lixoftConnectors (2024.1) were used to create diagnostic plots. Complete model files and datasets for both case studies are provided in the supplemental material.

In the first case study, we evaluate naproxen compared to placebo for pain relief in osteoarthritis, as assessed by a continuous pain score measure. In the second case study, we assess the comparative efficacy of canakinumab, a drug candidate in Phase 2 development for rheumatoid arthritis, relative to the standard of care, using a binary measure. These analyses, previously reported by Boucher and Bennetts [11] and Demin et al. [8], respectively, using other software, are shown here using MonolixSuite.

2. Case Study 1: Naproxen in Osteoarthritis

2.1. OA and the WOMAC Pain Score

Osteoarthritis (OA), the most common form of arthritis, involves progressive degeneration of cartilage that cushions the ends of long bones. Key indicators include joint pain, stiffness, tenderness, reduced flexibility, and swelling [18]. The WOMAC (Western Ontario and McMaster Universities Osteoarthritis Index) pain score is a validated, disease‐specific tool used to assess disability and efficacy in OA studies. Its pain subscale consists of five items scored on a 0–20 scale [19]. Naproxen, a nonsteroidal anti‐inflammatory drug (NSAID), is indicated to relieve pain and inflammation in OA, rheumatoid arthritis, and other painful musculoskeletal conditions [20].

This case study seeks to answer the following two questions:

1. How does the onset of action and maximum effect compare between naproxen and placebo treatments?

2. What is the impact of a flare design (i.e., a study design that stops existing treatment so that the condition flares up) on the treatment effect?

2.2. Model Development

2.2.1. Data

The dataset [11] includes summary‐level data from 18 placebo‐controlled longitudinal trials of naproxen in OA using WOMAC pain scores. Twelve trials used a flare design where patients were washed out of their pain medication prior to randomization, resulting in higher baseline pain. The WOMAC pain scores declined rapidly following treatment in both the naproxen and placebo arms, generally plateauing within 2 weeks (Figure 1A).

FIGURE 1.

Data and assessment of the structural model using data and predictions back transformed to the original scale for model base_SE_r01.mlxtran. (A) Data by treatment and flare or non‐flare design, (B) visual predictive check showing 90% prediction intervals for the median, 10th, and 90th percentiles, compared to the empirical percentiles, and (C) individual fits of the data to the model predictions by study with ARM = 0 showing placebo and ARM = 1 showing naproxen.

For use in Monolix, the data should contain columns for study, arm, time, observation, weighting (sample size or empirical standard error), and potential covariates. Transforming the observation column will be explained below. For the exploratory data analysis, load the dataset in Monolix and tag the columns in the “Data” tab (Table 1). With summary‐level data, tag the column for study as ID and that for treatment arm as OCCASION, as BSV is analogous to inter‐individual variability and BTAV to inter‐occasion variability in an individual‐data population model. Monolix includes a built‐in visualization module (Figure 2) useful for data exploration, and the plots can be stratified by covariates, here treatment [TRT] and flare design [FLARE], to inform whether and how to incorporate them in the model. There is evidence of a placebo effect, given pain scores improve over time in both the active and placebo arms and that the baseline pain score is higher for the flare design (Figures 1A and 2).

TABLE 1.

Dataset columns and column tagging in Monolix for Case Study 1.

Dataset variable	Description	Monolix data tag
STUD	Study or publication index equivalent to the ID identifier (either integer or string) in a population analysis	ID
ARM	Placebo or active drug identifier (0 for placebo and 1 for naproxen). Must be an integer	OCCASION
TRT	Name of the administered drug: placebo or naproxen	CATEGORICAL COVARIATE
TIME	Independent variable time since first dose in weeks	TIME
Nindiv	Number of individuals in the respective arm of the study	IGNORE
FLARE	Indicator variable, 1 = flare design, 0 = non‐flare design	CATEGORICAL COVARIATE
WP	WOMAC pain score (original scale)	IGNORE a
WPSE	Observed SE for the WOMAC pain score	REGRESSOR
transWP	WP transformed by dividing by the WPSE column	OBSERVATION
INT_TRT_FLARE	Interaction variable, 1 = naproxen with flare design, 0 = otherwise	CATEGORICAL COVARIATE

Abbreviations: ID, identifier; SE, standard error; WOMAC, Western Ontario and McMaster Universities Osteoarthritis Index.

^a Can be tagged as OBSERVATION for exploratory data analysis.

FIGURE 2.

Data visualization in Monolix for the observed data shown on the transformed scale (i.e., divided by SE). The left panel shows the options to configure the plot, including stratifying and/or coloring by covariates, and the right panel shows the other plots available. The plots are interactive, so hovering over a datapoint (here the point at approximately [6, 41]) will show that it comes from study 17 and arm 0 and highlight the other points from the same study in yellow, as well as highlighting the same study in other plots.

2.2.2. Longitudinal Nonlinear Mixed‐Effects Model Description

Based on the shape of the data and following Boucher and Bennetts [11], an E _max model (assuming increasing effect over time followed by a plateau) can be fit to the observed pain score data, $y_{\mathit{ijk}}$, for study i, time j, and treatment arm k:

$$y_{\mathit{ijk}}=E{0}_i-E_{{\max }_{\mathit{ik}}}\frac{t_{\mathit{ijk}}}{t_{\mathit{ijk}}+\mathit{ET}{50}_k}+{\epsilon }_{\mathit{ijk}},$$ (1)

where $E{0}_{\mathrm{i}}$ is the baseline WOMAC score $\left(\mathrm{at}t=0\right)$, $E_{{\max }_{\mathit{ik}}}$ is the maximum effect over time, $\mathit{ET}{50}_{\mathrm{k}}$ is the time to reach $50\%$ of $E_{\max }$, $t_{\mathit{ijk}}$ is time, and ${\epsilon }_{\mathit{ijk}}$ is the residual error.

Although the response variable could be modeled with a logit‐normal distribution to take into account the data limits, the data are unimodal and do not approach the boundaries, so a normal distribution is a reasonable approximation (fitting a bounded response is demonstrated in Case Study 2). For summary‐level data, the residual error varies based on the error in the underlying studies and is largely driven by the sample size. Observed standard errors were available or imputed for all studies [11], thus could be used for weighting rather than the sample size N _ijk . The residual error ${\epsilon }_{\mathit{ijk}}$ is thus normally distributed with mean 0 and fixed variance ${\mathrm{SE}}_{\mathit{ijk}}^2={\mathrm{SD}}_{\mathit{ijk}}^2/N_{\mathit{ijk}}$.

For the statistical model, a normal distribution is assumed for the parameters E ₀ and E _max. While a logit‐normal distribution could in principle be used for these parameters, a normal distribution is chosen instead for simplicity (like the response error model). Since ET₅₀ must be strictly positive, it is modeled using a log‐normal distribution. To consider BSV for E ₀ and E _max, random effects are included as ${\eta }_{E0,i}~\mathcal{N}\left(0,{\omega }_{E0}^2\right)$ and ${\eta }_{E\max ,i}~\mathcal{N}\left(0,{\omega }_{E\max }^2\right)$, respectively. The BSV for ET₅₀ is ignored for simplicity. Here we ignore BTAV, but Case Study 2 demonstrates including BTAV and how to weight it. Given the observed differences in baseline value and treatment effect for flare vs. non‐flare designs, a covariate effect for flare designs is fit for E ₀ and E _max using an indicator variable, FLARE (non‐flare = 0, flare = 1). To compare naproxen to placebo (question 1), a covariate effect of treatment, TRT (naproxen or placebo), is fit to E _max and ET₅₀. Therefore, E ₀ is modeled as

$$E{0}_i=E{0}_{\mathrm{pop}}+{\beta }_{E0\_\text{FLARE}}\left[\text{if FLARE}=1\right]+{\eta }_{E0,i},$$ (2)

where $E{0}_{\mathrm{pop}}$ is the population parameter for the baseline WOMAC score for a non‐flare design, ${\beta }_{E0\_\text{FLARE}}$ is the difference in the baseline pain score for a flare design, and ${\eta }_{E0,i}$ represents the random effects for BSV. E _max is modeled as

$$\begin{array}{c}{E}_{{\max }_{\mathit{ik}}}=E\mathrm{ma}{\mathrm{x}}_{\mathrm{pop}}+{\beta }_{E\max \_\mathrm{TRT}}\left[\text{if}\mathrm{TRT}=\text{Naproxen}\right]\hfill \\ +{\beta }_{E\mathrm{ma}{\mathrm{x}}_{\text{FLARE}}}\left[\text{if FLARE}=1\right]+{\eta }_{E\max ,i}\hfill \end{array},$$ (3)

where $E\mathrm{ma}{\mathrm{x}}_{\mathrm{pop}}$ is the population parameter for the maximum treatment effect for placebo with a non‐flare design, ${\beta }_{E\max \_\mathrm{TRT}}$ is the difference between naproxen compared to placebo, ${\beta }_{E\max \_\text{FLARE}}$ is the difference between the flare and non‐flare design with respect to the maximum treatment effect, and ${\eta }_{E\max ,i}$ represents the random effects for BSV. $E_{{\max }_{\mathit{ik}}}$ varies between arms due to the treatment covariate though BTAV is not included. The model for ET₅₀ is

$$\log \left(\mathrm{ET}{50}_k\right)=\log \left(\mathrm{ET}{50}_{\mathrm{pop}}\right)+{\beta }_{\mathrm{ET}50\_\mathrm{TRT}}\left[\text{if}\mathrm{TRT}=\text{Naproxen}\right],$$ (4)

where $\mathrm{ET}{50}_{\mathrm{pop}}$ is the population parameter for the time to reach $50\%$ of $E_{\max }$ for placebo and ${\beta }_{\mathrm{ET}50\_\mathrm{TRT}}$ is the difference between naproxen and placebo for time to half‐maximum efficacy. Thus, we can answer the first question, comparing treatment effect between naproxen and placebo, with the parameters ${\beta }_{\mathrm{ET}50\_\mathrm{TRT}}$ and ${\beta }_{E\max \_\mathrm{TRT}}$. For the second question, the impact of a flare design on treatment effect, we fit a second model incorporating an interaction between treatment and flare design. To model the interaction, we add a covariate on E _max, ${\beta }_{E\max \_\mathrm{TRT}\_\text{FLARE}}$, based on the indicator variable TRT_FLARE which is 1 in the case of a flare design and naproxen, and 0 in all other cases.

2.2.3. Monolix Implementation

Incorporating the weighting, the residual error is ${\epsilon }_{\mathit{ijk}}~\mathcal{N}\left(0,{\mathrm{SE}}_{\mathit{ijk}}^2\right)$ with SE _ijk varying across study, arm and possibly time. However, in Monolix, the residual error must be defined with constant variance. Thus, we can redefine the residual error variance as SE _ijk $\times \tilde{\epsilon }$, where $\tilde{\epsilon }$ has constant variance, $\tilde{\epsilon }~\mathcal{N}\left(0,a\right)$. We can further set the parameter a equal to 1 to enforce the assumption that the residual error is fully captured by SE _ijk . Then, starting with Equation (1) and dividing both sides by SE _ijk gives

$$\begin{array}{c}{y}_{\mathit{ijk}}/{\mathrm{SE}}_{\mathit{ijk}}=f\left(t_{\mathit{ijk}},\mathrm{E}{0}_{\mathit{ik}},E_{{\max }_{\mathit{ik}}},\mathrm{ET}{50}_k\right)/{\mathrm{SE}}_{\mathit{ijk}}+\tilde{\epsilon }\hfill \end{array}.$$ (5)

Therefore, we need to create a new column in the dataset for the observation y _ijk /SE _ijk and divide the prediction $f$ in the structural model by SE _ijk (transf in the structural model below). To do so, the SEs need to be included in the dataset and tagged as REGRESSOR and then declared in the structural model to indicate that the variable is read from the corresponding dataset column.

The BSV on E ₀, as well as the FLARE covariate, which varies only among studies, can be included in the user interface (UI). However, due to how occasions are implemented in Monolix, it is not possible to add covariates varying between arms on parameters having BSV but no BTAV. This is the case for TRT on E _max. Thus, for E _max, we separate the fixed effect E _max (on which covariates can be added via the UI) and the BSV random effects eta_BSV_Emax. The E _max parameter with both fixed and random effects (Emaxre) is then reassembled in the structural model. The mlxtran code for the structural model is as follows, with t representing time:

[LONGITUDINAL]input = {E0, Emax, ET50, eta_BSV_Emax, WPSE}WPSE = {use=regressor}EQUATION:Emaxre = Emax + eta_BSV_Emaxf = E0 – Emaxre × t/(t + ET50)transf = f/WPSEOUTPUT:output = transf

Save the structural model as text file and load it in the “Structural Model” tab. Then go to the “Statistical model & Tasks” tab and set the error model to constant. With a constant error model, the equation for the transformed response (i.e., divided by SE) is $\text{transWP}=\text{transf}+a\times \tilde{\epsilon }$ with $a$ the residual variance and ${\tilde{\epsilon }}_{\mathit{ijk}}~\mathcal{N}\left(0,1\right)$. Select the appropriate parameter distributions (normal for E ₀ and E _max, log‐normal for ET₅₀, and normal for eta_BSV_Emax). Random effects for BSV should be enabled at the ID level (i.e., study level) for E ₀ and eta_BSV_Emax. Finally, the covariate effects should be added: FLARE on E ₀ and E _max and TRT on E _max and ET₅₀. By default, naproxen is the reference category for the TRT covariate. In order to consider “placebo” as reference, use the “Add covariate” button to add a new covariate based on TRT with placebo as the reference. Next, in the “Initial estimates” tab, initial values should be selected to capture the key features of the model [16], here the baseline value at time 0 for flare and non‐flare designs and the decline in pain score observed for both treatments. Initial values can be selected using the “auto‐init” function, which will compute initial values from pooled data. However, covariate effects are not included in auto‐init, and non‐zero values for the covariate effects are necessary for good model convergence in this case. We thus select initial values by visual examination (Figure 2).

The parameter eta_BSV_Emax is itself composed of a population component (eta_BSV_Emax_pop) and random effects. Because this parameter should only represent the random effects, eta_BSV_Emax_pop should be fixed to 0 using the gear icon and selecting “fixed.” Finally, we assume that the residual error is fully captured by the SEs, which are already included in the weighting of the observations and model prediction, and thus the error parameter a can be fixed to 1.

2.2.4. Parameter Estimation

After saving the project, launch the estimation tasks with the “run” button. This will estimate the population parameters using the stochastic approximation expectation maximization (SAEM) algorithm with default settings and the study‐specific and arm‐specific parameters (equivalent to the empirical Bayes estimates for individual‐level data). Standard diagnostic plots are automatically created, including individual fits, observations vs. predictions, and residuals, which can be used to evaluate model fit (see Traynard et al. [16] for more details). Before proceeding further with model analysis, it is important to verify the stability of the likelihood and of the parameter estimates. This can be performed using the “Convergence assessment” task in Monolix, which, by default, launches five runs with different random seeds and samples initial values for parameters from specified ranges.

2.3. Model Results

The likelihood estimation is stable, varying less than one unit across five runs started from randomly sampled initial values. Most parameter estimates are also stable (Figure S1A), including the treatment effect for naproxen (research question 1). However, there is more variation in both Emax_pop and ${\beta }_{E\max \_\text{FLARE}}$, showing the difficulty of partitioning the placebo effect between flare and non‐flare designs. This is not an issue for addressing our two research questions, which are focused on ${\beta }_{\mathrm{ET}50\_\mathrm{TRT}}$, ${\beta }_{E\max \_\mathrm{TRT}}$, and ${\beta }_{E\max \_\mathrm{TRT}\_\text{FLARE}}$.

Model diagnostics show that the model captures the data well, and no model misspecification is visible (Figures 1B,C and S2). Diagnostic plots in the UI show the transformed observations. The provided R code (Supporting Information) includes generic functions to back‐transform data and model predictions to the original WOMAC pain scale, enhancing interpretability for individual fits (i.e., study‐ and arm‐level fits) and visual predictive checks (Figure 1B,C).

For Question 1, naproxen shows a 0.8‐point greater maximum effect (from the parameter beta_Emax_tTRT_G_Naproxen) with a 95% confidence interval (CI) [0.67, 0.92] and a 2.2‐day (95% CI [1.3, 3.7]) shorter time to 50% of maximum effect relative to placebo (calculated as exp(beta_ET50_tTRT_G_Naproxen) × 7 due to ET₅₀ being log‐normally distributed and to convert from weeks to days; Table S1). However, caution must be exercised regarding the reliability of the estimate of the time to 50% of maximum effect, as data collection began 1 week after treatment initiation.

For Question 2, when the interaction between treatment and flare design is added to the model, its estimated value is small (beta_Emax_INT_TRT_FLARE_1 = −0.036), and not significantly different from zero (Wald test: p = 0.72), indicating there is no additional naproxen effect in a flare design compared to a non‐flare design, relative to placebo (Table S2).

We compared our results to those of Boucher and Bennetts [11] using a matching reparameterized model (ET₅₀ log‐transformed and effect E_max positive in the structural model, see model files in the Supporting Information). Monolix parameter estimates closely match those from NONMEM, the NLME R package [21], and BUGS (Table S3) with slightly larger differences for Emax_pop and beta_Emax_FLARE_1. As noted during the convergence assessment, these parameters are less stably estimated and sensitive to initial estimates, but consistent with the range of values reported in Boucher and Bennetts [11] across the three applications. The advantage of performing the convergence assessment is a clearer picture of parameter identifiability. The model struggles to exactly partition the placebo effect between flare and non‐flare designs, though this is not a focus of the analysis, and parameters relating to the effect of naproxen are all well estimated.

2.4. Simulations

To inform a possible future trial design with regard to study duration, we can simulate naproxen versus placebo effects over time, excluding BSV (i.e., simulating a typical study) and residual error, and including population parameter uncertainty. We therefore perform 200 study replicates using parameter samples from the variance–covariance matrix of the estimates of the non‐interaction model. Because the interaction term is not statistically significant, only a non‐flare design is simulated.

To perform the simulations, export the final project from Monolix to Simulx. This automatically creates elements in the “Definition” tab of the new Simulx project, including the structural and statistical model, estimated parameter values including uncertainties, and covariate and treatment information. These elements can be used in the “Simulation” tab to set up a simulation or new elements can be created to define a new simulation scenario.

Several elements need to be created for the simulation. We will simulate an idealized scenario of an infinitely large population without BSV or residual error. Create a regular output (i.e., without residual error) in the “Output” tab for the intermediate output f (the untransformed prediction) every week for 12 weeks. Then, to simulate the two treatments, create two covariates in the “Covariate“ tab, one with TRT = Naproxen and one with TRT = Placebo, both with FLARE = 0. Finally, in the “Regressor” tab, create a regressor for WPSE = 1, which will not have an effect as we are using the unweighted prediction for the output. Note that, as we will not resimulate the existing studies, the occasion element specifying the IDs and ARMS of the studies in the data can be deleted.

Next, go to the “Simulation” tab where we will set up simulation groups for the two treatments. First, set the parameters to “mlx_PopUncertain.” These are the population parameters estimated by Monolix including the variance–covariance matrix for the uncertainty in the parameter estimation. This means that for each replicate simulation, parameter values will be drawn from this distribution and the uncertainty in the population parameter estimates will be propagated to the simulation predictions. Next, set the output and regressor to the newly created elements. Finally, add a second simulation group, and click the arrow next to “Covariates” to set it to differ between groups, with one group set to naproxen and the other to placebo. Now we can set the number of individuals (i.e., studies) to 1 and the number of replicates to 200 to simulate 200 replicate trials, each with parameters sampled with uncertainty.

In order to facilitate the comparison, in the “Outcomes & Endpoints” section, add an outcome for weeks 1, 2, and 4 using the created regular output with the output processing “at a custom point per id” for the respective time points. Then enable the group comparison, using a direct comparison with “difference < 0” and placebo as the reference group. This will give a true/false result for each simulated trial at each of the defined time points for whether naproxen had a lower predicted pain score than placebo.

After saving and running the project, the results are available in the form of tables and plots. In the “Results” tab, the different subtabs contain tables for the parameters and outputs in each simulation, for the specified outcomes and endpoints, and the differences and success/failure for the group comparisons. In the “Plots” tab, plots are available for the distribution of the pain scores over time and box plots comparing the endpoints at specific time points. The simulation results show little variation in the difference between naproxen and placebo over time, with naproxen having a median (95% CI) pain score of 3.82 (2.30, 5.55), 3.69 (2.16, 5.49), and 3.61 (2.07, 5.45) at weeks 1, 2, and 4, compared to 4.68 (3.55, 6.35), 4.59 (3.26, 6.46), and 4.51 (3.09, 6.48) for placebo (Figure 3). Results at 1 week were nearly identical to those at 2 or 4 weeks, with naproxen having a larger reduction in the pain score than placebo in 78.5%, 77.5%, and 78.0% of trials, respectively.

FIGURE 3.

Results from simulations of clinical trials from fitted model with population parameter uncertainty. (A) WOMAC pain scores through time showing the median, 80% and 95% confidence intervals across 200 simulated trials for naproxen and placebo. (B) Difference between WOMAC pain scores between naproxen and placebo at weeks 1, 2, and 4.

2.5. Summary

We replicated the Boucher and Bennetts [11] case study to show how the same longitudinal E _max model could straightforwardly be implemented in Monolix using the same simplifying assumptions: normal distribution of WOMAC pain scores, including only BSV, and no covariates beyond those used to model study design. In the next case study, we show a more complicated example including both BSV and BTAV and bounding the response. With regard to the research questions, naproxen showed faster onset and greater maximum effect than placebo, and there was no additional benefit from naproxen with a flare design. Simulations taking into account the parameter uncertainty show that results at weeks 2 and 4 are very similar, suggesting that a study duration of 2 weeks could be sufficient for future trials (and indeed two of the existing studies used a duration of 2 weeks).

3. Case Study 2: Canakinumab, a Drug Candidate for Rheumatoid Arthritis

This case study builds on Demin et al.'s [8] analysis that evaluated canakinumab's efficacy and market positioning in rheumatoid arthritis (RA) [22]. RA is a chronic systemic autoimmune disease, characterized by joint inflammation, pain, and potential loss of function, affecting up to 1% of the global population [23]. Methotrexate, a disease‐modifying anti‐rheumatic drug, is the first line of treatment for RA, which, if ineffective, may be combined with biologics [23]. Despite the higher efficacy of biologics, limited remission rates highlight an unmet need, which the development of canakinumab aimed to address.

A placebo‐controlled, parallel‐group dose‐finding Phase 2b study investigating canakinumab versus placebo, both in addition to methotrexate, was conducted in patients with active RA despite stable methotrexate treatment [24]. Results showed that the addition of canakinumab to methotrexate treatment provided statistically significant benefit compared to the addition of placebo. Despite positive Phase 2b clinical study results, canakinumab's efficacy versus approved treatments for RA was unclear, which prompted a longitudinal MBMA to support a go/no‐go decision for its clinical development.

This case study focuses on a selection of clinical studies involving two biologic treatments for RA available at the time (abatacept [25] and adalimumab [26]) compared to canakinumab. These two compounds were selected because they had the most extensive published data. As in Demin et al. [8], we focus on longitudinal measurements of the 20% improvement in the American College of Rheumatology response criteria (ACR20) [27]. The ACR20 represents a 20% reduction in tender or swollen joint counts in a patient and is a binary categorical endpoint, representing either clinical response (achievement of at least 20% reduction) or no response. Note that canakinumab was not compared to abatacept and adalimumab in head‐to‐head trials, and that the canakinumab data only go until 12 weeks while typical RA studies last 24 weeks. An MBMA approach is thus warranted.

This case study seeks to answer the following question:

1. Is there a greater treatment effect for canakinumab compared to existing approved treatments in RA?

3.1. Model Development

3.1.1. Data

Clinical studies with adalimumab and abatacept were identified based on the results of the literature search performed by Demin et al. [8] Among these, only studies including patients unresponsive to methotrexate on approved dosing regimens were selected to be included in the current analysis. Two additional relevant studies published later were identified; thus, a total of eight published studies comparing adalimumab or abatacept plus methotrexate to placebo plus methotrexate were included for analysis [28, 29, 30, 31, 32, 33, 34, 35]. An overview of publication sources and design characteristics for each study is presented in Table S4.

The published summary‐level data from the selected studies were digitized (WebPlotDigitizer, version 4) [36] and extracted for the analysis dataset. For canakinumab data, mean ACR20 was calculated at each time point for each treatment arm. Summary‐level ACR20 represents the proportion of subjects meeting the criteria and can range from 0% to 100%, that is, it is a bounded continuous variable. In contrast, individual‐level ACR20 is a binary categorical variable.

Following analysis dataset assembly, the summary‐level data should be explored after loading the dataset into Monolix and tagging the columns (Table 2). In the exploratory plots generated by Monolix, the observed data can be stratified by the covariates DRUG and TRT (Figure 4A). Note that even in the “placebo” arms, ACR20 improves over time due to the methotrexate treatment. In addition, we clearly see that the treatment methotrexate + biologic drug has better efficacy compared to methotrexate + placebo. For canakinumab, only the Phase 2 study is available.

TABLE 2.

Dataset columns and column tagging in Monolix for Case Study 2.

Dataset variable	Description	Monolix data tag
STUDY	Study or publication index equivalent to the ID identifier in a population analysis	ID
ARM	Placebo or active drug identifier (0 for placebo + methotrexate and 1 for drug + methotrexate)	OCCASION
DRUG	Name of the administered drug in the arm in addition to methotrexate: • Placebo • Adalimumab • Abatacept • Canakinumab	CATEGORICAL COVARIATE
DRUGinteger	Numeric form of DRUG variable	CATEGORICAL COVARIATE
TIME	Independent variable time since first dose in weeks	TIME
Y	Dependent variable ACR20 (proportion of patients with at least a 20% improvement in their symptoms)	IGNORE a
transY	Dependent variable logit transform of the Y variable weighted by N arm (transY = logit(Y) × √N arm)	OBSERVATION
Publication	First author of the publication from which the data was extracted	IGNORE
Narm	Number of subjects in the study arm	REGRESSOR
TRT	Name of the investigative drug in the given study	CATEGORICAL COVARIATE
DiseaseDuration	Mean disease duration at study start (years)	CONTINUOUS COVARIATE
CReactiveProtein	Mean C‐reactive protein level (mg/dL)	CONTINUOUS COVARIATE
SwollenJointCount	Mean number of swollen joints	CONTINUOUS COVARIATE

Abbreviations: ACR20, 20% improvement in the American College of Rheumatology response criteria; ID, identifier; Narm, number of individuals per arm.

^a Can be tagged as OBSERVATION for exploratory data analysis.

FIGURE 4.

Data and assessment of the structural model and treatment effects using data and predictions back transformed to the original scale. (A) Data by treatment, (B) individual model predictions versus observed ACR20 data, (C) visual predictive check showing 90% prediction intervals for the median, 10th, and 90th percentiles, and (D) evaluation of treatment effects via Wald tests as displayed in the UI.

3.1.2. Longitudinal Nonlinear Mixed‐Effects Model Description

Based on the visual exploration of the time course of the summary‐level ACR20 data across the different studies and arms in the dataset, an E _max structural model is used:

$$f\left(t\right)=E_{\max }\frac{t}{t+T_{50}},$$ (6)

where f(t) represents the proportion (between 0 and 1) of patients achieving ACR20 as a function of time t in weeks, E _max describes the maximum effect which is assumed to be logit‐normally distributed, and T ₅₀ is the time (weeks) to reach 50% of the maximum effect, assumed to be log‐normally distributed. No baseline parameter is included because the proportion of patients meeting the ACR20 criteria is zero at the beginning of the studies.

To estimate BSV and residual error, while ensuring predictions remain bounded between 0 and 1, the response prediction f(t) is transformed with a logit function (i.e., the response is assumed to follow a logit‐normal distribution). The estimated residual error term for the logit‐transformed model prediction is normally distributed and the proportion of responders in the i‐th study at the j‐th time point and the k‐th treatment arm is given by

$$\text{logit}\left(y_{\mathit{ijk}}\right)=\text{logit}\left(E_{\max ,\mathit{ik}}\frac{t_{\mathit{ijk}}}{t_{\mathit{ijk}}+T_{50,\mathit{ik}}}\right)+{ϵ}_{\mathit{ijk}},$$ (7)

where y _ijk is the dependent variable on the original scale (0–1), while $E_{\max ,\mathit{ik}}$ and $T_{50,\mathit{ik}}$ are the maximum effect of the drug and the time to reach 50% of the maximum effect, and ${ϵ}_{\mathit{ijk}}$ is the residual error term distributed following a normal distribution:

$${ϵ}_{\mathit{ijk}}=\mathcal{N}\left(0,\frac{{\sigma }^2}{N_{\mathit{ik}}}\right),$$ (8)

where ${\sigma }^2$ is the variance of the residual error variability, and N _ik is the number of individuals per arm used for weighting. The weighting could also vary with time for dropouts or disease progression, for example.

To focus on estimating and comparing the treatment‐specific E _max efficacy parameter, BSV and BTAV are estimated for E _max only. To prevent estimates from exceeding 1 (i.e., 100%), the E _max parameter follows a logit‐normal distribution (bounds 0–1) using a logit transformation:

(9)

where ${\eta }_{E_{max},i}^{\left(\mathrm{BSV}\right)}$ and ${\eta }_{E_{max},\mathit{ik}}^{\text{(BTAV)}}$ are the BSV for study i and BTAV for arm k of study i, respectively, and $\text{logit}\left(x\right)=\log \left(x/(1-x)\right)$. This ensures that under the inverse‐logit back‐transformation, ${\text{logit}}^{-1}\left(y\right)=e^y/\left(e^y+1\right)$, E _max estimates are between 0 and 1. The BSV and BTAV terms for the E _max parameter follow normal distributions:

$${\eta }_{E_{max},i}^{\left(\mathrm{BSV}\right)}~\mathcal{N}\left(0,{\omega }_{E_{max}}^2\right), {\eta }_{E_{max},\mathit{ik}}^{\text{(BTAV)}}~\mathcal{N}\left(0,\frac{{\gamma }_{E_{max}}^2}{N_{\mathit{ik}}}\right),$$ (10)

where ${\omega }_{E_{max}}^2$ is the variance of the BSV and ${\gamma }_{E_{max}}^2/N_{\mathit{ik}}$ is the variance of the BTAV including the weighting. Only the BTAV and residual error model components are weighted, as these are the variabilities affected by the number of individuals over which the study results are summarized. Unlike Case Study 1, the number of individuals per arm was chosen for weighting, as the standard error was not available [11].

3.1.3. Monolix Implementation

Like Case Study 1, to ensure a constant random‐effects model, the summary‐level ACR20 data in the analysis dataset (variable Y) and the model predictions need to be transformed for implementation in Monolix. The weighting can be coded in Monolix by defining the number of subjects per treatment arm (Narm) as a regressor (see Table 2), which can vary with time if necessary. Starting from Equation (7), we replace ${ϵ}_{\mathit{ijk}}$ with $\widetilde{{\mathrm{ϵ}}_{\mathit{ijk}}}/\sqrt{N_{\mathit{ik}}}$, where $\widetilde{{ϵ}_{\mathit{ijk}}}$ is a constant unweighted residual error term, then multiply both sides by the number of subjects in each arm:

$$\begin{array}{c}\text{logit}\left(y_{\mathit{ijk}}\right)=\text{logit}\left(f\left(t_j,E_{\mathit{max},\mathit{ik}},T_{50,\mathit{ik}}\right)\right)+\frac{\widetilde{{\mathrm{ϵ}}_{\mathit{ijk}}}}{\sqrt{N_{\mathit{ik}}}}\hfill \end{array}$$ (11)

$$⟺\sqrt{N_{\mathit{ik}}}\times \text{logit}\left(y_{\mathit{ijk}}\right)=\sqrt{N_{\mathit{ik}}}\times \text{logit}\left(f\left(t_j,E_{\mathit{max},\mathit{ik}},T_{50,\mathit{ik}}\right)\right)+\widetilde{{\mathrm{ϵ}}_{\mathit{ijk}}}.$$ (12)

Therefore the summary‐level ACR20 data are transformed as $\text{trans}{Y}_{\mathit{ijk}}$ = $\sqrt{N_{\mathit{ik}}}\times \text{logit}\left(y_{\mathit{ijk}}\right)$ and the model prediction (pred in the structural model below) as $\text{trans}f\left(t_j,E_{\max ,\mathit{ik}},T{50}_{\mathit{ik}}\right)=\sqrt{N_{\mathit{ik}}}\times \text{logit}\left(f\left(t_j,E_{\max ,\mathit{ik}},T_{50,\mathit{ik}}\right)\right)$.

We also need to weight the BTAV. Because the random effects defined in the UI need to have a constant standard deviation, we separate the fixed (EmaxFE) and the BSV and BTAV random effects (etaBSVEmax and etaBTAVEmax) and reassemble the E _max parameter in the structural model, and weight the BTAV term (etaBTAVEmax). In addition, to ensure that E _max stays bounded between 0 and 1, we apply the logit transformation.

The structural model code for the summary‐level longitudinal ACR20 model is shown below:

[LONGITUDINAL]; Specification of structural model parameters and Narm as input:;; EMAXFE: Emax fixed effect;; T50: T50 fixed effect;; etaBSVEmax: BSV for Emax;; etaBTAVEmax: BTAV for Emax;; Narm: dataset variable indicating the number of subjects in each arminput = {EmaxFE, T50, etaBSVEmax, etaBTAVEmax, Narm}; Narm is tagged as a regressor to weight the BTAV term and residual errorNarm = {use = regressor}; Definition of Emax parameter transformation, structural ACR20; model, and model predictionsEQUATION:; Logit transformation of Emax fixed effecttEmax = logit(EmaxFE); Adding BSV and weighted BTAV on tEmaxtEmaxRE = tEmax + etaBSVEmax + etaBTAVEmax/sqrt(Narm); Back-transforming to the original scale using the inverse logit functionEmax = exp(tEmaxRE)/(1 + exp(tEmaxRE)); Longitudinal model for the summary-level ACR20 with time defined as tACR20 = Emax × (t/(T50 + t)); Adding protection against logit(0) (undefined) when t = 0ACR20sat = min(max(ACR20,0.01),0.99); Transforming predicted ACR20 including the residual error weightingpred = logit(ACR20sat) × sqrt(Narm)OUTPUT:output = pred

To implement the model in Monolix, add the structural model code in the “Structural model” tab. Then, in the “Statistical model & Tasks” tab, set the error model to constant and define the parameter distributions (Figure 5). Select a logit‐normal distribution with limits between 0 and 1 for EmaxFE, a log‐normal distribution for T₅₀, and a normal distribution for etaBSVEmax and etaBTAVEmax. EmaxFE represents only the fixed effects, so random effects should be disabled, as well as for T₅₀. The etaBTAVEmax is the BTAV (analogous to inter‐occasion variability), so random effects need to be disabled at the study level (shown as “ID” in Monolix's UI) and enabled at the ARM level. Then, since both etaBSVEmax and etaBTAVEmax represent only random effects, their means must be set to zero and fixed in the “Initial estimates” tab.

FIGURE 5.

The “Statistical model & Tasks” tab in Monolix, showing the estimation tasks, the observation model to set the error model and distribution for the response, and the individual model, to set the distributions for the parameters in the model and any included random effects, correlations, and covariates.

To complete the model, DRUG must be added as a covariate on EmaxFE and T₅₀ to estimate separate EmaxFE and T₅₀ values for each treatment (placebo, adalimumab, abatacept, and canakinumab) in the “Statistical model & Tasks” tab. DRUG can be redefined as tDRUG with placebo as the reference using the “Add covariate” button.

3.1.4. Parameter Estimation

Reasonable initial values should be chosen for each parameter in the “Initial estimates” tab, either manually or with the “auto‐init” function. Next, save the Monolix project, and launch the parameter estimation task. As before, it is important to assess model convergence and stability by running the parameter estimation with different initial values and random number seeds. To do so, run the convergence assessment task, making sure to select all parameters and enable the calculation of standard errors and log‐likelihood.

The robustness of estimates can be evaluated by assessing the variability from run to run in the convergence trajectories, final estimates, and log‐likelihood values. Using the default parameter estimation settings, the convergence assessment shows that the covariate effects tend to converge to very different values from run to run (Figure S1B). This can be improved by changing the algorithm settings.

The SAEM algorithm draws parameter values from their marginal distributions, which is only possible for parameters with variability. For parameters without variability, such as EmaxFE and associated covariate effects, several methods can be used for estimation. By default, these parameters are optimized using the Nelder–Mead simplex algorithm (“no variability” option in Monolix). If the default method leads to unstable trajectories, the estimation method can be adjusted to opt for either: (1) artificial variability (i.e., random effects) that is progressively forced to zero in the first stage and the Nelder–Mead simplex algorithm in the second stage (“variability at first stage”) or (2) artificial variability that is progressively forced to zero so that the parameter has no variability at the end of the estimation process (“add decreasing variability”). In this case study, selecting the “variability in the first stage” method results in satisfactory convergence (Figure S1C).

3.2. Model Results

The estimated parameters are shown in the “Pop param” tab of the “Results” (as well as Table S5). Most parameters are estimated with good confidence (low %RSE) except for a few covariate effects, including T₅₀ and E _max for canakinumab. The significance of the covariate effects can be investigated further in the “Tests” tabs, where the results of Wald tests [16] are shown, assessing if the estimated parameters representing the difference between the placebo and the other drug categories are significantly different from zero, given their uncertainty (Figure 4D).

The significant p‐values (α = 0.05) of the Wald test of the E _max estimates for abatacept (p < 2.2e‐16) and adalimumab (p = 3.0e‐11) indicate a statistical difference from placebo, unlike canakinumab (p = 0.24). Note that when analyzing the richer individual data for canakinumab, the efficacy of canakinumab was significantly better than that of placebo [24]. To see E _max and T₅₀ estimates by category, select the option “display fixed effects by category” in the “Pop param” table. The estimated coefficients for T₅₀ are similar for all compounds except adalimumab. Consequently, we can make the modeling assumption of an identical effect for abatacept, canakinumab, and placebo on T₅₀ by creating a new covariate Grouped_Drug with two categories (group 1: abatacept, canakinumab, and placebo; group 2: adalimumab). While this assumption can allow estimating the remaining parameters with improved reliability, one disadvantage is that we can no longer compare T₅₀ for canakinumab to other treatments which could be of interest [8]. After running this new model, the Wald test suggests the new covariate should remain in the individual model on T₅₀, as the difference between adalimumab and the grouped canakinumab/abatacept/placebo is significant (p = 2.7e‐9). As a side effect of the grouped covariate categories on T₅₀, the estimate of the canakinumab effect on E _max becomes more precise (Wald test p = 0.008, RSE = 38%), suggesting the maximum effect of canakinumab is different from placebo, though with a larger confidence interval than other treatments due to more limited data.

The estimated E _max BSV (omega_etaBSVEmax) is 0.25, and the estimated E _max BTAV (gamma_etaBTAVEmax) is 2.00. The BTAV may seem high but recall the BTAV is expressed as ${\eta }_{E\max ,\mathit{ik}}^{\left(\text{BTAV}\right)}~\mathcal{N}\left(0,\frac{{\gamma }_{E\max }^2}{N_{\mathit{ik}}}\right)$. Because it depends on the number of individuals per arm, we can take a typical value of 200 individuals, then dividing 2.00 by the square root of 200 yields a BTAV of approximately 0.14. Note that since the means of random effects (BSVEmax_pop and BTAVEmax_pop) are fixed to zero, the percent coefficient of variation (CV%) displayed in the Monolix UI is “infinity”, as CV = mean/SD.

Model diagnostics show no apparent model misspecification. Like for Case Study 1, the data and model predictions can be back‐transformed to the original scale for easier interpretation, using the provided R code which includes generic functions to back‐transform predictions used to plot the individual fits and visual predictive check (Figure 4C).

The BSV can stem from the different inclusion criteria across studies. We have information on population characteristics for each included study, such as the means of disease duration, C‐reactive protein level, and swollen joint count, which can be tested to explain part of the BSV. These covariates could be tested univariately on the E _max parameter in a forward selection or in a backward elimination approach. Here, a backward elimination approach is used, including all covariates in the model on E _max simultaneously, followed by evaluation for exclusion. Both the p‐values from the Wald test and the likelihood ratio test results show that none of the tested covariate effects is significant predictors in the model (disease duration p = 0.28, C‐reactive protein level p = 0.75, swollen joint count p = 0.14).

Thus, the final E _max model has BSV and BTAV on E _max and no variability on T₅₀. No covariates are included on model parameters, except the drug effect. The fixed effect EmaxFE incorporates a drug effect with the four drug categories (abatacept, adalimumab, canakinumab, and placebo), with placebo as the reference. The T₅₀ parameter incorporates a drug effect using a different version of the covariate that consists of only two categories (adalimumab versus the reference abatacept/canakinumab/placebo).

3.3. Simulations

The final model can be used to answer our research question: to assess the efficacy of canakinumab compared to abatacept and adalimumab, relative to placebo. While we could just compare E_max values, we can also take into account the uncertainty of estimation. To assess “true efficacy,” we can ignore BSV, BTAV, and residual error (i.e., setting to zero) to simulate an infinitely large population for 52 weeks. To consider the uncertainty of the estimated population parameters, we will draw multiple sets of population parameters from their uncertainty distribution (i.e., the variance–covariance matrix of the estimates).

To do this, export the final project from Monolix to Simulx, which will include the structural model and statistical model, estimated population parameter values, uncertainty distributions, and original dataset as automatically generated elements in the “Definition” tab. Among the elements created automatically during model export to Simulx is the occasion structure, which is not needed for these simulations as we are setting BTAV to zero and can be deleted.

Next, create the new elements needed for the simulation. Because we will ignore residual error, we can use the untransformed ACR20. To do this, create a new regular output for ACR20 weekly from week 0 to week 52. To simulate the treatments, represented in the model as a categorical covariate, create four covariates in the “Covariates” tab, each with DRUG corresponding to one of the four treatments to simulate. Finally, a regressor needs to be defined for Narm as it was used in the model; however, it will not impact the simulations because BTAV and residual error are ignored.

Next, go to the “Simulation” tab. To begin by just simulating a typical study, set the group size to 1 and parameters to “mlx_Typical,” which contains Monolix‐estimated population parameters but with random effects BSV and BTAV set to 0. Create four simulation groups and set each group to have one of the created covariate elements for the four treatments. After running the simulation, the “Plots” tab displays the output distribution with the response curves for the four treatment groups, which can be combined by merging the splits in the “Stratify” panel (Figure 6A).

FIGURE 6.

Results from simulations from fitted model with population parameter uncertainty. (A) Model predictions over time by treatment for a single simulated replicate, (B) distribution of the replicates of ACR20 at week 52 across all four treatment groups, and (C) model predictions over time by treatment for 200 replicates.

The efficacy of canakinumab can be quantified by defining outcomes and endpoints. In the “Outcomes & Endpoints” section, define a new outcome for ACR20 with output processing “last value per id” (i.e., week 52). Enable the group comparison and set canakinumab as the reference since the efficacy of canakinumab will be compared with treatments already on the market. Then choose direct comparison and set “difference < 0.” This is because a negative difference means $\mathrm{ACR}{20}_{\text{last}}\left(\text{abatacept}\right)-\mathrm{ACR}{20}_{\text{last}}\left(\text{canakinumab}\right)\le 0$ which is equivalent to $\mathrm{ACR}{20}_{\text{last}}\left(\text{canakinumab}\right)\ge \mathrm{ACR}{20}_{\text{last}}\left(\text{abatacept}\right)$, meaning canakinumab is more effective (and likewise for adalimumab and placebo). After running the task, the “Group Comparison” tab under “Results” shows that the comparison was successful (i.e., canakinumab was more effective) against placebo, but not against abatacept and adalimumab.

Taking into account the uncertainty in parameter estimation will allow a prediction interval for canakinumab's true effect to be calculated. To do this, change from a single simulation to 200 replicates in the “Simulation” tab and also change the parameters from “mlx_Typical” to “mlx_TypicalUncertain.” This element contains the estimated variance–covariance matrix of the population parameters estimated by Monolix with SDs of the random effects set to 0. Thus, each replicate study will have different population parameter values, such that the uncertainty of population parameters is propagated to the predictions. The definition of outcomes and endpoints remains unchanged, so we can save and rerun the project. The output distribution plot (Figure 6C) shows the response curve with the percentiles of the simulated data calculated from all replicates.

In the “Endpoint distribution“ tab of the plots, the distribution of the replicates of ACR20 at week 52 is shown across all four treatment groups. All treatments with active compounds are more effective than the placebo group. However, over the 200 study replicates, the ACR20 value at week 52 tends to be lower for canakinumab than for abatacept and adalimumab, but with larger differences between studies for canakinumab: ACR20 values range from 40% to 82% for canakinumab depending on the replicate, while they vary from 67% to 80% for abatacept and from 57% to 74% for adalimumab (Figure 6B). This is due to less data being available for canakinumab, leading to a larger uncertainty of the true effect of canakinumab in comparison to the other drugs.

In the “Group Comparison” tab under “Results,” a table summarizes success (i.e., higher ACR20 for canakinumab) across all 200 replicates for each treatment. We can conclude that, at week 52, there is a 9% chance that canakinumab outperforms abatacept, a 39% chance that canakinumab outperforms adalimumab, and a 100% chance that canakinumab outperforms placebo. This finding supported the decision to discontinue clinical development of canakinumab for RA, as reported in Demin et al. [8].

3.4. Summary

This case study shows how MBMA can support critical decisions during the drug development process using Monolix. In this case study, a similar modeling approach was used as Demin et al. [8] to show that canakinumab was unlikely to outperform 2 marketed drugs. Consequently, clinical development of canakinumab was halted, resulting in substantial cost savings.

4. Discussion

This tutorial demonstrates that MonolixSuite is an integrated, straightforward, and robust tool for performing longitudinal MBMA with practical applications of Monolix handling various types of summary‐level endpoint data. Unlike previous tutorials, the current analyses were performed using the SAEM algorithm [37] in Monolix. The SAEM algorithm avoids convergence to local minima, is robust in situations with sparse data, and is useful for fitting complex mechanistic models, including those involving multiple differential equations [38]. However, care must be taken with the algorithm settings when estimating parameters without variability, as shown in this tutorial when random effects are estimated as separate terms. The automatically generated and interactive plots allow quick model diagnosis, and the convergence assessment module makes it simple to verify model stability.

To summarize how to implement an MBMA in MonolixSuite, here is a checklist to follow:

1. Set up the dataset. Create a transformed observation, applying weighting and any other transformation such that the residual error model is constant. Tag study as ID and arm as OCCASION, as well as the transformed observations as OBSERVATION, the weighting column as REGRESSOR and any additional covariates.

2. Set up the model. If BTAV with weighting is needed, or covariate varying between arms will be included on parameters with only BSV, split up parameters into the fixed effect and the random effects, and transform them as appropriate in the structural model. Also transform the prediction in the structural model to match the transformed observation in the dataset. Finally, set the parameter definition appropriately in the statistical model. For separate random‐effect terms, use a normal distribution and fix the mean value to 0.

3. Estimate parameters. Assess the estimation stability using the convergence assessment and consider if the SAEM settings may need to be adapted for parameters without variability (e.g., if some parameters were split into multiple terms).

4. Evaluate model fit. Use automatically generated plots for observations vs. predictions, the residuals, random effects, etc. for model diagnostics. Use provided R code to back transform the individual fits and visual predictive check to the original scale of the data.

5. Evaluate possible covariates. Use automatically generated plots and statistical tests to evaluate potential covariates.

6. Run simulations. Export the Monolix project to Simulx to conduct simulations (e.g., evaluate clinical trial scenarios).

The BTAV (random effects) represents variability among treatment arms despite randomization, that is, mean values per arm differ because different individuals are recruited in each arm. Given the law of large numbers, this variability is expected to be lower when the number of individuals per treatment arm is high. Thus, the BTAV should be weighted when using summary‐level data. For the BTAV, weighting is usually done using the number of individuals or sometimes ignored for simplicity. Similarly, the residual error must also be weighted. The empirical SE extracted from the included studies is an estimation of this remaining unexplained variability, and thus it is preferred for weighting the residual error (shown in Case Study 1). However, it is not always available, necessitating using the number of individuals per arm instead (shown in Case Study 2), which assumes a constant SD σ across all studies and arms of the response and thus a standard error $\mathrm{SE}=\sigma /\sqrt{N_{\mathrm{arm}}}$. Whether empirical SE or N _arm is used, time‐varying weights (e.g., due to dropouts) can be accounted for in the analysis, since regressor variables in Monolix can vary by time.

Our case studies have focused on longitudinal MBMA, where time is the independent variable. Using an exposure‐response model in an MBMA is also possible, using a measure of exposure (such as dose or average concentration) as a regressor and dummy time values. Furthermore, a similar workflow can be employed for other data types such as count, categorical (more than two levels), and time to event. Overall, the efficient SAEM algorithm for parameter estimation as well as the built‐in plots allow users to efficiently analyze, diagnose, and improve the model. The inter‐operability between Monolix and Simulx also facilitates the transfer of the Monolix project to Simulx to easily define and run simulations.

Funding

The authors have nothing to report.

Conflicts of Interest

C.B., A.T., Y.K.L., H.B., A.d.l.P., and G.C. were employees of Simulations Plus Inc., the company developing and commercializing the MonolixSuite software, while this work was conducted.

Supporting information

Data S1: psp470158‐sup‐0001‐Supinfo1.docx.

Data S2: psp470158‐sup‐0002‐Supinfo2.zip.

Bracis C., Taneja A., Lyauk Y. K., Barcomb H., de la Peña A., and Cellière G., “Model‐Based Meta‐Analysis With MonolixSuite: A Tutorial for Longitudinal Categorical and Continuous Data,” CPT: Pharmacometrics & Systems Pharmacology 15, no. 2 (2026): e70158, 10.1002/psp4.70158.