Toward Evidence Synthesis of Adverse Events in Imbalanced Time‐to‐Event Data

ABSTRACT

Background

The routine approach in evidence synthesis of adverse events is to estimate the odds ratio or risk ratio of each individual study and then synthesize the study‐specific effects for a pooled average estimate, while seldom consider the potential imbalanced duration of exposures of study arms. This article aims to investigate the potential impact of imbalanced exposure time on harm effects.

Methods

We simulated individual participant time‐to‐event data based on Cox proportional hazard model, with Weibull function to reshape the distribution of the hazards. We further collapsed the data into aggregated one and fitting both hierarchical Binomial regression model and hierarchical Poisson regression model to estimate the pooled RR and incidence rate ratio (IRR). The percentage bias, mean squared error, and coverage probability were examined.

Results

Our results suggested that imbalanced exposure time between study arms can have substantial impact on the estimation of harm effects in evidence synthesis, especially when the extent of the imbalance exceeds 20%. Estimating an IRR to address the imbalanced exposure time only made sense for non‐recurrent events when the between‐study heterogeneity is small or moderate. A case study by 22 ongoing trials verified the potential biased estimation when exposure time was imbalanced between study arms.

Conclusions

It is inappropriate to ignoring exposure time when there is a large difference (> 20%) between study arms; while the IRR could be used in some cases, collecting individual participant data for evidence synthesis of adverse events for time‐to‐event data should be the primary consideration.

1. Introduction

While medications are intended to prevent or treat health conditions, they may cause expected or unexpected adverse effects. Safety assessment of medications is an indispensable process to identify the risk of adverse events to prevent patients from harms. For a specific adverse outcome, observed events are often rare in a single study owing to small sample size, short follow‐up time, and/or low event rate [1, 2]. Evidence synthesis (e.g., meta‐analysis) serves as a valid approach to mitigate issues individual studies face with regard to sparse data as it can considerably increase both the total sample size and available adverse event data [3].

In evidence synthesis practice, for a binary safety assessment, the routine approach is to estimate the harmful effects of each individual study and then synthesize the study‐specific effects for a pooled average estimate, measured by risk ratio (RR), odds ratio (OR), or risk difference (RD) [4]. An important assumption behind this approach is that the duration of exposures is similar within a study (i.e., arm level), as most trials are designed with equal exposure duration for study arms. This assumption overlooks the potential danger of misleading overall harm effects estimates that may arise from unequal or differential durations of exposure. This is because not all trials had equal duration, even when the designed duration was equal, as the actual duration achieved during the trial might be different (e.g., due to different discontinuation [5]). As highlighted in the Clinical Safety Review Guideline by the US Food and Drug Administration, in the assessment of adverse events, the duration of exposure should be examined and taken into account [6].

There is empirical evidence of variation in durations of exposure amongst study arms of clinical trials. According to a recent survey on the duration of exposure of oncology clinical trials, in about 47.8% of the cases, the duration of exposure in the intervention group was longer than the control group (with an active treatment), with a median difference of 1.1 months (6.0 vs. 4.9 months) [7]. This indicates that the assumptions of similar exposure duration in study arms often do not hold. One valid approach to account for the varying duration is to synthesize the study‐specific hazard ratio (HR) for an average hazard ratio or the Kaplan–Meier curves for an overall time‐to‐event relationship [8, 9, 10]. However, this is not often possible in real‐life practice because safety profile in clinical trials is mostly reported as per the total event counts, cases, or a crude rate, and the individual participant data is commonly not accessible [11].

An alternative method based on a standard meta‐analytic procedure is to account for the total exposure person‐time is to estimate the incidence rate ratio (IRR) or incidence rate difference (IRD) for the harms identified in each study [12]. A more sophisticated procedure is to model time in a hierarchical Poisson regression (see appendix)—while accounts for the issue of imbalanced duration, it is also conducive to mitigating the issue of zero events, especially for studies with zero events in both arms [13, 14, 15]. The concern with these two variant methods is that they assume a constant hazard over time [16]. We realized that in the case of the absence of individual participant data or pointwise data (e.g., the Kaplan–Meier curve), these methods may be the most viable solution for tackling the issue of imbalanced duration of exposure.

This article presents a simulation study aiming to address two questions in evidence synthesis of adverse events: (1) What is the impact of not taking exposure duration into account when there are imbalances in study arms? (2) What would be the effects of estimating the IRR instead when the time of exposure is imbalanced and only aggregated data is available? We focus on the hierarchical models for both of these two questions due to its advantage in dealing with zero‐events. Our article is structured as follows. We first illustrate the time information related to adverse events in trials, and then present results of simulated aggregated trial data by collapsing generated individual participant data under various scenarios. The data were further fitted using the hierarchical Poisson model for synthesis, and the model's performance was investigated. The article ends with a discussion and several practical recommendations.

2. Methods

2.1. Time Information for Adverse Events

We refer to adverse events as any untoward medical occurrence in a patient or subject in clinical practice [17]. For an individual who experiences a specific adverse event, there are two sources of time information involved for the event, namely, the time‐to‐first‐event and the duration of exposure.

• −Time‐to‐first‐event is the time period from the start of the intervention to the occurrence of the first adverse event, which is only determined by the occurrence of the first event.
• −Duration of exposure, sometimes called treatment duration, differs from time‐to‐first‐event. It refers to the total exposure time of the treatment throughout the trial, which is often determined by the occurrence of efficacy endpoints or the designed period, and in very few cases determined by the occurrence of the adverse event that causes discontinuation. Duration of exposure may be measured by the period of the first dose to the last dose, with or without a short “washout” period (half‐life of the drug, e.g., 30 days after the last dose).

Therefore, the duration of exposure may contain recurrent events of the same adverse endpoint and would generally be longer than the time‐to‐first‐event. In addition, one should distinguish the concept of the duration of exposure from follow‐up duration; the latter may involve a follow‐up period while receiving treatment and may include after the termination of the treatment course. Figure 1 presents the differences in the above time information of individuals.

FIGURE 1.

Time information for adverse events in clinical trials.

For a group of participants, the mean or median time‐to‐first‐event and duration of exposure could be estimated at the population level. Since the hazard of the adverse events may differ between treatment and control group, the mean or median time‐to‐first‐event could differ even when the designed treatment duration is equal. For the duration of exposure, the mean or median value may also differ in the two groups due to different discontinuation or hazards of survival. For example, a higher discontinuation rate or a shorter survival in the control group commonly leads to fewer patients receiving the following interventions, resulting in a smaller mean or median duration of exposure (Figure 1). This further leads to imbalanced time related to adverse events of the two groups.

2.2. Data Generation Mechanisms

We started with the Cox proportional hazard model [18], given by $h(t|Treat)=h_0(t)\exp (\beta \ast \mathrm{Treat})$, where $h(t|\mathrm{Treat})$ denotes the hazard in time $t$, $h_0(t)$ denote the baseline hazard, $\mathrm{Treat}$ is the group indicator (i.e., 1 for treatment and 0 for control), and $\beta$ is the regression coefficient of $\mathrm{Treat}$, say, log HR. The survival function was then established as $S(t|\mathrm{Treat})=\exp (-H_0(t)\exp (\beta \ast \mathrm{Treat})$, with $H_0(t)$ the cumulative baseline hazard, defined as $\underset{0}{\overset{t}{\int }}{h}_0(u)\mathrm{d}u$. The distribution function of the Cox model, $F(t|\mathrm{Treat})$, is derived by $1-S(t|\mathrm{Treat})=1-\exp (-H_0(t)\exp (\beta \ast \mathrm{Treat}))$. To simulate the survival time $T$, a random variable $Y$ of $F(t|\mathrm{Treat})$ was introduced, with $U=F(Y)$, which follows a uniform distribution from 0 to 1 [19]. Thus, $S(t|\mathrm{Treat})=1-U$, and the survival time $T$ can be computed by $T=H_0^{-1}[-\log (U)\exp (-\beta \ast Treat)]$. By introducing the inverse of an appropriate cumulative baseline hazard function ($H_0^{-1}$), the survival (or event) time could be generated. To fit a flexible distribution of the hazards instead of a constant hazard, the Weibull distribution was utilized based on a scale parameter λ and a shape parameter v; then, $H_0(t)=\lambda t^v$, and $T={(-\frac{\log (U)}{\lambda exp(\log (\mathrm{HR})\ast \mathrm{Treat})})}^{1/v}$ [20, 21, 22, 23].

The above procedure allowed us to simulate the time‐to‐first‐event of adverse events. This is appropriate when the event rate is low, and recurrent events are uncommon. While for some adverse events, recurrent events may occur, as shown in Figure 1. To simulate the scenario with recurrent events, we further introduced a vector for the number of time intervals ($c$) for each individual, and for simplicity, we assumed that the first event and recurrent events were independent based on the Andersen–Gill model [24]. By expanding each individual into $c$ intervals, and generating the $T$ for each interval across the individuals, the recurrent event could then be established by the simulation.

2.3. Simulation Settings

The setting of the parameters of the simulation was based on the distribution of the real‐life trial‐level dataset [25]. We first estimated the group sample size ratio of each study, the group size of the control, and the number of studies within each meta‐analysis. To link the sample size information of intervention and control, the parameter of sample size ratio was used together with the parameter of the sample size in the control group to generate the sample size of the intervention group. Based on previous studies [14, 26], the group sample size of the control was fitted using a log‐normal distribution (log $n_2$ ∼ N (4.826406, 1.135147)). The sample size ratio was taken from the percentile values of 5 and 95 of a uniform distribution (ratio ∼ U (0.89, 2.12)). The parameter for the number of studies of each meta‐analysis was taken from the percentile values of 5 and 95 from the empirical dataset, k ∼ U (5, 34).

After generating the aggregated trial data, we expanded the data by the group size for each trial to generate the individual participant data, and then generated the survival time for each individual per study via the aforementioned formula and distribution. We took a fixed true value of log HR, 0.25, with a variance of 0.1, 0.3, and 1 that represented low, moderate, and high between‐study heterogeneity, respectively, based on the median (interquartile range) value of our real‐life dataset. The scale and shape parameters, λ and v, were taken by fitting the real‐life dataset to the Weibull distribution, which were 0.86 and 1.07 separately. For the number of time intervals of each subject, we set it as 5 as we expected the recurrent events to hardly exceed 5 for each patient [27].

To simulate imbalanced exposure time for the treatment and control groups, we assigned the time (t ₀) for the control group by setting two fixed percentiles of censoring of the survival time, at P ₁₀ and P ₂₅, since adverse events tend to have a relatively low incidence rate. An increment ($\mathrm{\Delta }$) of the time in treatment group (t ₁) over control group was used to generate the time of treatment group, where $t_1=t_0\ast (1+\mathrm{\Delta })$, and $\mathrm{\Delta }$ took a random value of $U(0,1)$, which allows studies of a meta‐analysis to have different amounts of imbalance on exposure time. To further identify the potential cutoff of $\mathrm{\Delta }$ and study the extent of imbalance that could raise a substantial impact on the effects, we also took the following fixed values for $\mathrm{\Delta }$: 5%, 10%, 20%, 30%, 40%, 50%, and 100%. Figure S1 presents the simulation settings.

After modeling the individual participant data in the hierarchical Cox proportion hazard regression, we collapsed the data to aggregated one, in order to investigate the performance of estimation of an IRR and RR based on the method described below. Based on the above specifications, there were 12 scenarios for balanced time and 96 scenarios for imbalanced time, with a total of 108 scenarios, and for each scenario, 5000 meta‐analyses were generated.

2.4. The Hierarchical Regression Models for Evidence Synthesis

The hierarchical regression model is based on a one‐stage framework where estimates of study‐specific effects are not needed [28]. Instead, the included studies are considered as a special multicenter trial with each study as a center, and the overall effect can be directly estimated by treating arm as level 1 and study as level 2. For binary outcomes, both Binomial distribution and Poisson distribution can be applied in the model. The Binomial distribution allows the estimation of RR (with link function as log or cloglog), OR (with link function as logit) or RD (with link function as identity), without addressing the time information. On the other hand, the Poisson distribution modeling the time information in the analysis; thus, it allows the estimation of IRR (with link function as log) or IRD (with link function as identity).

There are three commonly used hierarchical models in evidence synthesis: the fixed‐effect model (fixed treatment effect with random baseline risks), the standard random‐effects model (random treatment effects with fixed baseline risk), and the “hyper” random‐effects model (random treatment effects with random baseline risks) [29]. In the current study, we focused on the standard random‐effects model because simulation studies have demonstrated that it outperforms the “hyper” random‐effects model for heterogeneous studies, and with fewer convergency issues [14, 26, 29]. In addition, it also produced almost unbiased estimation for homogenous studies [30].

2.5. The Standard Random‐Effects Hierarchical Binomial Model

Let's assume $i$ the indicator of study, $j$ the indicator of intervention status ($j$ = 0 for control and $j$ = 1 for treatment arm), $\pi$ the event risk, and $\gamma$ the indicator of baseline risk with the value within 0 to 1 [31]. The standard random‐effects hierarchical Binomial model is then:

$$\log ({\pi }_{ij})=\gamma +j{\theta }_i,$$

where ${\theta }_i$ ∼ ($\theta$, ${\tau }^2$) indicates random‐treatment effects with a mean value of $\theta$ and variance of ${\tau }^2$; the latter (${\tau }^2$) is the between‐study variance. The baseline risk $\gamma$ is set as fixed. The coefficient $\theta$ of the model is the log RR. When the logit link function was applied, the model could estimate log OR.

2.6. The Standard Random‐Effects Hierarchical Poisson Model

We further assume $nT$ represents the total person‐time, $e$ is the event count of the specific outcome, and $u$ is the expected event count [13]. The standard random‐effects hierarchical Poisson model is then:

$$\log \left(u_{ij}\right)=\gamma +j{\theta }_i+\log \left(n{T}_{ij}\right);$$

$$u_{ij}\sim E\left(e_{ij}\right).$$

Again, ${\theta }_i$ ∼ ($\theta$, ${\tau }^2$) indicates random‐treatment effects with a mean value of $\theta$ and variance of ${\tau }^2$. The latter is the between‐study variance. While $\gamma$ is the baseline rate and set as fixed. The coefficient $\theta$ here now is the log IRR.

2.7. Type of Data Used in the Models

The two hierarchical models were fitted in the aggregated data that collapsed by the individual participants data. As a comparison, we also fitted the hierarchical Cox proportional model for a pooled HR based on the generated individual participant data. For the total person‐time in the two models, the mean value of the time (first‐to‐event‐time and duration of exposure) was utilized. We did not consider median time because the mechanism to generate adverse events was based on large censoring, where the median time would always be the time of censoring. For the recurrent events, we modeled both all events and time‐to‐first‐event (i.e., RR for all events, RR for the first event, IRR for time‐to‐all‐events, IRR for time‐to‐first‐event) in the two hierarchical models, and the robust variance estimator was employed as suggested in Ref [32].

2.8. Statistical Performance

Our primary outcome was the statistical performance of the above two methods, as compared to the hierarchical Cox proportional model. The following metrics were used to measure the statistical performance: percentage bias (calculated as $\exp [\frac{\widehat{\theta }-\theta }{\theta }]$), mean squared error (MSE, calculated as ${(\widehat{\theta }-\theta )}^2+Va{r}_{\widehat{\theta }}$), and interval coverage (95%, the probability of the interval covers the $\theta$) [33]. The percentage bias reflects the point estimates, and the interval coverage reflects the variance estimates, while the MSE reflects both [34]. To avoid the contamination of potential extreme values (due to computational issues) on the mean value of these metrics, we dropped those meta‐analyses with extreme values on MSE, and defined an MSE larger than 50 or smaller than 0.001 as an extreme value.

The data generation and data analyses were conducted in the Stata SE 16.0 (Stata Corp LCC, College Station, TX), and the output visualization in Excel 2021 (Microsoft, Washington). The analyses of the current study did not involve statistical tests, and thus no significant level was defined. The code is presented in the supplementary file.

3. Results

3.1. Imbalanced Time for Non‐Recurrent Events

Figures 2 and 3 present the results of the simulation for non‐recurrent events (${\tau }^2$ = 0.1), with the percentile of censoring at P ₁₀ and P ₂₅. When exposure time was imbalanced in the study arms, the hierarchical Poisson model of IRR achieved both good point estimates and variance estimates, with a small bias, MSE and good coverage across all the situations. There was a slight increase in the bias and a decrease in the coverage as the extent of the imbalance in exposure time became larger. For the hierarchical Binomial model of RR, there was a large bias of the point estimates, which increased rapidly as the extent of imbalance became larger. The coverage deteriorated rapidly to lower than 80% when the extent of imbalance on exposure time exceeded 20% of the study arms in all of the scenarios. The hierarchical Cox proportion hazard model based on individual participant data had a larger bias than the hierarchical Poisson model, while almost the same coverage due to a larger variance. The performance was similar when the between‐study heterogeneity increased to moderate (${\tau }^2$ = 0.3; see Figures S2 and S3).

FIGURE 2.

Simulation results for non‐recurrent events with the percentile of censoring at P ₁₀ (${\tau }^2$ = 0.1). Extreme values were removed due to computation issues. MSE, mean squared error.

FIGURE 3.

Simulation results for non‐recurrent events with the percentile of censoring at P ₂₅ (${\tau }^2$ = 0.1). Extreme values were removed due to computation issues. MSE, mean squared error.

However, we noticed that when the exposure time was balanced in the study arms, the hierarchical Poisson model of IRR showed worse performance compared to the hierarchical Binomial model of RR and the hierarchical Cox proportion hazard model, regardless of the amount of between‐study heterogeneity (Figures 2, 3, S2, and S3). More importantly, as the between‐study heterogeneity increased to high (${\tau }^2$ = 1), the hierarchical Poisson model showed poor performance with large bias and poor coverage for all scenarios of imbalanced exposure time (Figures S4 and S5).

3.2. Imbalanced Time for Recurrent Events

Figure 4 presents the results of the simulation for recurrent events under homogenous studies (${\tau }^2$ = 0.1), with the percentile of censoring at P ₁₀. In contrast to the scenarios of non‐recurrent events, the hierarchical Poisson model for an estimated IRR performed poorly for both accounting for all events and time‐to‐first‐event, with coverage lower than 50% even when the extent of imbalance of the exposure time is small (e.g., $\mathrm{\Delta }$ = 5%). For the hierarchical Binomial model, again, when modeling the RR for all events, the bias was large and increased rapidly, especially when the extent of imbalance on exposure time exceeded 30% of the study arms, leading to a coverage lower than 11%. When modeling the RR for the first event, the hierarchical Binomial model showed good coverage when the extent of imbalance on exposure time did not exceed 50% of the study arms, and the bias was less than 30% (–0.1% to 30.6%) under such cases. The hierarchical Cox proportion hazard model based on individual participant data again performs well in almost all of the cases, although the coverage dropped to under 80% when the extent of imbalance on exposure time exceeds 50%.

FIGURE 4.

Simulation results for recurrent events with the percentile of censoring at P ₁₀ (${\tau }^2$ = 0.1). Extreme values were removed due to computation issues. MSE, mean squared error.

Due to the poor performance of the hierarchical Poisson model under the scenarios of low between‐study heterogeneity with the percentile of censoring at P ₁₀, we no longer reported the results for the remaining scenarios. For the hierarchical Binomial model, as the heterogeneity increased, the performance for modeling RR for the first event showed a mild deterioration, while still keeping good coverage and tolerable bias (less than 30%) when the extent of imbalance on exposure time did not exceed 50% (Figures S6 and S7).

4. Real‐Life Example

Twenty‐two ongoing Phase I or II open‐label randomized controlled trials from the drug clinical trial center of Anhui Provincial Hospital were used for the real‐life example. As required by the trial sponsors, adverse events information was allowed to use in current study for academic purpose analysis without disclose the intervention and participants’ information. Here we took the outcome of serious adverse events (grade III or above) recorded in the data repository of these trials as our example. For the 22 trials, 5 were Phase I and 17 were Phase II, with the sample size ranged from 11 to 305 and a median sample size of 43.5 (interquartile range [IQR]: 32 to 99). The incidence of serious adverse events ranged from 0.0% to 12.4%, with a median incidence of 4.0% (IQR: 0.0% to 8.7%). For the research purpose, we treat the real‐life dataset as a pseudo meta‐analysis with 22 trials. This is reasonable as we focused on serious adverse events with the expectation that all interventions had the risk of leading to serious adverse events.

The median time‐to‐first‐event (day) of the 22 trials was imbalanced in 16 of the trials, with the ratio ranged from 0.3 to 10.8. The median time‐to‐first‐event was 234 vs. 169 days (exceeded 38%) for the comparative groups when treating the 22 trials as one mega‐trial. The hierarchical Cox proportional hazard model showed that the HR for serious adverse events was 2.07 (95% CI: 1.32 to 3.22, p < 0.01) for intervention against control group. When not considering the imbalanced time by modeling the hierarchical Binomial model, the RR for serious adverse events was 1.19 (95% CI: 0.67 to 2.11, p = 0.56), and the IRR by hierarchical Poisson model was 3.41 (95% CI: 1.92 to 6.05, p < 0.01). In this case, it is clear that when not considering time information or when not appropriately modeling time, both would lead to inaccurate results even conclusions.

5. Discussion

Our simulation study confirmed that imbalanced exposure time between study arms can have substantial impact on the estimation of harm effects in evidence synthesis, especially when the extent of the imbalance exceeds 20%. We also found that estimating an IRR to address the imbalanced exposure time only made sense for non‐recurrent events when the between‐study heterogeneity is small or moderate. For recurrent events, the estimation of an IRR performed poorly in all simulated cases, regardless of modeling for time‐to‐all‐events or for time‐to‐first‐event. In addition, for the case that the exposure time was balanced in study arms, estimating an IRR (to incorporate exposure duration) showed worse performance than estimating an RR directly.

We found that for recurrent events, when the exposure time was imbalanced between the study arms, estimating an RR for the first event directly (not addressing the imbalanced time) could achieve good coverage when the difference in the duration of exposure of study arms do not exceed 50%, while this was not the case for estimating RR for all events. This is likely due to the fact that the difference in the exposure time does not suggest a difference in the time‐to‐first‐event; the latter may still be similar for the study arms even if the exposure time differs. However, from such a perspective, it is hard to explain the extremely poor performance in estimating IRR for the first event. A more rational explanation could be that recurrent events had a more flexible distribution of the hazard than the non‐recurrent events, which emphasizes the major limitation of IRR, which is that it assumes a constant hazard over time. The RR does not have such an assumption, although it overestimated the point estimates, and the larger variance (leading to a wider confidence interval) offset the overestimation and thus resulted in better coverage.

In our simulation, the application of individual participant data for a hazard ratio could always achieve an accurate and precise estimation of the harm effects. It should be noted that, because the censoring of the generated time‐to‐event data would result in censoring bias, these metrics would be adversely influenced, generally to be deteriorated, while it always performs better than the two hierarchical models. The results suggested that, whenever possible, collecting individual participant data for evidence synthesis of adverse events of time‐to‐event data should be the primary consideration.

Researchers should notice the differences on the meaning of the RR, IRR, and HR [35, 36]. RR is a measure of the relative difference of the risk in two comparative groups, which refers to the cumulative incidence of a time period; IRR is a measure of the relative difference of the incidence rate for a defined period of time, which refers to the “average” incidence of a time period. Both are constant for the given time period. While HR reflects relative difference of the instantaneous incidence rate of two groups of a given time point of the period that could differ for different time points, it could measure how fast an event occurs in one group to another group. The differences on the magnitude and direction of the three estimates have been well‐illustrated in our simulation.

Based on the simulation, several practical recommendations could be formed for the evidence synthesis of adverse events.

• −First, before conducting evidence synthesis of adverse events, the researchers should check the duration of exposure within the study arms. It is reasonable to use RR/OR/RD as an effect estimator and synthesize the effect with conservative methods (e.g., random‐effects models) when the duration is balanced or almost balanced (i.e., difference < 20%), regardless of the outcome is non‐recurrent or recurrent events.
• −Second, when the duration of exposure differs substantially between the study arms (i.e., difference > 20%), the primary consideration should be to obtain the individual participant data, or the Kaplan–Meier curve, or the HR.
• −Third, when the duration of exposure differs substantially between the study arms (> 20%) and the HR/Kaplan–Meier curve could not be accessed, it is not recommended to conduct quantitative evidence synthesis based on aggregated data.

To our knowledge, this is the first study investigating the issue of evidence synthesis of adverse events with time‐to‐event data when exposure‐time is imbalanced between study arms. We have clearly illustrated the impact of time information in adverse outcomes, and conducted a large simulation based on empirical dataset, which covers numerous real‐life scenarios for the performance of potential methods for addressing the imbalanced time. Our study highlights the importance of addressing imbalanced exposure for the assessment of harm effects. The findings of our study will have important implications for the evidence synthesis practice and informed decision‐making.

One major limitation should be highlighted of our study. Namely, the simulation of the recurrent event did not consider the correlations between the recurrent events and the previous event, which may “distort” the simulated data from the real‐life circumstance, which may undermine the external validity of the findings. Another limitation is the simulation failed to consider the severity of the AEs, which serve as an important issue in evidence synthesis of adverse events. Even so, we believe such limitations are insufficient to alter the findings of the current study.

In conclusion, based on our simulation, there is a substantial impact of imbalanced exposure time of study arms on the estimation of harm effects in evidence synthesis, especially when the extent of the imbalance exceeds 20%. Estimating an IRR to address the imbalanced exposure time only make sense for outcomes of non‐recurrent events when the between‐study heterogeneity is small or moderate. Whenever possible, collecting individual participant data for evidence synthesis of adverse events for time‐to‐event data should be the primary consideration.

Funding

The current study was supported by the National Natural Science Foundation of China (72204003, 72574229), Research Funds of Center for Big Data and Population Health of IHM (JKS2022021), and Shanghai Shenkang Hospital Development Center (SHDC12025626). Luis Furuya‐Kanamori was supported by an Australian National Health and Medical Research Council Fellowship (APP1158469). The funding bodies had no role in any process of the study (i.e., study design, analysis, interpretation of data, writing of the report, and decision to submit the article for publication).

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Figure S1: Simulation settings.

Figure S2: Simulation results for non‐recurrent events with the percentile of censoring at P ₁₀ (${\tau }^2$ = 0.3).

Figure S3: Simulation results for non‐recurrent events with the percentile of censoring at P ₂₅ (${\tau }^2$ = 0.3).

Figure S4: Simulation results for non‐recurrent events with the percentile of censoring at P ₁₀ (${\tau }^2$ = 1).

Figure S5: Simulation results for non‐recurrent events with the percentile of censoring at P ₂₅ (${\tau }^2$ = 1).

Figure S6: Simulation results for recurrent events with the percentile of censoring at P ₁₀ (${\tau }^2$ = 0.3).

Figure S7: Simulation results for recurrent events with the percentile of censoring at P ₁₀ (${\tau }^2$ = 1).

Data Availability Statement

The code used in the data generating and analysis is supplied in the supplementary file. The data used in real‐life example could be obtained upon request from the corresponding author.