Individual participant data (IPD) meta-analysis: An introduction – Narrative review

Abstract

Systematic reviews and meta-analyses (MA) are accepted modalities for evidence synthesis in evidence-based medicine. However, as MA uses aggregate data that includes averaging patient characteristics and pooled effect estimates, it has limitations when considering personalised medicine. In contrast, individual participant data meta-analysis (IPD-MA) includes and segregates individual patient data to study new outcomes, identify outcome predictors, and analyse multiple covariate effects on treatments. IPD-MA requires data from multiple investigators, review board approvals, clear communication with collaborators, and statistical recalculation of cumulative data. IPD-MA can be performed as a single-stage process where data from all included studies is pooled and reanalysed or as a two-stage process where additionally the data from individual studies is re-analysed before being pooled. This review aims to orient clinicians about IPD-MA, including the process of performing it, comparing it with other types of meta-analyses and considering the potential barriers in conducting it.

INTRODUCTION

In the era of evidence-based medicine, systematic review and meta-analysis (SRMA) have emerged as well-established research methodological tools for synthesising the available evidence. Since the term ‘meta-analysis’ was introduced by Gene Glass in 1976, medical literature adopting SRMA has witnessed exponential growth.[1] Although the SRMA technique is well established, it provides summary results, which limits its use in personalised medicine. Moreover, using averaged patient characteristics and pooled effect estimates reduces power and limits data analysis.[2,3,4]

An improved understanding of the limitations of summary/aggregate data presented in an SRMA has led to the inception of the individual participant data meta-analysis (IPD-MA) approach. Moreover, the current paradigm shift towards personalised medicine has refined evidence synthesis techniques. Over the last two decades, the IPD-MA technique has mostly been utilised to assess treatment or intervention effects and time-to-event analyses. However, the scope of IPD-MA is rapidly expanding to include the appraisal of studies involving the performance of risk prediction scores, the accuracy of diagnostic tests, and prognostic factors.[5] The IPD-MA approach is preferred when there is a need to explore treatment effects in participant sub-groups, explore covariates (a variable which can influence the outcome), and improve consistent reporting regarding outcomes or participant covariates. Additionally, the IPD-MA approach allows the study of newer outcomes, identifies predictors of outcomes, and analyses the simultaneous effects of multiple covariates on treatment/intervention effects.[2,5]

This narrative review aims to share a basic understanding of the aims, methodology, advantages, limitations, and future of IPD-MA, focusing on building robust evidence on any clinical question.

Definition of IPD-MA

IPD-MA is defined as the analysis of centrally collected data on participants sought from both published or unpublished cohorts.[6] The data includes the demographic characteristics of participants, exposure under investigation, and events and follow-up information. IPD-MA is a collaborative project with large-scale data to improve the quality and quantity of data and generate robust results. IPD-MA requires significant logistical effort to gather IPD from various investigators, including published and unpublished cohorts. This is in contrast to the aggregate data meta-analysis (ADMA), where, commonly, the data is extracted from published journal articles.[6] The principal investigators acquire the review board approvals for the data request and analysis and re-publication of the cumulative data, which undergoes statistical re-calculation. This requires clear and close communication with all the collaborators.[6]

Proposed advantages of the IPD-MA over the aggregate data MA approach

The IPD-MA approach has numerous proposed advantages over the conventional aggregate data MA technique[2,4,7,8,9,10,11] [Table 1]. This is exemplified in the study by the childhood acute lymphoblastic leukaemia collaborative group, where the age for inclusion was standardised as ≤21 years to resolve the differences in the age cut-offs used in trials across the USA (up to 21 years) and the UK (≤14 years).[11] Furthermore, refinement of inclusion/exclusion criteria of participants and standardisation of participant characteristics/exposures across trials or definitions used across the trials to report outcomes are possible with this approach.[4,7,10] The multi-disciplinary collaborators from the IPD-MA team can provide a comprehensive review of the implications of their results for clinical practice.[2,9] The results from IPD-MA may be used to formulate treatment guidelines or investigation pathways that are specific and more suitable for certain patient groups.[10] Furthermore, results from IPD-MA projects have significantly impacted the use of chemoradiation and adjuvant therapeutic agents in oncology. As the discussion regarding the role of the IPD-MA in cancer is beyond the scope of this article, readers are referred to an excellent review of the same by Clarke M et al.[12] As an illustrative example of the advantages of IPD-MA over conventional systematic review (SR), in subfertility, women undergo multiple in vitro fertilisation treatment cycles. These often pool the results of these multiple observations per woman in a conventional systematic review. This pooling of results could lead to the ‘unit of analysis’ error, and the estimated treatment effect following pooling is usually per treatment cycle. IPD-MA offers more valid results by providing treatment estimates per woman and avoids the ‘unit of analysis’ error. In conventional SRs, if the dichotomous manner (e.g. pain score above or below a cut-off score above or below a cut-off), IPD-MA allows the conversion of these outcomes into continuous variables and adjustment of the results.[13,14]

Table 1.

Proposed advantages of IPD-MA

• Allows data from published and unpublished trials to be included in the analysis. • Data from cohort or case-control studies (if appropriate) and newer trials or studies can be included. • Allows an additional data check for previously published studies by the IPD-MA authors. • Missing or refinement of inclusion/exclusion criteria can be exemplified by harmonising definitions of clinical endpoints. • Various techniques to handle missing or unreported data can be utilised. • Allows the reduction of ecological bias and poor reporting. • Allows adjustment for baseline differences in the study design and appropriate sub-group analyses. • IPD-MA approach pools numerous studies, allowing for the analysis of a larger sample to prove the statistical power. • IPD-MA approach allows for the assessment of additional newer or long-term outcomes and the analysis of time-to-event outcomes at different time points. • IPD-MA offers a better ability to analyse effect modifiers, detailed exploration of patient factors, and analysis of combinations of patient factors or different tests. • Heterogeneity at the participant level and study level can be better explored using the IPD-MA approach. • Can be updated, and more complete follow-up data can be obtained. • Allows minimisation of publication and reporting bias and checks for randomisation of participants from previous RCTs.

IPD-MA=individual participant data meta-analysis, RCT=randomised controlled trial

Overview of the process of the IPD-MA

The IPD-MA needs to follow specific steps [Table 2].[6,10,15] It can be done using a one-stage or two-stage approach.

Table 2.

Process of IPD-MA

	Steps	Process
A	Creating a protocol	Details of how IPD-MA will be performed based on prespecified criteria for inclusion, data collection, sub-group analysis, and interaction analysis.
B	Literature search phase	The central team will identify eligible studies through a systematic review of the published and non-published literature. If eligibility criteria cannot be identified, the principal investigators are approached personally, and the investigators are invited to participate officially. Both the eligible and in-eligible studies are documented in the management sheet. This minimises the effort of reapproaching the same authors again in the future.
C	Recruitment phase	Starts once studies are selected based on eligibility. The IPD central team invites the principal investigators formally via mail, phone, or video calls. Once recruited, the IPD central team requests the necessary documents (ethics approval and data sharing agreement) from the investigators.
D	Master code book	Developed to standardise the codes used for the collected data pooling IPD-MA.
E	Pre-harmonise data check	Once enrolled in the IPD, the individual investigators can share the complete de-identified data. The initial data check and its details are documented in the dataset tracking sheet. The central team saves two copies of the dataset. The IPD central team evaluates the data for variation in the variables, dataset structures, protocols, and publication status.
F	Mapping phase	Study variables are mapped to the IPD variables. The most efficient way to map is by phone. The data is checked for heterogeneity and integrity.
G	Harmonising Code	Created by reviewing the study variables. The IPD master code book and the investigators’ final harmonised code are developed using mapping instructions. The harmonised code is saved for all future references.
H	Post-harmonised data evaluation	After the data are harmonised, the study and central IPD teams ensure that the outcomes and captured data are harmonised appropriately. This may require multiple phone call-based follow-ups.
I	Analysis phase	Data is then analysed across the studies, producing outcomes that may or may not be similar to the published trials. This methodology allows the modification of individual-level interactions within the studies, thus providing improved power to the research and minimising the bias. There are two statistical approaches for IPD-MA, as described later.
J	Reporting	Follow the PRISMA-IPD checklist, which, in addition to the standard PRISMA checklist, includes methods used to check IPD integrity and report variations in the study- and participant-level characteristics affecting outcomes.
K	Publication bias	A funnel plot can be created to rule out publication bias, which can be detected by the presence or absence of asymmetry.

IPD-MA=individual participant data meta-analysis, IPD=individual participant data

One-stage approach

The one-stage IPD-MA approach pools the patient data from all the trials and analyses the complete data in a single step. It uses a random effect model for clustering the data within the studies. In the one-stage approach, interactions between study or participant characteristics and the intervention effect can be examined by adding extra covariates to the mixed-effect model. This approach is recommended because it uses exact likelihood specification, avoiding assumptions of normality.[16,17] The major problem with the one-stage approach is the intensive computation and convergence problems.[16,18] The one-stage approach focuses on specific models for continuous, binary, and time-to-event outcomes regarding treatment effect estimates. The one-stage approach is a flexible approach that allows the analyst to make modifications such as a fixed rather than random treatment effect, a random rather than stratified trial, and the inclusion of common adjustment terms.[19] The one-stage approach’s flexibility comes at the cost of computational challenges, particularly with non-convergence in complex models.

Two-stage approach

A two-stage meta-analysis is the preferred mode of conducting IPD-MA. The first stage of the two-stage approach is analysing the data from each study separately to obtain aggregate data. The second stage includes using a fixed effect or random effects meta-analysis model.[3,20,21] Though the two-stage IPD-MA approach is similar to the ADMA, the advantage of IPD-MA over ADMA is that study effect sizes can be estimated consistently using the same model, whereas in ADMA, the original RCTs estimate their effect sizes, and this can vary across the studies. In the two-stage approach, sub-group analysis or meta-regression analysis in the second stage can assess the intervention effects. For example, suppose N studies are selected for IPD-MA to investigate a treatment effect. In that case, the first stage includes separate analyses for N studies, resulting in N treatment effects estimates and variances. Generally, the estimates and standard errors utilise a regression model suitable for the outcome of interest. To overcome confounding errors, covariate adjustment is also performed. The second stage includes combining the treatment effects estimates of N studies and then combining them either as fixed or random across the trials. The fixed-effect model assumes that all estimates are of the same underlying treatment effect in all studies. The random model uses mostly the inverse variance method to estimate the treatment effect.

The above two approaches can result in discriminative advice; thus, it is important to declare the approach while proposing the project.[2]

Recommendation to use the one-stage or two-stage approach for IPD-MA

1. Document the approach for IPD-MA. If the best approach is unclear, use both approaches to analyse and compare the results, and any difference should be justified.

2. Predefine the model assumptions and parameter specifications regardless of the approaches.

3. One stage is preferred for rare outcomes and small-sized studies as it avoids using approximate normal sampling distributions and continuity corrections.

4. In a one-stage IPD-MA, treatment–covariate interactions should be separated into within-trial and across-trial interactions to avoid ecological bias. This is automatically avoided in a two-stage analysis when within-study interaction estimates are obtained in each trial and then synthesised.

5. In a one-stage approach, it is best to include separate adjustment terms (when included) and separate residual variance terms (for continuous outcomes) for each trial, as this makes fewer assumptions than a model with common adjustment terms and residual variances. Only when estimation issues arise might it be necessary to move away from this, for which random effects on adjustment terms may be helpful.

6. Where random effects models are used, consider methods to derive 95% CIs for the summary effect that account for full uncertainty in the estimated variances in the meta-analysis. For example, for the two-stage approach, the use of methods such as the HKSJ variance estimator and t-distribution to estimate CIs might be considered, and for the one-stage approach, the use of Kenward–Roger variance estimator and t-distribution may be helpful.

7. A standard two-stage approach does not automatically account for correlation between the parameters of the regression model estimated in the first stage; this may lead to a loss in precision and different summary estimates than the one-stage approach, which automatically accounts for such correlation. This is especially important if there are missing outcome data, for example, missing outcomes at some time points in a meta-analysis of longitudinal data. To account for correlation in a two-stage meta-analysis, a multivariate model is required in the second stage.

Debray et al. looked into the differences in the outcomes of the two approaches and concluded that both approaches produce similar results, with potential estimation challenges.[12] This is not always true; sometimes, the results from two approaches can differ. Most differences between one-stage and two-stage approaches occur because of different modelling assumptions, including the specification of the likelihood and included parameters, the choice of fixed or random effects, and the utilisation of correlation. Different assumptions, parameter specifications, and estimation methods lead to different percentage study weights in the one-stage and two-stage meta-analyses. Therefore, summary results can differ because of a change in weighting. Thus, the choice of approach depends on many factors, including the clinical question, the parameter(s) of interest, the desired specification of the model, the desired estimation method, the assumptions willing to be made, the potential for non-convergence and missing data, and the likelihood of small study sizes and rare events.

Interpretation of IPD -MA

The first step is to examine the pooled estimates from multiple studies at the individual level to draw more nuanced conclusions about the treatment effect and outcomes. The various steps towards interpreting IPD-MA are important for meaningful conclusions [Table 3].

Table 3.

Interpretation of IPD-MA

Steps	Process
Assessment of the treatment effect	It allows for understanding different factors, such as age, gender, and disease, that affect outcomes.
Exploring heterogeneity	Looking for variability in the treatment effects across the studies. This is an important step towards understanding which sub-groups of the participants are benefitting the most from the intervention, helping to suggest customised treatment strategies for sub-groups or particular subsets of the population.
Adjusting for the confounders	Adjusting for confounders is an advantage as it is not generally possible in aggregate MA.
Sensitivity analysis	Sensitivity testing aims to assess the robustness of the analysis, for example, testing the impact of missing data or the effect of different statistical models on the overall results.
Generalisation to a larger population	Increased the external validity of the findings.

IPD-MA=individual participant data meta-analysis, MA – meta-analysis

How is IPD different from other meta-analyses?

Meta-analysis is a quantitative approach to synthesise information from multiple studies to answer a research question. It is regarded as the highest form of evidence in the hierarchy of research methodology. Combining the results of various studies provides a more precise estimate of effect size through a larger sample size, higher statistical power, and reduced impact of bias in individual studies. Depending upon the type of data used for synthesis, the included studies enable the investigators to draw inferences and sometimes flexibility in answering questions not posed by the initial study researchers. There are various methodologies for conducting meta-analyses, and it is essential to understand the types and distinctions between them.

1. Pooled analysis – Pooled analysis is the aggregation of data from multiple studies with similar studies and statistical designs following a systematic review. It is important to note that pooling data is not feasible when study populations are heterogeneous. Quak et al. conducted a retrospective pooled analysis of 10 observational studies and concluded that there was no significant difference between regional and general anaesthesia for 30-day mortality after leg amputations.[21] However, not all pooled data can be considered for quantitative analysis, and SWiM (synthesis without meta-analysis) reporting structure is recommended for use when quantitative analysis cannot be performed.[22] When quantitative analysis is feasible, meta-analysis should be conducted as described in the Cochrane Handbook for systematic reviews of interventions.[23]

2. Meta-analysis – Meta-analysis is conducted to combine results of studies comparing the same experimental intervention with comparator intervention or two experimental interventions. In meta-analysis, as opposed to pooled analysis, weights are assigned to data based on the sample size. Meta-analysis utilises summary statistics (such as effect size, relative risk, or odds ratio) from individual studies to collate data. Results are aggregated in a forest plot that depicts the effect sizes of individual studies and the pooled estimate of all studies with confidence intervals. Additionally, trial sequential analysis aids in controlling type I and II errors and can elucidate the necessity or futility of further research for that research question. Singh et al. compared aprepitant with ondansetron alone or in combination for the prevention of postoperative nausea and vomiting.[24] When each drug is used independently, the meta-analysis reveals the superiority of aprepitant over ondansetron (Relative Risk: 0.45, Confidence Interval: 0.29, 0.72; P < 0.001) in preventing postoperative nausea and vomiting. Furthermore, the trial sequential analysis indicates sufficient evidence of the superiority of aprepitant over ondansetron; thus, no further trials may be necessary. In the same study, the comparison of the combination of aprepitant, dexamethasone, and ondansetron versus dexamethasone and ondansetron demonstrated the superiority of the former combination in reducing postoperative nausea and vomiting (Relative Risk: 0.38, Confidence Interval: 0.19, 0.76; P = 0.006). However, a comparison of aprepitant alone with the combination of ondansetron and dexamethasone or an individual comparison of each aprepitant, ondansetron, and dexamethasone cannot be performed in this meta-analysis. The limitation of conventional meta-analysis is that it can compare only two interventions at a time and only if they have been compared directly one against the other. These challenges can be addressed with network meta-analysis.

3. Network meta-analysis – This type of meta-analysis compares three or more groups of interventions with control or each other to allow for both direct and indirect comparisons between studies. To conduct such an analysis, transitivity should be present, which refers to the similarity between study characteristics that could affect the outcome. Additionally, coherence between the direct and indirect estimates of outcome should be established. Once these criteria are met, the interventions can be ranked based on the effect size to allow overall comparison. One such challenging clinical dilemma is the selection of an anti-emetic agent from a large armamentarium of drugs. Weibel et al. performed a network meta-analysis to compare the efficacy of 44 anti-emetic drugs alone or in combinations.[25] Aprepitant was the most effective drug, followed by ramosetron, granisetron, dexamethasone, and ondansetron, to prevent postoperative nausea and vomiting. One of the challenges of network meta-analysis is the accuracy of data, and the optimal solution for this is to revisit the raw data from the studies included. This approach for meta-analysis is achieved through IPD collection and resynthesis in IPD-MA.

4. IPD-MA – IPD-MA is a labour-intensive process of collecting raw data from published studies to re-analyse the data for more accurate and consistent results. It is particularly efficacious in studying sub-groups where combining data improves the statistical power of relationships. Sadeghirad et al. conducted a meta-analysis to investigate the predisposing factors associated with postoperative delirium in patients[26] undergoing non-cardiac surgery. They aggregated individual data from 21 studies encompassing more than 8000 patients, which enabled a more comprehensive examination of the factors contributing to delirium. It is noteworthy that when compared to a meta-analysis conducted by Abate et al.[27] (Abate et al.) on the same research question, there was a discrepancy in the role of alcohol consumption contributing to delirium. While Abate et al. found alcohol consumption to be a significant factor contributing to delirium (Odds ratio: 2.4, 95% Confidence Interval: 1.07, 5.37), Sadeghirad et al.[26] demonstrated that there was no increased risk due to alcohol. While studying the results of conventional and IPD-MA in 39 studies, Tudur Smith et al.[4] observed a 20% possibility of disagreement between the results of conventional meta-analysis and IPD-MA, including the size and direction of effect. This discrepancy is likely attributable to the fact that the IPD-MA faces challenges of a smaller number of included studies when compared to the conventional meta-analysis due to non-participation from authors. However, the data included is probably more authentic due to transparency and inclusion of authorship to the participating teams. These strengths and limitations of IPD analysis should be considered when extrapolating the results to clinical practice.

Potential barriers to conducting the IPD

1. Many specific barriers to IPD MA are related to data ‘D’ of IPD. After the literature review, the authors are contacted to submit their data for IPD-MA, and the major hurdles noted at this stage are related to inability to contact the authors, hesitancy to share data, unclear ownership, and lost data. Following the data collection, data delivery and legal procedures involves handling challenges related to getting new data consent, ethical approval, and different laws in different countries. Missing data, verification of data, and non-standardised variables are potential barriers. With improved data storage, a few of the hurdles may remain, but the data will continue to become richer and more consistent, resulting in an improvement in the power of the studies.

2. IPD is a resource-intensive project and requires judicious time management from contributing groups. The process of data collection and checking and analysing IPD is more complex than aggregate data. Ensuring the standard for IPD MA is a challenge.

3. IPD MA may have to face some biases:

   1. Selection bias: Variable outcomes of trials can result in study selection biases, which can be minimised by prospectively defining the eligibility criteria.

   2. Publication bias will be minimal in IPD as both published and unpublished studies will be recruited.

4. IPD-MA entails data sharing between collaborators from multiple countries, and depending on the origin of the data, challenges may emerge due to newer data privacy regulations, making it difficult for data to be shared across borders. Regulatory barriers such as the general data protection regulations, which are followed in the European Union and protect individual privacy, and the Health Insurance Portability and Accountability Act in the United States can make it difficult to share data from these regions.

Scope and impact of IPD-MA in future

IPD-MA is a powerful tool that allows researchers to explore variations in patient-level characteristics that influence outcomes. This approach can potentially transform clinical decision-making by providing personalised interventions based on individual patient characteristics and genetic profiles. It may also augment machine learning-driven predictive models by integrating IPD in predictive algorithms. These applications could be particularly valuable in managing pain, sepsis, and clinical scenarios, which involve a complex interplay of factors that affect outcomes. Furthermore, the combination of IPD-MA and artificial intelligence could potentially elucidate previously unknown risk factors or treatment interactions, paving the way for novel therapeutic strategies in complex medical conditions.

CONCLUSION

IPD-MA is emerging as the gold standard for evidence synthesis due to its ability to provide pooled estimates from validated data. It gives more precise and reliable estimates of treatment effects and uncovers insights that may be obscured in conventional ADMA. While IPD-MA has its limitations, these can be overcome by a clear protocol with all pre-specified details. Furthermore, there is a growing effort to harmonise the outcomes of interest across different specialities, which will eventually help in collaborative projects such as IPD-MA.

Conflicts of interest

There are no conflicts of interest.

Funding Statement

Nil.