From association to causation: interpreting PS-based analyses in real-world evidence

Abstract

Propensity score (PS) methods are widely used in real-world evidence studies to reduce confounding and to approximate randomized comparisons. However, statistical adjustment is often misinterpreted as evidence of causality, leading to overgeneralization and inappropriate clinical conclusions. In this statistical round, we present a clinically oriented framework for interpreting PS-based analyses, focusing on key principles of causal inference, including the definition of the estimand (average treatment effect [ATE] vs. average treatment effect on the treated [ATT]), covariate selection based on causal structure, assessment of positivity and overlap, and evaluation of robustness to unmeasured confounders. Using recent anesthesiology studies, we demonstrate that discrepancies between the analytical methods and interpretations are common. PS matching typically estimates the ATT and may not be generalizable to the entire population, whereas inverse probability of treatment weighting aims to estimate the ATE but may yield unstable estimates in the presence of limited overlap. Even with well-balanced covariates, unmeasured confounders remain a critical limitation. Failure to account for these issues may lead to biased estimates and overinterpretation of observational associations as causal effects. Causal interpretations of PS-based analyses require an alignment among the target estimand, underlying assumptions, and analytical methods. Rather than relying solely on statistical adjustment, researchers and clinicians should critically evaluate overlap, confounders, and generalizability. Transitioning from association to causation requires both advanced statistical methods and rigorous and transparent interpretations grounded in causal reasoning.

Introduction

Real-world evidence (RWE) studies are increasingly used in anesthesiology, providing large sample sizes and enhanced generalizability compared to randomized controlled trials [1]. To address confounding in observational data, propensity score (PS) methods such as propensity score matching (PSM) and inverse probability of treatment weighting (IPTW) have become standard analytical approaches [2–5]. These methods aim to balance the measured covariates between treatment groups and are often interpreted as approximating randomized comparisons.

However, despite their widespread adoption, PS-based analyses are frequently misunderstood. In many studies, statistical adjustment is implicitly equated to causal inference even though PS methods rely on strong assumptions that are rarely fully assessed in practice. From a causal inference perspective, a valid interpretation requires careful consideration of the underlying causal structure, often represented using directed acyclic graphs (DAGs), and the appropriate identification of confounders [6]. In addition, PS methods can only adjust for measured variables, and unmeasured confounders remain a major limitation [7].

A critical but often overlooked issue is that different PS methods may target different causal estimands, potentially leading to misinterpretation when the results are generalized beyond the target population [5,8,9]. Another key assumption is positivity, which requires sufficient overlap in PS distributions between the treatment groups [10]. Violations of this assumption may result in the exclusion of patients in matching analyses or in unstable estimates driven by extreme weights in the IPTW [10,11]. Despite its importance, overlap is often inadequately assessed. Furthermore, recent developments in causal inference and machine learning have emphasized that statistical methods alone cannot ensure valid causal inferences without appropriate assumptions and study design considerations [12–14].

Recent anesthesiology studies have further illustrated the importance of aligning analytical methods with causal interpretations [15,16]. Although these studies applied PS techniques appropriately, their conclusions extended beyond the limitations of observational data, including generalizations beyond the target population or implicit suggestions of causal or interventional effects.

Although previous methodological work has established the theoretical foundations of PS methods [2–5] and highlighted the importance of causal inference frameworks [12–14,17], a gap for clinicians and reviewers in translating these concepts into practical guidance remains. In particular, few studies have explicitly integrated key elements such as estimands, positivity, and sensitivity to unmeasured confounders with real-world examples from anesthesiology research.

Therefore, in this statistical round, we aimed to provide a clinically oriented framework for evaluating causal interpretations in PS-based RWE studies. We focus on three key aspects: (1) the distinction between the average treatment effect (ATE) and the average treatment effect on the treated (ATT) and its implications for interpretation, (2) the role of the positivity assumption and overlap in determining validity and generalizability, and (3) the importance of sensitivity analyses, including the assessment of unmeasured confounders [7]. Using recent anesthesiology studies as illustrative examples, we demonstrate how discrepancies between analytical methods and interpretations can lead to misleading conclusions. Ultimately, this review emphasizes that causal inference depends not only on statistical adjustment but also on the underlying assumptions, study design, and critical interpretations.

Conceptual framework for causal interpretation

This study adopts a conceptual framework based on the established principles of causal inference to guide the interpretation of PS-based analyses in RWE studies. Rather than focusing on statistical implementation, this framework emphasizes the alignment between analytical methods and causal interpretations.

The framework consists of four key components: (1) definition of the causal estimand, (2) appropriate covariate selection based on the causal structure, (3) assessment of the positivity assumption and overlap, and (4) evaluation of robustness to unmeasured confounders. Collectively, these elements determine the validity of causal interpretations in observational studies [6–8,17].

Definition of the causal estimand

A fundamental step in causal inference is the explicit definition of the causal estimand, which specifies the target population and effect of interest. The estimand determines not only what is being estimated but also to whom the results apply. Therefore, the interpretation of study findings must be grounded in the estimand rather than in the statistical method alone [8].

In PS-based analyses, different analytical approaches correspond to different estimands. PSM typically estimates the ATT because it restricts the analysis to individuals who can be successfully matched. Consequently, the target population becomes a subset of the original cohort, and the estimated effect reflects the outcomes among treated individuals with comparable controls [5,8].

In contrast, IPTW is generally used to estimate the ATE by constructing a weighted pseudo-population in which the treatment assignment is independent of the measured covariates. Under appropriate assumptions, this approach aims to represent the entire study population rather than a selected subgroup [5].

Despite these fundamental differences, PS-based results are often interpreted without reference to the underlying estimand. In particular, findings derived from the ATT are frequently generalized to all patients, implicitly assuming an ATE interpretation. This mismatch between estimand and interpretation can lead to overgeneralization and misrepresentation of the study findings.

The choice between the ATE and ATT should be guided by the underlying clinical question. For example, if the aim is to evaluate whether a drug should be applied broadly to all surgical patients, estimating the ATE may be appropriate. In contrast, if the goal is to understand the effect of a drug among patients who receive it in clinical practice, the ATT may be more relevant.

Recent anesthesiology studies have illustrated this issue. For example, a multicenter study using PSM reported an association between preoperative hyperglycemia and adverse cardiac outcomes [15]. However, because the analysis was based on matched data, the estimated effect corresponded to the ATT and therefore should not have been directly generalizable to the entire patient population. Applying such findings to all patients is a common but important misinterpretation.

Accordingly, clear identification and reporting of the estimand are essential for the valid interpretation of PS-based analyses. Researchers should explicitly state whether their analysis targets the ATE or ATT and readers should interpret the results within the appropriate target population defined by the estimand. These considerations highlight the importance of aligning interpretations with the target estimand (Table 1).

Table 1.

Key Considerations for Interpreting PS-based Analyses

Domain	Key considerations
Estimand
Key question	What is the target effect (ATE vs ATT)?
Common pitfall	Interpreting the ATT as ATE
Recommendation	Clearly define and report the estimand
Covariate selection
Key question	Are variables selected based on causal structure?
Common pitfall	Including colliders or mediators
Recommendation	Use DAGs to identify appropriate adjustment sets
Positivity (overlap)
Key question	Is there sufficient overlap between groups?
Common pitfall	Ignoring non-overlap or extreme weights
Recommendation	Assess PS distributions and consider trimming or truncation
Unmeasured confounders
Key question	Are unmeasured confounders considered?
Common pitfall	Assuming balance implies no bias
Recommendation	Perform sensitivity analyses (e.g., E-value)
Interpretation
Key question	Are conclusions aligned with the method and assumptions?
Common pitfall	Overgeneralization or overstatement of causality
Recommendation	Interpret results within the limits of the study design

Positivity: assumption that all individuals have a nonzero probability of receiving each treatment level given their covariates, overlap: extent to which the distribution of PS is similar between treatment groups, E-value: minimum strength of association that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain the observed association, ATE: average treatment effect, ATT: average treatment effect on the treated, DAG: directed acyclic graph; PS: propensity score.

Covariate selection based on causal structure

Appropriate covariate selection is a critical prerequisite for making valid causal inferences. In PS-based analyses, the variables included in the PS model determine which sources of confounders are addressed, and consequently, whether the estimated effect can be interpreted causally. Importantly, covariate selection should be guided by the underlying causal structure rather than by statistical associations alone [6].

From a causal perspective, the goal is to adjust for confounders (variables that influence both treatment assignment and outcome) while avoiding adjustment for colliders or mediators. Conditioning on colliders can introduce spurious associations between the treatment and outcome, whereas adjusting for mediators may block part of the causal pathway, leading to biased estimates of the total effect. DAGs provide a systematic framework for identifying appropriate adjustment sets and visualizing relationships among confounders, mediators, colliders, treatments, and outcomes [6]. Fig. 1 illustrates representative clinical examples relevant to anesthesiology, demonstrating how inappropriate adjustment for mediators or conditioning on colliders using commonly encountered clinical variables may distort causal interpretations. For example, variables such as age, American Society of Anesthesiologists physical status score, baseline hemodynamic status, and comorbidities often act as confounders as they influence both anesthetic selection and postoperative outcomes. Conversely, intraoperative variables such as drug dosage or hemodynamic responses may lie on the causal pathway and should be carefully considered to avoid inappropriate adjustments, which may attenuate or distort true treatment effects and lead to misleading conclusions.

Fig. 1.

Directed acyclic graphs illustrating key causal structures in PS-based analyses. (A) Confounder structure, in which a confounder (X) influences both the treatment (T) and outcome (Y), potentially generating a non-causal association between treatment and outcome. (B) Mediator structure, in which part of the treatment effect is transmitted through a mediator (M). (C) Collider structure, in which conditioning on a collider (K) may induce a spurious association between the treatment and outcome. Clinical examples relevant to anesthesiology are provided for each structure. Solid arrows indicate direct causal relationships, whereas dashed arrows indicate observed or induced associations between the treatment and outcome.

However, PS models are often constructed using variables selected based on their availability, statistical significance, or association with automated procedures. Such approaches may result in inappropriate adjustments, including the omission of important confounders or the inclusion of variables that introduce bias. Moreover, including variables that are strongly associated with the treatment but not directly associated with the outcome, sometimes referred to as potential instrumental variables, may increase the variance without reducing bias, further complicating interpretations [3].

Another common misconception is that including a larger number of variables necessarily improves causal validity. Although comprehensive adjustment may reduce residual confounding, it does not compensate for incorrect specification of the causal structure. The validity of PS-based analyses depends not only on the number of variables but also on whether the selected variables appropriately represent the causal relationships among the treatment, outcomes, and covariates.

Therefore, covariate selection should be based on subject-matter knowledge and explicit causal reasoning. When possible, researchers should articulate their assumptions using causal diagrams and justify their choices. Without this justification, even well-balanced PS models may yield estimates that are difficult to interpret causally.

At the conceptual level, covariate selection should be guided by the underlying causal structure rather than by purely statistical considerations. Confounders that influence both the treatment and outcome should be included, whereas conditioning on mediators or colliders may introduce bias. These relationships can be represented using DAGs that distinguish among confounders, mediators, and colliders, together with representative clinical examples relevant to anesthesiology (Fig. 1). Failure to account for the causal structure, including inappropriate adjustment for mediators or colliders, may result in biased estimates despite apparent covariate balance (Fig. 1, Table 1).

Assessment of positivity and overlap

The positivity assumption requires that all individuals have a nonzero probability of receiving each treatment level, given their covariates [10]. Practically, this assumption is reflected in sufficient overlap in the PS distribution between treatment groups. Without adequate overlap, causal comparisons cannot be made reliably because certain subgroups are observed under only one treatment condition.

Positivity violations have important implications for both internal validity and generalizability. In PSM, the lack of overlap leads to the exclusion of unmatched individuals, resulting in estimates that apply only to a restricted subset of the population. Consequently, the estimated effect corresponds to the ATT within a limited region of the covariate space, rather than the entire study population [5].

In IPTW, limited overlap often manifests as extreme or unstable weights assigned to individuals with a very low or high PS, particularly in skewed PS distributions. In such situations, observations located in the tail regions may be predominantly represented by one treatment group, making causal comparisons unstable or unreliable. To improve stability, approaches such as trimming non-overlapping regions, truncating extreme weights, or using stabilized IPTW may be considered, although these methods may cause the target population to be further restricted and the results should therefore be interpreted with caution. These extreme weights can produce unstable estimates that are disproportionately influenced by a small number of observations, thereby increasing variance and sensitivity to model misspecification [10,11]. In such cases, the estimated ATE may be driven by extrapolation beyond the common support region.

Despite its importance, the positivity assumption is often insufficiently assessed. Many studies report covariate balance after PS adjustment, but do not evaluate PS distribution or the extent of overlap between groups. However, achieving a balance does not guarantee adequate overlap, and both aspects should be considered when interpreting PS-based analyses.

To assess positivity and weight stability, graphical inspections of PS distributions, overlap between treatment groups, weight histograms, and Love plots may be helpful. In addition, diagnostic approaches, such as trimming non-overlapping regions or truncating extreme weights, may be used to improve stability, although these approaches further restrict the target population and should be interpreted accordingly.

Overall, the validity and generalizability of PS-based estimates depend critically on the presence of sufficient overlap. Without this, the causal interpretation becomes unreliable, regardless of the apparent success of the statistical adjustment. For instance, in clinical practice, certain anesthetic agents may be preferentially administered to high-risk patients, such as elderly individuals or those with significant comorbidities. In such situations, a limited overlap between treatment groups may occur, making it difficult to reliably estimate treatment effects across the entire population.

The validity of PS-based methods depends on the positivity assumption, which requires sufficient overlap in PS distribution between the treatment groups. When the overlap is adequate, comparisons between the groups are more reliable. By contrast, limited overlap may lead to unstable estimates, particularly in weighting-based approaches (Fig. 2).

Fig. 2.

Illustration of positivity and overlap in PS-based analyses. Panel (A) shows adequate overlap in propensity score distributions between the treatment and control groups, supporting valid comparisons. Panel (B) shows limited overlap, in which extreme propensity scores may lead to unstable estimates, particularly in inverse probability of treatment weighting. Limited overlap reduces both the validity and generalizability of causal effect estimates.

Therefore, an assessment of overlap is essential for both the validity and generalizability of PS-based estimates (Table 1).

Sensitivity to unmeasured confounders

Even when covariate balance is achieved using PS methods, bias due to unmeasured confounders cannot be eliminated. PS-based approaches adjust only for the observed variables, and their validity depends on the assumption that all relevant confounders have been measured and appropriately included in the model. In practice, this assumption is rarely fully satisfied [7,14].

Unmeasured confounders may arise from variables that are unavailable, imperfectly measured, or omitted due to a lack of awareness. In clinical studies, these factors may include disease severity, physician decision-making, or institutional practices that influence both treatment assignment and outcomes. Consequently, even well-balanced PS analyses may yield biased estimates if important confounders remain unaccounted for.

Sensitivity analyses provide a framework for evaluating the robustness of the findings to potential unmeasured confounders. One widely used approach is the E-value, which quantifies the minimum strength of association that an unmeasured confounder needs to have with both the treatment and outcome to fully explain the observed association [7]. A large E-value suggests that substantial unmeasured confounders would be required to negate the observed effect, whereas a small E-value indicates that the results may be sensitive to relatively weak confounders.

Despite its importance, sensitivity analyses are often omitted from PS-based studies. Without such assessments, determining whether the observed associations are robust or attributable to a hidden bias is difficult. This limitation is particularly relevant when the results are interpreted as causal or used to support clinical recommendations.

Therefore, causal interpretations of PS-based findings should be accompanied by explicit considerations of unmeasured confounders. Authors should acknowledge this limitation and, when possible, provide quantitative or qualitative assessments of its potential impact. Without such an evaluation, the conclusions derived from the observational data remain inherently uncertain (Table 1).

Clinical implications

In clinical practice, anesthesiologists frequently apply PS-based methods such as PSM and IPTW to reduce confounding in observational studies. However, despite appropriate implementation, discrepancies between analytical methods and interpretations remain common.

To illustrate these pitfalls and provide practical guidance, we present two representative examples from recent anesthesiology studies published in the Korean Journal of Anesthesiology [15,16], focusing on (1) the misinterpretation of estimands and (2) causal overinterpretation in weighting-based analyses. These practical considerations are summarized in the proposed framework shown in Fig. 3.

Fig. 3.

Practical framework for designing and interpreting PS-based analyses. The proposed framework summarizes key sequential steps in PS-based analyses, including defining the target estimand, assessing overlap and balance, evaluating unmeasured confounders, and making interpretations within the assumptions of causal inference. ATE: average treatment effect, ATT: average treatment effect on the treated, IPTW: inverse probability of treatment weighting, PS: propensity score, PSM: propensity score matching, SMD: standardized mean difference.

Case-based interpretations of PS-based studies

Case 1: Propensity score matching and interpretation of the ATT

Clinical scenario and PS method

A recent multicenter cohort study evaluated the association between preoperative hyperglycemia and adverse cardiac events after noncardiac surgery using PSM [15]. The authors demonstrated an improved covariate balance after matching and reported a significant association between hyperglycemia and postoperative outcomes.

Common misinterpretation

From a methodological perspective, the use of PSM was appropriate for reducing measured confounding. However, as PSM restricts the analysis to matched individuals, the estimated effect corresponded to the ATT rather than the ATE in the overall population [5,8]. Despite this distinction, ATT-based findings are often generalized to all surgical patients without considering the exclusion of unmatched patients or the limitations in generalizability.

Interpretation using the PS-INTERPRET framework

Therefore, interpretations should be restricted to the matched population represented by the ATT. A more appropriate conclusion would be the following: “Among patients with similar baseline characteristics, preoperative acute hyperglycemia was associated with a higher risk of adverse cardiac events.”

In addition, residual confounders cannot be completely excluded because PSM can only account for measured variables. Therefore, causal interpretations should be made cautiously and be consistent with the limitations of observational data. Table 2 summarizes the contrast between common misinterpretations and a more appropriate interpretation for this example.

Table 2.

Incorrect and Appropriate Interpretations in Representative PS-based Studies

	Incorrect interpretation	Appropriate interpretation
Case 1 [15]: Hyperglycemia and postoperative cardiac events (PSM)
Interpretation	“Preoperative hyperglycemia increases adverse cardiac events in all surgical patients.”	“Among matched patients with similar baseline characteristics, preoperative hyperglycemia was associated with a higher risk of adverse cardiac events.”
Key issue	ATT interpreted as ATE	Interpretation restricted to the matched population
Case 2 [16]: Tricuspid valve incompetence and LT outcomes (IPTW)
Interpretation	“The findings support clinical interventions targeting tricuspid valve incompetence.”	“The observed association should be interpreted cautiously considering overlap and residual confounders.”
Key issue	Direct causal interpretation of observational associations	Interpretation consistent with IPTW assumptions and limitations

PSM: propensity score matching, ATT: average treatment effect on the treated, ATE: average treatment effect, LT: liver transplantation, IPTW: inverse probability of treatment weighting.

Case 2: Inverse probability weighting and causal overinterpretation

Clinical scenario and PS method

Another study evaluated the association between anatomical tricuspid valve incompetence and outcomes following liver transplantation using IPTW [16]. By applying weighting techniques, the authors aimed to estimate the ATE in the overall study population.

Common misinterpretation

Although IPTW is designed to estimate population-level effects, its validity critically depends on key assumptions including positivity, sufficient overlap, and weight stability. In the absence of adequate overlap, extreme weights may arise, resulting in unstable estimates that are disproportionately influenced by a small number of observations [10,11]. However, observational associations derived from IPTW analyses are sometimes interpreted as direct evidence supporting clinical interventions or treatment strategies despite the absence of randomized evidence.

Interpretation using the PS-INTERPRET framework

Therefore, the interpretation of IPTW-based findings should remain consistent with the assumptions and limitations of causal inference in observational studies. A more appropriate interpretation would focus on the observed association while acknowledging the potential impact of limited overlap and residual confounders. For example, the conclusions should emphasize that the findings reflect associations observed under specific assumptions rather than definitive causal effects, as in the following: “However, given the observational nature of this study and the limitations of causal inference using PS-based methods, these findings should be interpreted with caution.”

Accordingly, in IPTW-based analyses, causal or interventional implications should not be overstated unless supported by additional evidence. Table 2 summarizes the contrast between common misinterpretations and a more appropriate interpretation for this example.

Key lessons from case-based interpretations

Together, these cases highlight the following three practical considerations for interpreting PS-based analyses: (1) the estimand defines the target population and must guide interpretation, (2) valid causal inference requires the assessment of key assumptions, including positivity and overlap, and (3) observational associations should not be directly translated into clinical recommendations without supporting evidence from appropriately designed studies.

Discussion

This statistical round provides a structured framework for interpreting PS-based analyses in RWE studies, emphasizing that causal interpretations depend on the alignment between the analytical methods, underlying assumptions, and the target estimand. Although PS methods are widely adopted to reduce confounding factors, our findings highlight that their use alone does not ensure a valid causal inference.

Unlike previous methodological reviews, this study explicitly integrated estimands, positivity, and sensitivity to unmeasured confounders into a unified, clinically interpretable framework.

A key issue identified in contemporary studies is the disconnection between methodological rigor and the interpretation of findings. PS techniques are often implemented correctly; however, the resulting estimates are extended beyond their intended scope. In particular, the failure to distinguish between estimands, such as interpreting the ATT derived from PSM as if it represents a population-level effect, remains a common source of overgeneralization. Similarly, an insufficient evaluation of assumptions, including positivity and weight stability in inverse probability weighting, may lead to unstable or non-generalizable estimates.

Our case-based examples illustrate how these issues manifest in clinical research. Even when covariate balance is achieved, causal interpretation may be compromised by limited overlap, unmeasured confounders, or inappropriate extrapolation. Moreover, observational associations are sometimes extended to support clinical interventions despite the absence of evidence supporting causality. These patterns underscore that methodological sophistication does not preclude errors in interpretation.

The primary strength of this study lies in the integration of key causal inference concepts into a clinically oriented framework. Rather than focusing on statistical implementation, we emphasized practical interpretations by linking theoretical constructs (e.g., estimands, causal structure, and sensitivity to unmeasured confounders) to real-world examples in anesthesiology. To facilitate practical interpretations, we summarize the key considerations for evaluating PS-based analyses in Table 1, propose a practical visual framework for study design and interpretation (Fig. 3), and include the PS-INTERPRET checklist we developed as a structured supplementary tool for evaluating PS-based analyses (Supplementary Material 1). This approach may facilitate a more critical appraisal of PS-based studies by clinicians and reviewers.

This study has several limitations. First, it is conceptual in nature and does not include empirical validation or a quantitative assessment of bias. The selected examples are illustrative, and may not capture the full diversity of PS applications. Additionally, while we highlight the core assumptions, the framework does not provide formal criteria for evaluating their adequacy in individual studies.

From clinical and methodological perspectives, our findings suggest that greater emphasis be placed on transparency and interpretation. Explicitly reporting estimands, assessing overlap, and evaluating robustness to unmeasured confounders are essential for meaningful interpretations. Importantly, conclusions should remain consistent with the limitations of observational data and unwarranted causal or interventional claims should be avoided.

As the use of RWE continues to expand in anesthesiology, the critical challenge is not the application of advanced statistical methods but the accurate interpretation of their results. Ensuring that analytical approaches, assumptions, and conclusions are coherently aligned is fundamental for valid causal inference and responsible clinical translation. In practice, clinicians and reviewers should explicitly verify the target estimand, assess overlap, and evaluate sensitivity to unmeasured confounders before interpreting PS-based results as causal.

In conclusion, PS-based methods have become integral to RWE-based research; however, their use alone does not establish causality. As highlighted in this statistical round, the validity of causal interpretations depends on more than statistical adjustment; it requires a clear definition of the estimand, careful assessment of key assumptions such as positivity and overlap, and explicit considerations of unmeasured confounders. Misalignments between analytical methods and interpretations are common. Results derived from specific estimands are often generalized beyond the target population and observational associations are frequently extended to causal or clinical implications without sufficient justification. Such practices risk overstating the evidence and may lead to inappropriate conclusions. Rather than focusing solely on methodological sophistication, greater emphasis should be placed on transparency, checking assumptions, and appropriate interpretations that are grounded in causal reasoning. In this context, PS methods should be viewed as tools that support, but do not replace, sound study design and critical thinking. Ultimately, valid causal interpretations in observational research depend not only on statistical adjustment but also on the appropriate alignment between estimands, underlying assumptions, and interpretations.

Supplementary Material

Supplementary Material 1.

PS-INTERPRET checklist for PS-based analyses.