Distinguishing cause from effect in psychological research: An independence‐based approach under linear non‐Gaussian models

Abstract

Distinguishing cause from effect – that is, determining whether x causes y (x → y) or, alternatively, whether y causes x (y → x) – is a primary research goal in many psychological research areas. Despite its importance, determining causal direction with observational data remains a difficult task. In this study, we introduce an independence‐based approach for causal discovery between two variables of interest under a linear non‐Gaussian model framework. We propose a two‐step algorithm based on distance correlations that provides empirical conclusions on the causal directionality of effects under realistic conditions typically seen in psychological studies, that is, in the presence of hidden confounders. The performance of the proposed algorithm is evaluated using Monte‐Carlo simulations. Findings suggest that the algorithm can effectively detect the causal direction between two variables of interest, even in the presence of weak hidden confounders. Moreover, distance correlations provide useful insights into the magnitude of hidden confounding. We provide an empirical example to demonstrate the application of our proposed approach and discuss practical implications and future directions.

1. INTRODUCTION

The primary driver underlying many studies in psychological sciences is to uncover cause–effect relationships, because causal (rather than associational) evidence provides the strongest foundation from which psychologists can develop targeted interventions and therapies to address the root causes of mental and behavioural health challenges (Grosz et al., 2020; Hamaker et al., 2020). Randomized experiments have traditionally been viewed as the gold standard for making causal inferences (Neyman, 1923; Rubin, 1974, 1990). Yet, it is impractical, unethical or even impossible to randomize subjects or to manipulate the desired independent variable in many scenarios (Grosz et al., 2020). Consequently, even when a research question is explicitly rooted in establishing causality, observational data often emerge as the only viable option for psychologists to investigate associated hypotheses (Grosz et al., 2020).

Deriving valid causal conclusions from observational studies poses significant challenges, given the variety of potential threats to causal inference in these situations. One of the primary threats to establishing causality in observational data is the issue of (hidden) confounding, also known as the third‐variable bias or omitted variable bias. A confounder is a common cause of both the predictor (x) and the outcome (y), thus artificially introducing a relationship between the two (Hill, 1965; Pearl, 1988, 2009). From a causal perspective, all possible confounders must be properly adjusted or controlled for when examining the relationship between x and y; otherwise, bias is likely to be introduced when estimating the corresponding causal effect. In response to these pitfalls, a variety of design strategies and statistical techniques (e.g., propensity score methods, Rosenbaum & Rubin, 1983; Thoemmes & Kim, 2011) have been put forward to reduce or even eliminate bias in observational studies due to confounders.

1.1. The issue of reverse causation

Even with appropriate adjustments for all potential confounders, however, other threats to causal inference persist. In this paper, we focus on such another challenge of establishing causal inference in observational data: reverse causation.¹ Reverse causation arises when the hypothesized statistical model is mis‐specified with respect to the causal direction (e.g., estimating a model of the form y → x instead of the causally correct model x → y).² Therefore, the primary goal of the current paper is discovering the causal effect directionality from the data – that is, determining whether x causes y (x → y) or, alternatively, whether y causes x (y → x) – rather than estimating the mere magnitude of a postulated causal effect.

Distinguishing cause from effect per se can be a primary research goal for many psychological studies because correctly identifying causal effect directionality has profound theoretical and empirical implications. For instance, being able to differentiate the causal models (1) self‐esteem → depression and (2) depression → self‐esteem is crucial for designing effective interventions. Under the first causal model, self‐esteem → depression, one hypothesizes that addressing low self‐esteem can mitigate symptoms of depression. However, when the causal pathway takes the form depression → self‐esteem, interventions solely focusing on boosting self‐esteem may prove ineffective in ameliorating depressive symptoms.

Despite its importance, methodologists in psychology have paid less attention to the issue of determining the causal direction of effects compared to other research areas. Prevalent methods for detecting causality typically involve using temporal ordering information from longitudinal data or relying on established theories (or intuition) to support a hypothesized model. For instance, researchers may conclude that low self‐esteem causes depression if self‐esteem is measured before depression under the assumption that a cause must precede its effect in time. However, relying solely on temporal precedence is not sufficient to conclusively support the causal model of self‐esteem → depression (Bellemare et al., 2017). Even if self‐esteem is measured at a time point (T1) that precedes the measurement of depression (T2), researchers cannot rule out the possibility that the actual causal model may be depression → self‐esteem, as depression from an earlier, unobserved time point (T0) might have contributed to both low self‐esteem (at T1) and depression symptoms (at T2). Aside from the insufficiency of longitudinal data in establishing the direction of causality, many researchers often do not have access to longitudinal data and can only rely on cross‐sectional data. The current study aims to distinguish causes from effects without relying on temporal information, focusing instead on cross‐sectional data.

Psychologists may also distinguish cause from effect based on common sense or established theories. However, substantive theories may not always be available. Furthermore, there are many cases where competing theories disagree on the direction of causation. Reconsider the causal link between self‐esteem and depression. The vulnerability model suggests that low self‐esteem is a causal factor for depression (i.e., self‐esteem → depression; Klein et al., 2011; Orth & Robins, 2013). In contrast to the vulnerability model, the scar model suggests that low self‐esteem is an effect of depression, rather than its cause (i.e., depression → self‐esteem; Orth & Robins, 2013; Shahar & Davidson, 2003).

In the absence of an established substantive argument or in the presence of competing theories, it would be helpful to test the direction of causation empirically. However, frequently used linear models, such as ordinary linear regression models and structural equation models (SEMs), depend only on first‐ and second‐order moments (i.e., means, variances and covariances) and cannot be used to distinguish between cause and effect. That is, when fitting ordinary linear regression models or SEMs, the two causally competing models, x → y and y → x, are statistically equivalent (or symmetric), thus, providing an equal representation of the information contained in the data.

1.2. Causal discovery: An advanced statistical perspective

In recent years, various causal discovery algorithms (e.g., Hyvärinen et al., 2001; Mooij et al., 2016; Peters et al., 2014; Shimizu et al., 2006) and statistical methods for directional dependence (e.g., Dodge & Rousson, 2000; Sungur, 2005; von Eye & DeShon, 2012; Wiedermann et al., 2021; Wiedermann & von Eye, 2015) have been developed. These advancements have enabled researchers to uncover the direction of causal relationships between variables using cross‐sectional, observational data. Although many causal discovery algorithms (e.g., Shimizu et al., 2011; Spirtes et al., 2001) are designed to explore the causal structures in multivariate data with multiple variables, we focus on direction dependence analysis (DDA), which is a confirmatory causal discovery technique aimed at discerning cause from effect in the case of two variables (i.e., x → y vs. y → x; while adjusting for potential background covariates) under a linear non‐Gaussian regression model ³ framework (Shimizu et al., 2006; Wiedermann et al., 2021).

Under linear non‐Gaussian models, causally competing models, x → y and y → x, become distinguishable (or asymmetric) using higher‐moment information (e.g., skewness, kurtosis, co‐skewness and co‐kurtosis) available in non‐normally distributed data. DDA offers a comprehensive framework to integrate various asymmetry properties of observed variables and regression errors. In this article, we concentrate on the property based on the independence of the predictors and regression errors, which is considered the most general DDA property. Statistical tests based on the independence of the predictor and regression errors (hereafter referred to as independence tests) have been proposed, allowing for the determination of the causal direction between two variables and the detection of hidden confounders (Wiedermann et al., 2021).

Although independence tests for causal discovery hold potential, they are widely used in psychological research. This might be attributable to a couple of factors. First, many applied psychologists might not be well acquainted with the property of independence within linear non‐Gaussian regression models, given that most training in psychology focuses on statistical models that assume normal distributions (e.g., ordinary linear regression models). This is a key missed opportunity, as the distributions of many psychological variables deviate from normality in practice (Blanca et al., 2013; Micceri, 1989). Second, the application of independence tests comes with a set of assumptions and intricate details. For instance, a common and critical assumption for many independence test‐based algorithms is the absence of unobserved confounders (known as the ignorability assumption in causal inference literature). This assumption is often violated in practical scenarios, however. Currently, there is no comprehensive guide (and few algorithms available) to help researchers discern causal directions while taking the issue of potential unobserved confounders into consideration.

1.3. Current study

Given the above, our study objectives are twofold. First, we aim to demystify the concept of independence asymmetry for causal discovery rooted in linear non‐Gaussian regression models, making it more accessible to applied psychologists. Second, we seek to enhance existing applications of independence testing by introducing a new algorithm that aids researchers in better ascertaining the causal direction of effects, in particular when unobserved confounders are expected to be present.

To achieve our objectives, the structure of the article is as follows: we first review the basics of linear non‐Gaussian regression models, focusing on one specific DDA component – asymmetry in the independence structure of predictors and the error terms. Next, we propose a two‐step algorithm based on distance correlations (Székely et al., 2007) to distinguish cause from effect. The performance of the proposed algorithm is then evaluated using Monte‐Carlo simulations (Study I). Specifically, we consider realistic psychological research settings involving the presence of hidden (latent) confounders. In Study II, we further explore the undesirable conditions identified in Study I, where hidden confounding is strong, and aim to offer additional guidelines to help researchers draw more robust conclusions. Moreover, to examine the generalizability of study findings, we conduct additional simulation experiments using a random factor design by generating simulation conditions from a wide range of plausibly encountered data scenarios (Study III). A real‐world data example is then provided to demonstrate the applicability of the proposed algorithm. We conclude the article by reflecting on the implications of our research, offering practical guidance for researchers and suggesting avenues for further investigation.

2. CAUSAL DISCOVERY IN LINEAR NON‐GAUSSIAN MODELS: INDEPENDENCE PROPERTIES OF PREDICTORS AND ERRORS

Let x and y be two continuous variables of interest. Given that x and y are correlated, there exist at least three potential explanations (or competing statistical models) regarding the direction of causation:

1. Model I: x is the cause and y is the effect, that is, x → y.

2. Model II: y is the cause and x is the effect, that is, y → x.

3. Model III: an unobserved common cause u accounts for the effect, that is, x ← u → y.

The three models listed above do not encompass all possible scenarios. For instance, reciprocal causal paths between x and y (i.e., x → y and, simultaneously, y → x) may be present, representing an example of cyclic models (Anderson, 1978; Park et al., 2023). As noted by Wiedermann and Sebastian (2020) and Wiedermann (2022), incorrectly ignoring a reciprocal causal path has similar statistical consequences to omitting a confounder. Therefore, from a mathematical perspective, reciprocal causality can be subsumed in Model III. In this study, we only focus on the acyclic case, where the causal relationship between the two variables is not reciprocal in nature.⁴

We first consider the unconfounded case. Assuming that the relationship between x and y is linear, and Model I constitutes the true causal mechanism, the ‘true’ model can be expressed using the following linear equation (without loss of generality, we assume zero intercepts):

$$y={\beta }_{\mathit{yx}}x+e_{\mathit{yx}}$$ (1)

In addition, let

$$x={\beta }_{\mathit{xy}}y+e_{\mathit{xy}}$$ (2)

describe the causally mis‐specified model (i.e., the model that erroneously treats x as the outcome and y as the predictor). In the above equations, ${\beta }_{\mathit{yx}}$ and ${\beta }_{\mathit{xy}}$ represent the population slopes (the intercepts are omitted without loss of generality), and $e_{\mathit{yx}}$ and $e_{\mathit{xy}}$ denote the error terms in each model.

In a similar manner, we can express the population model in the presence of an unobserved confounder (u) (Model III). Under hidden confounding (Model III), both x and y are caused by u; however, the true causal relationship between x and y can follow different patterns (e.g., x → y, y → x or no causal relationship between x and y). If we again assume a true causal mechanism of the form x → y, Model III can be expressed as:

$$y={\beta }_{\mathit{yx}}x+{\beta }_{\mathit{yu}}u+e_{\mathit{yu}}$$ (3)

$$x={\beta }_{\mathit{xu}}u+e_{\mathit{xu}}$$ (4)

where ${\beta }_{\mathit{xu}}$ and ${\beta }_{\mathit{yu}}$ are the causal effects from u to x and y, respectively. $e_{\mathit{yu}}$ and $e_{\mathit{xu}}$ represent corresponding error terms for variables x and y. All other notations are the same as defined above.

It is well recognized that Models I, II and III are not distinguishable when fitting ordinary least squares (OLS) linear regression models. Specifically, by observing only x and y and relying solely on first‐ and second‐order moments (i.e., means, variances and covariances), all three models provide an identical representation of the information contained in the data. However, as we discuss below, under linear non‐Gaussian models, asymmetry properties of Models I, II and III exist using higher than second moments information from the available data (i.e., x and y), which enables one to empirically distinguish the three models from each other.

Here, we focus on the asymmetry property derived from the independence structure between the predictors and error terms under the following assumptions: (1) the ‘true’ cause and/or the ‘true’ error term are non‐normally distributed; (2) the relationship between x and y is approximately linear; (3) there is the absence of outliers and influential data points that spuriously generate non‐normality; (4) the causal effect is homogeneous (i.e., constant across subjects); and (5) the putative predictor is free of measurement error. It is noted that assumptions 2–5 are commonly made in OLS regression models.

Under the above assumptions, the three causally competing models are distinguishable based on the independence structures between the predictors and model‐implied error terms of x → y and y → x. The independence patterns that uniquely distinguish Models I–III are summarized in Box 1.

BOX 1. Independence patterns for distinguishing competing Models I–III.

1. Model I (x → y): x is independent of $e_{\mathit{yx}},$ and y is not independent of $e_{\mathit{xy}}$.

2. Model II (y → x): x is not independent of $e_{\mathit{yx}},$ and y is independent of $e_{\mathit{xy}}$.

3. Model III (in the presence of u): x is not independent of $e_{\mathit{yx},}$ and y is not independent of $e_{\mathit{xy}}$.

A proof of these independence properties is available by applying the Darmois–Skitovich theorem (Darmois, 1953; Skitovich, 1953), as summarized in Box 2. We first consider the case of Models I and II. If x is the true cause, and y is the true effect (Model I), and we fit the causally correctly specified model (i.e., x → y), then the predictor x and the regression error term $e_{\mathit{yx}}$ must be independent under the linear regression exogeneity assumption. However, if we fit the causally incorrect model by mis‐specifying y as the predictor and x as the outcome variable (i.e., y → x), the predictor y and the regression error term $e_{\mathit{xy}}$ are no longer independent for the following reason.

BOX 2. The Darmois–Skitovich theorem (Darmois, 1953; Skitovich, 1953).

Suppose $v_1$ and $v_2$ are two linear functions of k (k ≥ 2) independent variates such that

$$v_1={\sum }_{i=1}^k{\alpha }_i{w}_i\text{and}{v}_2={\sum }_{i=1}^k{\beta }_i{w}_i.$$

If $v_1$ and $v_2$ are stochastically independent, and ${\alpha }_i{\beta }_i\ne 0$, then all $w_i$ must be normally distributed. Thus, when at least one common $w_i$ is non‐normally distributed, $v_1$ and $v_2$ must be stochastically non‐independent.

In Equation (1), the predictor y is expressed as $y={\beta }_{\mathit{yx}}x+e_{\mathit{yx}}$ in the population. In the causally mis‐specified model (y → x), the error term $e_{\mathit{xy}}$ can be rewritten, given Equations (1) and (2), as $e_{\mathit{xy}}=\left(1-{\beta }_{\mathit{xy}}{\beta }_{\mathit{yx}}\right)x-{\beta }_{\mathit{xy}}{e}_{\mathit{yx}}$. The above results imply that the false predictor y and the regression error term $e_{\mathit{xy}}$ share two common variates: x and $e_{\mathit{yx}}$. According to the Darmois–Skitovich theorem, x and $e_{\mathit{yx}}$ will not be independent whenever at least one of their common variates is non‐normally distributed.

Similarly, when unobserved confounders are present in the data‐generating model (i.e., Model III) but ignored in the analysis, the predictors and the corresponding error terms implied in both mis‐specified models, x → y and y → x (i.e., both models erroneously ignore u), are non‐independent. Specifically, the regression error terms under mis‐specified Models I (given by $y={\beta }_{\mathit{yx}}^{\prime }x+e_{\mathit{yx}}^{\prime }$) and II ($x={\beta }_{\mathit{xy}}^{\prime }y+e_{\mathit{xy}}^{\prime }$) are

$$e_{\mathit{yx}}^{\prime }=\left[{\beta }_{\mathit{yu}}+\left({\beta }_{\mathit{yx}}-{\beta }_{\mathit{yx}}^{\prime }\right){\beta }_{\mathit{xu}}\right]u+\left({\beta }_{\mathit{yx}}-{\beta }_{\mathit{yx}}^{\prime }\right)e_{\mathit{xu}}+e_{\mathit{yu}}$$ (5)

$$e_{\mathit{xy}}^{\prime }=\left[{\beta }_{\mathit{xu}}-{\beta }_{\mathit{xy}}^{\prime }\left({\beta }_{\mathit{yu}}+{\beta }_{\mathit{yu}}{\beta }_{\mathit{xu}}\right)\right]u+\left(1-{\beta }_{\mathit{xy}}^{\prime }{\beta }_{\mathit{yx}}\right)e_{\mathit{xu}}+{\beta }_{\mathit{xy}}^{\prime }{e}_{\mathit{yu}}.$$ (6)

As can be seen, $x$ and $e_{\mathit{yx}}^{\prime }$ share the confounder u and the regression error term $e_{\mathit{xu}}$ as their common variates, and y and $e_{\mathit{xy}}^{\prime }$ share u and $e_{\mathit{yu}}$ as their common variates. Thus, when x and y are non‐normal, $\text{both}x$ and $e_{\mathit{yx}}^{\prime }$, as well as y and $e_{\mathit{xy}}^{\prime }$, are non‐independent.

In summary, in linear non‐Gaussian models, causally competing Models I, II and III can be distinguished based on the independence patterns between the model predictors and error terms. In finite samples, bivariate causal discovery can be performed by fitting two causal models (x → y and y → x) using OLS linear regression, and testing, for both models, whether the observed predictors and estimated residual terms exhibit stochastic independence.

Notably, by construction, predictors and residual terms are always linearly uncorrelated (i.e., a Pearson correlation of zero) when fitting OLS linear regression models. Building on this observation, it is essential to highlight that stochastic independence and uncorrelatedness are related but distinct concepts. Statistically, stochastic independence implies uncorrelatedness, but uncorrelatedness does not imply stochastic independence.⁵ For instance, taking two random variables, x and y, stochastic independence implies that knowing the value of x does not reveal any information about the value of y. Uncorrelatedness, on the contrary, is a much weaker condition than independence, meaning that there is no linear relationship between x and y; however, x and y can still be dependent in some non‐linear way. As a result, in the context of non‐Gaussian causal discovery, predictor‐error independence needs to be assessed by independence measures that capture dependence structures beyond linear uncorrelatedness.

In the current study, we focus on the distance correlation coefficient (dCor; Székely et al., 2007). The distance correlation coefficient constitutes a natural extension of the standard Pearson correlation coefficient by providing an omnibus measure that can detect any form of dependence with finite second moments. A more technical definition of the distance correlation is given in Box 3.

BOX 3. The distance correlation coefficient (dCor, Székely et al., 2007).

For two samples of random variables $x_i$ and $y_i$ (i = 1, …, n, with n denoting the sample size), let $a_{\mathit{ij}}=‖x_i-x_j‖$ and $b_{\mathit{ij}}=‖y_i-y_j‖\left(i,j=1,\dots ,n\right)$ be the $n\times n$ distance matrices, which contain all pairwise distances, with $‖\bullet ‖$ denoting the Euclidean norm. The centred pairwise distances of x and y are given as $A_{\mathit{ij}}=a_{\mathit{ij}}-{\overline{a}}_{i·}-{\overline{a}}_{·j}+{\overline{a}}_{··}$, and $B_{\mathit{ij}}=b_{\mathit{ij}}-{\overline{b}}_{i·}-{\overline{b}}_{·j}+{\overline{b}}_{··}$, with ${\overline{a}}_{i·}$ and ${\overline{b}}_{i·}$ being the row means, ${\overline{a}}_{\bullet j}$ and ${\overline{b}}_{\bullet j}$ being the column means and ${\overline{a}}_{··}$ and ${\overline{b}}_{··}$ being the grand means of the distance matrices. The distance correlation coefficient between x and y, or dCor(x, y) is defined as

$$\text{dCor}\left(x,y\right)=\frac{\text{dCov}\left(x,y\right)}{\sqrt{\text{dCov}\left(x,x\right)·\text{dCov}\left(y,y\right)}},$$

with $\text{dCov}\left(x,y\right),$ representing the distance covariance of x and y, defined as $\text{dCov}\left(x,y\right)=\sqrt{\frac{1}{n^2}\sum \sum A_{\mathit{ij}}{B}_{\mathit{ij}}}$. $\text{dCov}\left(x,x\right)$ and $\text{dCov}\left(y,y\right)$ represent the distance variances of x and y, which can be computed as $\text{dCov}\left(x,x\right)=\sqrt{\frac{1}{n^2}\sum A_{\mathit{ij}}^2}$ and $\text{dCov}\left(y,y\right)=\sqrt{\frac{1}{n^2}\sum B_{\mathit{ij}}^2}$, respectively.

Like the Pearson correlation, dCor is a standardized association measure. However, unlike the Pearson correlation, dCor cannot take negative values; therefore, dCor is bounded on the interval [0, 1]. dCor equals 0 if and only if the two variables are stochastically independent. If the variables of interest are dependent, dCor is expected to be greater than zero.

2.1. Using dCor tests of independence for causal discovery

By applying the assumptions and properties described above, dCor tests of independence are well suited for causal discovery using the procedure outlined below. First, two linear models with different causal directions, that is, (1) x → y and (2) y → x, are fitted to the data. Next, distance correlation coefficients between the predictor and error from both models, namely dCor(x, $e_{\mathit{yx}}$) and dCor(y, $e_{\mathit{xy}}$), are estimated and tested for statistical significance (in practice, one uses the estimated residuals to approximate the ‘true’ errors). For simplicity, we denote dCor(x, $e_{\mathit{yx}}$) and dCor(y, $e_{\mathit{xy}}$) – population distance correlations derived from the causal models x → y and y → x – as dCor(x → y) and dCor(y → x), respectively, from this point forward. Finally, causally competing models can be distinguished using the independence patterns described in Box 1. More specifically, independence testing based on dCor can be conducted following one of the two different strategies: (1) testing the two distance correlations separately (separate tests) and (2) testing the difference in the two distance correlations (difference tests). In what follows, we will review each of these two approaches, discuss their pros and cons and introduce an integrative algorithm that combines both approaches, aiming to generate more informative conclusions for causal discovery.

2.1.1. Independence testing based on separate tests

In the first strategy, null hypothesis significance testing for the two distance correlations [i.e., H₀: dCor(x → y) = 0 and H₀: dCor(y → x) = 0] is conducted separately. Formal statistical tests for dCor using sample data are available, such as asymptotic tests based on t distributions (Székely & Rizzo, 2013) or ${\chi }^2$ distributions (Shen et al., 2022) and tests based on non‐parametric bootstrapping (Wang et al., 2015). In the current study, we focus on the permutation test as suggested by Székely and Rizzo (2013). The decision rule for the strategy of applying separate tests is summarized in Box 4.

BOX 4. Causal decision rule for separate tests.

• Condition 1: H₀: dCor(x → y) = 0 is retained, and H₀: dCor(y → x) = 0 is rejected. Decision: select the causal model x → y (Model I).

• Condition 2: H₀: dCor(x → y) = 0 is rejected, and H₀: dCor(y → x) = 0 is retained. Decision: select the causal model y → x (Model II).

• Condition 3: H₀: dCor(x → y) = 0 and H₀: dCor(y → x) = 0 are rejected. Decision: unmeasured confounder(s) exist, and the direction of causality is undetermined (Model III).

• Condition 4: H₀: dCor(x → y) = 0 and H₀: dCor(y → x) = 0 are retained. Decision: inconclusive information, and no decision can be made.

In summary, given no unobserved confounders, the strategy of separate tests allows researchers to differentiate between the causal models x → y and y → x. Using separate tests also allows researchers to identify the presence of hidden confounders, which can facilitate making more accurate statistical decisions. However, conclusions regarding causal directions derived from this strategy might be limited for at least two reasons.

First, previous simulation results have shown that the separate tests approach can have lower statistical power in detecting true causal directions (Pornprasertmanit & Little, 2012; Wiedermann & von Eye, 2015) compared to the difference test approach (introduced later in this section). The reason for this difference in statistical power is that the procedure requires only one of the two tests to retain the null hypothesis, a requirement that may often lead researchers to inconclusive decisions (i.e., Condition 4 in Box 4). This may be especially likely to occur under circumstances like small sample sizes and/or only minor deviations from normality. Second, when unmeasured confounders exist (i.e., Condition 3 in Box 4), the separate testing approach does not provide any information regarding the direction of causation. As discussed earlier, the strict assumption of no unmeasured confounding is likely to be violated in most psychological studies. In this sense, the direction of causality is more likely to be undetermined as sample sizes increase even if only weak confounders are ignored, because both null hypotheses are more likely to be rejected with increasing sample sizes.

2.1.2. Independence testing based on difference tests

To overcome some of the limitations associated with separate tests, testing causal directions can be conducted using an alternative strategy that computes the difference in distance correlations obtained from the two causally competing models, following ΔdCor = dCor(y → x) − dCor(x → y). The difference in distance correlations (ΔdCor) can be tested statistically using non‐parametric bootstrapping (Pollaris & Bontempi, 2020). Box 5 summarizes the decision rules for the difference testing strategy.

BOX 5. Causal decision rule for difference testing: ΔdCor = dCor(y → x) − dCor(x → y).

• Condition 1: the confidence limits of ΔdCor suggest ΔdCor >0. Decision: select the causal model x → y (over y → x).

• Condition 2: the confidence limits of ΔdCor suggest ΔdCor <0. Decision: select the causal model y → x (over x → y).

• Condition 3: zero falls within the confidence limits of ΔdCor. Decision: no decision can be made.

In the absence of unobserved confounders, separate and difference testing approaches are equivalent asymptotically. However, in finite samples, simulation results have shown that the difference tests approach can yield higher statistical power (Pornprasertmanit & Little, 2012; Wiedermann & von Eye, 2015). Moreover, researchers encounter fewer decision conditions by employing the difference tests as demonstrated in Box 5. Conclusions regarding causal directions can always be made, as long as the distance correlations from the two causally competing models are different from each other, even if they are both significantly greater than zero. The reason for this is that despite the presence of hidden confounding, the causally mis‐specified model can be expected to show stronger deviations from predictor‐error independence due to the presence of both reverse causation and confounder biases. This suggests that the difference test strategy may be used to identify causal directions regardless of minor violations of the ignorability assumption, and this conjecture has been supported by simulation results from Mooij et al. (2016).

Despite its merits, the trade‐off in applying the difference test strategy is that it requires researchers to make two additional assumptions, which may be hard to validate empirically. First, the difference test approach assumes the existence of a true direct causal path between x and y (i.e., either x → y or y → x). In other words, the difference test should not be applied when there is no causal relationship between x and y, meaning any association between x and y arises solely from u. Under this condition, both dCor(x → y) and dCor(y → x) are nonzero due to the presence of the confounder, and their magnitudes are determined by the characteristics of u (e.g., the magnitudes ${\beta }_{\mathit{xu}}\text{and}{\beta }_{\mathit{yu}}$). Therefore, as long as dCor(x → y) and dCor(y → x) are not equal, researchers run the risk of selecting a causal model (x → y or y → x), even though, in reality, no causal link exists.

Second, the validity of conclusions drawn from the difference test approach relies on the assumption that unmeasured confounding effects (if present) are weak to moderate the most. When hidden confounding effects are strong, the magnitudes of dCor(x → y) and dCor(y → x) are predominantly shaped by the confounded structure rather than the mis‐specified causal directionality. Consequently, under undesirable conditions where the true causal effect between x and y is weak and hidden confounding effects are strong, the application of the difference test may lead to erroneous inferences regarding causal directionality (e.g., concluding x → y when the true causal relationship is y → x). In summary, although the conclusions from the difference test approach may be more informative than those from separate tests, the additional information may come with greater uncertainty due to the extra assumptions. Quantifying the risk of erroneous causal conclusions under undesirable data conditions is among the aims of the presented Monte‐Carlo simulation experiments.

3. COMBINING THE TWO APPROACHES: A TWO‐STEP ALGORITHM

To capitalize on the strengths of each strategy, we extend existing dCor‐based causal discovery testing for linear non‐Gaussian models and propose a two‐step algorithm that combines separate and difference testing approaches. This two‐step algorithm is summarized in Box 6.

BOX 6. A two‐step algorithm for causal discovery using dCor.

1. Fit the linear Model I: x → y while controlling for all possible observed confounders. Estimate dCor(x → y).

2. Fit the linear Model II: y → x while controlling for all possible observed confounders. Estimate dCor(y → x).

Step 1: Separate tests

• 3Test H₀: dCor(x → y) = 0 to determine whether predictor and error term of Model I are independent.
• 4Test H₀: dCor(y → x) = 0 to determine whether predictor and error term of Model II are independent.
• 5 if H₀: dCor(x → y) = 0 is retained and H₀: dCor(y → x) = 0 is rejected, then conclude x → y.
• 6 if H₀: dCor(x → y) = 0 is rejected and H₀: dCor(y → x) = 0 is retained, then conclude y → x.
• 7 else proceed to step 2.

Step 2: Difference test

• 8Compute bootstrap confidence interval for ΔdCor = dCor(y → x) ‐ dCor(x → y).
• 9 if the confidence interval for ΔdCor suggests that ΔdCor >0, then conclude x → y.
• 10 if the confidence interval for ΔdCor suggests that ΔdCor <0, then conclude y → x.
• 11 else no decision can be made regarding the causal direction

end if

In the two‐step algorithm, researchers first conduct dCor‐based causal discovery tests using the separate testing strategy. To improve the accuracy of detecting the true causal direction, it is recommended to include all possible confounders during data collection to ensure that all observed confounders are included and controlled for when fitting the competing causal models. For the ease of presentation, both x and y will be considered covariate‐adjusted in the following discussion (see Wiedermann & Li, 2018; Wiedermann,  2021 for details on covariate adjustment in DDA). The true causal direction can be identified using the separate test strategy, provided there are no influential hidden confounders. The assumption of no unobserved confounders can also be tested using separate tests. If the results indicate either (1) the presence of an unmeasured confounder or (2) an inconclusive decision, researchers then proceed to step 2.

In step 2, the difference test (ΔdCor) allows researchers to reach more informative conclusions under either confounded or inconclusive scenarios mentioned above. First, when separate tests suggest the presence of hidden confounders, the difference test can still possibly recover the correct causal direction assuming that (1) there is a true direct causal path between x and y (i.e., either x → y or y → x), and (2) the level of omitted confounding effect is weak. Second, when separate tests suggest that both dCor(x → y) and dCor(y → x) are non‐significant, the ΔdCor test is more likely to recover the true causal direction given its higher statistical power, assuming there is a true path between x and y.

In summary, implementing a two‐step algorithm that combines separate and difference test strategies can yield more informative conclusions about the direction of causality under conditions that are likely to be observed in practice (e.g., a lack of statistical power due to small sample sizes or the presence of weak confounding factors). However, as previously discussed, drawing valid conclusions using the difference tests requires additional assumptions, which can be challenging to evaluate statistically. Therefore, in practice, conclusions must be made judiciously. We will revisit and further elaborate on these assumptions in later sections.

4. STUDY I: EVALUATING THE PERFORMANCE OF THE PROPOSED ALGORITHM USING MONTE‐CARLO SIMULATIONS

4.1. Data generation

In Study I, we conducted a Monte‐Carlo simulation study to evaluate the performance of the proposed algorithm. Specifically, we generated data from linear non‐Gaussian models and set x → y as the population causal model. We also introduced a confounding variable (u) that linearly caused both x and y (see Figure 1 for a path diagram of the population model) but omitted this variable in the data analysis to mimic a realistic research scenario where the assumption of no hidden confounders is violated. The confounding variable (u) was generated from a Gamma distribution with the levels of non‐normality manipulated, as discussed later. The error terms for both x and y were also generated from a Gamma distribution, with the scale parameter (θ) set to 1 and the shape parameter (k) set to 1.77, thereby fixing the population skewness at 1.50. Linear transformations were applied to the generated endogenous variables and error terms to set the population variances of all variables to one. Consequently, all the population path coefficients manipulated and discussed below are presented on standardized metrics. Within the simulation, we manipulated sample size, levels of non‐normality of the (hidden) confounder, effect size of the x → y path, effect size of the u → y path and effect size of the u → x path.

FIGURE 1.

A path diagram of the population model.

4.1.1. Sample size (N)

The sample sizes considered in this study were 200, 500 and 1000. These levels of sample sizes were selected to represent a broad range typically seen in behavioural and psychological studies, corresponding to small, medium and large sample sizes, respectively (Fraley et al., 2022; Shen et al., 2011).

4.1.2. Level of non‐normality of the confounding variable

The confounding variable (u) was generated from a Gamma distribution, with the scale parameter (θ) set to 1, and the shape parameter (k) manipulated to create three levels of population skewness: .75, 1.50 and 2.25. These population skewness values represent minor, medium and severe non‐normality, respectively, and are consistent with previous simulation studies (Cain et al., 2017; Wiedermann et al., 2017; Wiedermann & Li, 2018, 2019).

4.1.3. Effect size of path coefficients

The causal population path coefficient for x → y (${\beta }_{\mathit{yx}}$) was manipulated at three levels: .14, .39 and .59. These population values represented small, medium and large effect sizes, respectively (Cohen, 1988; MacKinnon et al., 2002; Shi et al., 2023). Additionally, we examined various levels of confounding by modifying the magnitude of the u → y and u → x paths. Specifically, for the u → y path (${\beta }_{\mathit{yu}}$), we considered three effect size levels: .14 (small), .39 (medium) and .59 (large). We considered the same three levels of effect sizes for the u → x path (${\beta }_{\mathit{xu}}$). We also explored scenarios of negative confounding by inverting the u → x path to negative values while maintaining the same magnitude. This approach resulted in six levels of effect size: −.14, −.39, −.59, .14, .39 and .59. When the u → x path sign is negative, creating opposite signs between the u → x path and the u → y path, u functions as a negative confounder. Therefore, we created the simulation conditions with 18 levels of confounding effect, ranging from weakly negative (${\beta }_{\mathit{xu}}$ = −.14; ${\beta }_{\mathit{yu}}$ = .14) to strongly negative (${\beta }_{\mathit{xu}}$ = −.59; ${\beta }_{\mathit{yu}}$ = .59), and from weakly positive (${\beta }_{\mathit{xu}}$ = .14; ${\beta }_{\mathit{yu}}$ = .14) to strongly positive (${\beta }_{\mathit{xu}}$ = .59; ${\beta }_{\mathit{yu}}$ = .59).

In summary, we considered a total of 486 different simulation conditions (three levels of sample size × three levels of skewness of the confounding variable (u) × three levels of effect size of the x → y path × three levels of effect size of the u → y path × six levels of effect size of the u → x path). Note that nine conditions were not included because they were inadmissible (i.e., conditions with ${\beta }_{\mathit{xu}}$ = ${\beta }_{\mathit{yu}}$ = ${\beta }_{\mathit{yx}}$ = .59, which yields negative error variances). Therefore, the final number of simulated conditions was 486–9 = 477. Each simulation condition was replicated 500 times using R 4.1.2 (R Core Team, 2023).

4.2. Data analyses and outcome variables

For each simulated dataset, we applied the proposed two‐step algorithm to determine the causal direction between x and y while disregarding the confounder u. That is, we first conducted separate tests using permutation tests with 1000 resamples (implemented in the R package ‘energy’ version 1.7–11; Rizzo et al., 2022), and then we performed difference tests for dCors employing bias‐corrected and accelerated (BCa) bootstrap confidence intervals based on 1000 bootstrap samples.

Based on the two‐step algorithm, three distinct conclusions can be drawn regarding the causal direction between x and y: (1) x → y, (2) y → x and (3) inconclusive. To assess the performance of the proposed algorithm to detect the ‘true’ causal model, we summarized the proportion of causal decisions made across 500 replications for each simulated condition. Specifically, we examined the hit rate, defined as the proportion of replications that correctly detect the true direction of causality (i.e., x → y). The error rate denotes the proportion of replications where an incorrect (reversed) causal direction (i.e., y → x) is concluded. An inconclusive decision (i.e., no decision can be made regarding causal direction at both steps) is classified neither as a hit nor an error. Therefore, we categorize the proportions of replications resulting in inconclusive decisions separately, referring to these as inconclusive rates.

4.3. Results

To facilitate the visualization and interpretation of the results, we present the various proportions across conditions using percentage‐stacked bar charts. The complete results, which report the detailed proportions of causal model decisions, are provided in Supplementary Material A (i.e., Tables A1–A3).

Figures 2, 3, 4 ⁶ show the hit rates, error rates and inconclusive rates for various simulation conditions, with each figure representing a specific level of non‐normality of the confounding variable. The performance of the two‐step algorithm exhibited similar patterns across the different distributions of the hidden confounding variable. In general, the algorithm effectively detected the true causal direction, even under simulated conditions with hidden confounders. Out of the 477 simulation conditions, 364 (76.3%) resulted in hit rates above 50%. It is important to note that low hit rates did not necessarily lead researchers to incorrect conclusions though. Instead, when the algorithm failed to detect the true causal direction of x → y, researchers generally made an inconclusive decision. Only 13 (2.7%) of the 477 simulated conditions yielded error rates above 50%. As shown in the figures, the performance of the algorithm noticeably improved, yielding higher hit rates and lower error rates, as the level of the hidden confounder decreased, the effect size of the x → y path increased, and, generally, as the sample size increased.⁷

FIGURE 2.

Proportions of causal effect directionality: Skewness of u = .75.

FIGURE 3.

Proportions of causal effect directionality: Skewness of u = 1.50.

FIGURE 4.

Proportions of causal effect directionality: Skewness of u = 2.25.

In conditions with a medium or large effect size for the x → y path, the two‐step algorithm effectively recovers the true causal direction, even when N = 200. Overall, 89.8% (283 out of 315) of the simulated conditions yielded hit rates above 50%, and 80.3% (253 out of 315) of the conditions achieved hit rates above 80%. Notably, the error rates were lower than 15% across all simulated conditions and were below 5% in 309 out of the 315 simulated conditions (98.1%). Consistent with earlier findings, higher hit rates and lower error rates were generally observed as the sample size increased and the confounding effects decreased. Holding the level of $\left|{\beta }_{\mathit{xu}}\times {\beta }_{\mathit{yu}}\right|$ constant, a decrease in the absolute value of the u → y path was associated with higher hit rates and lower error rates. For example, when the skewness of the hidden confounder was 1.50, with ${\beta }_{\mathrm{yx}}$ = .39, ${\beta }_{\mathit{xu}}$ = .39, ${\beta }_{\mathit{yu}}$ = .59 and N = 200, the hit rate, error rate and inclusive rate were 56.6%, 4.6% and 38.8%, respectively. By keeping everything else the same, but with ${\beta }_{\mathit{xu}}$ = .59, ${\beta }_{\mathit{yu}}$ = .39 and N = 500, the hit rate increased to 97.2%, the error rate decreased to 0%, and the inconclusive rate dropped to 2.2%.

When the effect size of the x → y path was small (i.e., ${\beta }_{\mathit{yx}}$ = .14), acceptable hit rates were only achieved for N ≥ 500. Specifically, when ${\beta }_{\mathrm{y}x}$ = .14 and N ≥ 500, 62.0% (67 out of 108) of the conditions yielded hit rates above 50%, and 40.7% (44 out of 108) of the conditions yielded hit rates above 80%. Only 17.6% (19 out of 108) of the conditions produced error rates above 20%, and 10.2% (11 out of 108) of the conditions produced error rates above 80%. Higher hit rates and lower error rates were associated with lower levels of the omitted confounding effect (e.g., $\left|{\beta }_{\mathit{xu}}\right|$ < .39 and/or $\left|{\beta }_{\mathit{yu}}\right|$ < .39). Given the same level of $\left|{\beta }_{\mathit{xu}}\times {\beta }_{\mathit{yu}}\right|$, higher hit rates and lower error rates were more likely to be observed when (1) the confounding effect was positive, and (2) the u → x path became larger in absolute value. For example, when the skewness of the hidden confounder was .75, with ${\beta }_{\mathit{yx}}$= .14, ${\beta }_{\mathit{xu}}$ = −.39, ${\beta }_{\mathit{yu}}$ = .59 and N = 500, the hit rate, error rate and inclusive rate were .4%, 70.8% and 28.8%, respectively. With ${\beta }_{\mathit{xu}}$ = .59 and ${\beta }_{\mathit{yu}}$ = .39, the hit rate increased to 90.8%, and the error rate dropped to .4%.

Undesirable conditions with error rates above 50% were observed only when the effect size of the x → y path was small (i.e., ${\beta }_{\mathit{yx}}$ = .14), and a strong negative confounding variable was omitted. Under these unfavourable conditions, the performance worsened as the sample size increased. For instance, as the sample size increased to 1000 in this case, the proportion of making an incorrect decision increased from 70.8% to 86.6%.

5. STUDY II: THE RELATIONSHIP BETWEEN THE DISTANCE CORRELATIONS AND THE LEVEL OF HIDDEN CONFOUNDERS

Results from Study I showed that, generally, the two‐step algorithm can detect the true causal direction between two variables, even under hidden confounding. However, high error rates may be observed under certain undesirable conditions when omitting a strong confounding variable, particularly if the true causal path has a small effect size. Therefore, when applying the proposed algorithm in practice, the presence or absence of unobserved confounding – and, if present, the expected level of confounding – should be justified through substantive theory or evaluated empirically.

As discussed earlier, the presence of hidden confounder(s) can be detected using separate dCor tests (i.e., step 1). If the results from the first step indicate the existence of unmeasured confounder(s), the algorithm's second step can still help researchers to differentiate between causally competing models using difference tests (i.e., ΔdCor), provided that hidden confounding is weak. Unfortunately, the strength of hidden confounding cannot be directly tested in practice because hidden confounders, by definition, are unobserved. However, when applying the algorithm, researchers can gain valuable insights into the magnitude of hidden confounders by analysing the values of dCor on their own, as we will explore next.

Recalling the independence properties discussed earlier, in correctly specified causal models, the predictor and the error term are independent (i.e., dCor = 0) in the absence of unmeasured confounders. However, when hidden confounders are present, the predictor and the error term become stochastically dependent (i.e., dCor >0), even when the causal model is correctly specified. In other words, in models with correctly specified causal directionality, dCor reflects only the influence of the unmeasured confounder(s). Thus, the magnitude of dCor obtained from the model with the correct causal directionality could serve as a potential measure of the hidden confounding effect.

To illustrate this point, in Study II, we further explored the relationship between the level of unmeasured confounders and the distance correlations (dCors). We expected that the distance correlation of the correctly specified causal model, that is, dCor(x → y), increases as the magnitude of the hidden confounding becomes stronger. We considered the same 477 simulation conditions as reported in Study 1 to investigate this hypothesis. For each simulated condition, we calculated the means (or expected values) of the distance correlations from the two causally competing models, dCor(y → x) and dCor(x → y). Figure 5 illustrates average distance correlations across various levels of confounding effects. The detailed results of distance correlations across various levels of confounding effects are reported in Supplementary Material B (i.e., Tables B1–B3).

FIGURE 5.

The relationship between distance correlations and the level of hidden confounders.

Results indicate that for correctly specified causal models (x → y), average distance correlations dCor(x → y) are associated with the level of the hidden confounding variable. That is, dCor(x → y) increased when the level of confounding became stronger, as shown at the two outer regions of the plots. Lower values of dCor(x → y) were associated with more minor levels of hidden confounding, as indicated around the centre of the plots. On the contrary, in general, models with mis‐specified causal directionality exhibited noticeably higher distance correlations, dCor(y → x) compared to dCor(x → y), with a clear separation. However, as expected, in cases of strong hidden confounding, dCor(y → x) and dCor(x → y) became less distinguishable, with dCor(y → x) occasionally yielding slightly smaller values.

In the current research setting, the true causal directionality is unknown in practice, as the goal of the proposed algorithm is to identify it. In step 2 of the algorithm, the final causal model was selected based on the model that yielded a significantly smaller distance correlation coefficient. Based on the patterns indicated above, the dCor value derived from the final model, that is, min[dCor] = min[dCor(x → y), dCor(y → x)], may serve as a rough indicator of the magnitude of hidden confounding effects. In this context, conclusions drawn from the difference testing in step 2 can be further validated when the distance correlation derived from the final causal model, or min[dCor], is relatively small.

Thus, a practical decision involves determining a threshold dCor value that is considered sufficiently small. We acknowledge that this decision is inherently subjective and empirical. In this exploratory study, we initially proposed using min[dCor] ≤ .10 as a potential cut‐off value, as indicated by the horizontal lines in Figure 5, to reduce the risk of drawing incorrect causal conclusions from the two‐step algorithm. This proposal was based on a visual inspection of the evidence shown in Figure 5 and the benchmark for effect sizes in correlation coefficients (Cohen, 1988). The decision rule for the difference tests in step 2 would then be modified by adding additional selection conditions outlined in Box 7.

BOX 7. Modified causal decision rule: Difference testing: ΔdCor = dCor (y → x) − dCor (x → y).

• Condition 1: the confidence limits of ΔdCor suggest ΔdCor >0 and dCor (x → y) ≤ .10. Decision: select the causal model x → y (over y → x).

• Condition 2: the confidence limits of ΔdCor suggest ΔdCor <0 and dCor (y → x) ≤ .10. Decision: select the causal model y → x (over x → y).

• Condition 3: all other outcomes. Decision: no decision is made.

It is worth noting that the modified decision rule adopts a conservative approach, aiming to minimize the chance of erroneous decisions under undesirable confounding conditions. However, using this modified decision rule is likely to reduce the hit rates of the proposed algorithm in detecting true causal directions under favourable conditions (e.g., when the x → y path has a medium‐to‐large effect size). Under these conditions, dCor coefficients from both causal models are expected to yield values above the .10 cut‐off. Nevertheless, the difference in dCors could still provide valid information regarding the true causal direction. For instance, as shown in Figure 5, under certain conditions where both dCor(x → y) and dCor(y → x) exceed .10, dCor(y → x) still yielded noticeably larger values than dCor(x → y).

We further evaluated our simulation results from Study II using the modified decision rule with a .10 cut‐off. The updated simulation outcomes are reported in Figure 6,⁸ with results showing that the hit rates decreased for many simulation conditions as expected; only 41.7% (199 out of 477) of the conditions yielded hit rates above 50%. However, this conservative strategy noticeably reduced the number of conditions with high error rates. Only 1 out of the 477 conditions (.2%) generated error rates above 50% (i.e., 57.4%), and most of the conditions (465 out of 477, or 97.5%) had a low error rate below 10%. In practice, researchers may find value in applying this conservative approach when they lack prior knowledge or theories about the effect size of the true causal effect or the magnitude of potential hidden confounding.

FIGURE 6.

Proportions of causal effect directionality using additional min[dCor] ≤ .10 cut‐off: Skewness of u = .75.

6. STUDY III: ASSESSING THE GENERALIZABILITY OF FINDINGS

The simulation study in Study I was comprehensive, encompassing 477 conditions. However, there were still limitations to the simulation design. First, we only manipulated the level of non‐normality in the confounding variables, whereas the distributions of the error terms for both x and y remained fixed. Additionally, we focused exclusively on non‐normal distributions characterized by both elevated skewness and kurtosis. In practice, the non‐normal distribution might also be symmetric (i.e., with population skewness = 0). Second, as with most simulation studies, although we manipulated various levels of many population parameters (e.g., the size of the x → y path), the values we considered were somewhat limited.

To examine the generalizability of the current findings and recommendations, we conducted additional simulations in Study III with a random factor design by generating simulated conditions from a wide range of plausibly encountered scenarios (Shear & Zumbo, 2013). For these additional stimulations, we considered the same population model (as shown in Figure 1). The simulation factors we manipulated included (1) the distribution of the confounding variable u, (2) the distribution of the error term for variable x, (3) the distribution of the error term for y, (4) the effect size of the x → y path, (5) the effect size of the u → x path and (6) the effect size of the u → y path.

Instead of fixing the manipulated variables to certain levels, we created various simulation conditions by randomly generating the levels (or values) of the simulated variables from a predetermined distribution discussed below. Specifically, the confounding variable (u) and the error terms for x and y were generated with either symmetric or asymmetric non‐normal distribution as determined by a Bernoulli process with equal probability. In the asymmetric conditions, the variables were generated from a Gamma distribution, with population skewnesses randomly generated from a uniform distribution within the range of [−2.5, 2.5]. Symmetric non‐normal distributions were generated from the Johnson family (Johnson, 1949), with pre‐specified population excess kurtosis values. These values were randomly generated from a uniform distribution within the range of [−1.2, 7] (note that the lower bound reflects the excess kurtosis of the uniform distribution). Finally, the population values for each of the three causal path coefficients (i.e., x → y, u → x and u → y) were generated from a uniform distribution within the range of [−.59, .59]. Based on findings from Study I, the sample size was fixed at 500. Using the six design factors, we generated 500 unique simulation conditions, with the specific level of each factor being randomly drawn from their respective distributions, as described above. For each condition, 500 replications were simulated. The performance of the two‐step algorithm was then evaluated across these 500 random conditions based on the hit rates, error rates and inconclusive rates. For brevity, the detailed results for Study III are reported in Supplementary Material C (i.e., Table C1).

Overall, the results showed that the two‐step algorithm performed similarly to Study I across the additional simulation conditions examined. Higher hit rates were associated with a larger effect size of the true causal path (i.e., x → y) and/or a weaker unmeasured confounding effect. In 254 out of the 500 (50.8%) conditions, hit rates exceeded 50%. It is noteworthy that even under conditions with lower hit rates, the error rates were not necessarily inflated. Of the 500 conditions, only 4 (or .8%) produced error rates above 50%, and only 47 (or 9.4%) had error rates above 10%. The chance of making incorrect decisions decreased further when additional selection criteria (i.e., min[dCor] ≤ .10) were applied; here, the number of conditions with error rates above 50% and 10% decreased to 2 (.4%) and 30 (6%), respectively. Taken together, results from all three simulation studies support the conclusion that the proposed algorithm shows adequate performance and suggest that our recommendations are generalizable.

7. AN EMPIRICAL EXAMPLE

We now demonstrate the application of the proposed framework using data from real‐world psychological research. The dataset comes from a published study (Zhang et al., 2022). In the current work, we focus on a sample of adult employees and aim to explore the causal direction between counterproductive work behaviours and sleep disturbances.

The data were collected from 603 adult employees in China, with 117 participants excluded for missing more than 1 out of 4 quality control items, leading to a final sample size of 486. In the final sample, 195 (40.1%) participants self‐identified as males and 291 (59.9%) participants self‐identified as females. The age of the participants ranged from 18 to 80 years (M _age = 29.38, standard deviation [SD] = 8.88). The final dataset included no missing observations.

Counterproductive work behaviour was measured using the 10‐item short version of the Counterproductive Work Behaviour Checklist (CWB; Spector & Fox, 2010). Respondents rated the frequency of 10 workplace behaviours (e.g., ‘insulted or made fun of someone at work’) on a five‐point Likert‐type scale ranging from 1 (never) to 5 (every day). Cronbach's alpha in the current sample was α = .87.

Sleep disturbance was measured using the four‐item subscale from the Physical Health Questionnaire (PHQ; Schat et al., 2005). Respondents rated the frequency of sleep‐problem‐related behaviours and symptoms (e.g., ‘how often have you had difficulty getting to sleep at night’) on a seven‐point scale ranging from 1 (‘not at all’) to 7 (‘all the time’). Cronbach's alpha in the current sample was α = .75.

In addition to the two variables of interest, several potential confounders were considered, including the Big 5 personality traits (measured by the 60‐item Big Five Inventory‐2; Soto & John, 2017), global job satisfaction (measured by the 9 items covering 9 aspects of job identified in Spector, 1985), aggression (measured by the 12‐item Short Form Aggression Questionnaire; Bryant & Smith, 2001) and perceived stress (measured by the 10‐item Perceived Stress Scale; Cohen et al., 1983). These confounders were selected based on substantive theories suggesting that they may potentially cause both counterproductive work behaviours and sleep disturbances (Berry et al., 2012; Czarnota‐Bojarska, 2015; Dalal, 2005; Gardani et al., 2022; Litwiller et al., 2017; Penney & Spector, 2005; Stephan et al., 2018; Van Veen et al., 2021).

For each construct, we used the sum score across items for data analysis. We first fit regression models to obtain the covariate‐adjusted scores for counterproductive work behaviours and sleep disturbances. The descriptive statistics with normality tests for both focal variables are summarized in Table 1. Univariate histograms and bivariate scatterplots (with a LOWESS curve superimposed) of the covariate‐adjusted scores are summarized in Figure 8. As shown in Table 1, the covariate‐adjusted scores are non‐normally distributed for both counterproductive work behaviours (skewness = .68, excess kurtosis = .87; Kolmogorov–Smirnov statistic = .061, p < .001; Shapiro–Wilk statistic = .972, p < .001) and sleep disturbances (skewness = .46, excess kurtosis = .12; Kolmogorov–Smirnov statistic = .052, p < .001; Shapiro–Wilk statistic = .986, p < .001). For the sum scores of the CWB, the percentages of minimum and maximum values were 7.2% and 0%, respectively, which were below the commonly used cut‐off (15%; Lim et al., 2015) for floor/ceiling effects. In terms of the PHQ sleep disturbance subscale, there were no observations among the participants with the maximum or minimum possible scores. Therefore, the non‐normality of the variable was not simply due to influential data points or floor/ceiling effects. The scatterplot and the LOWESS curve in Figure 7 suggest that the relationship between counterproductive work behaviours and sleep disturbances is sufficiently linear, with a positive Pearson correlation r = .14 (p < .01). In Table 2, we report the results for the two competing multiple regression models: (1) counterproductive work behaviours sleep disturbances and (2) sleep disturbances → counterproductive work behaviours, while controlling for the confounding variables listed above.

TABLE 1.

Descriptive Statistics for variables of interest.

	Counterproductive work behaviours	Sleep disturbances
SD	4.20	2.56
Skewness	.68	.87
Kurtosis	.46	.12
Kolmogorov–Smirnov test	D = .061 (p < .001)	D = .052 (p < .001)
Shapiro–Wilk test	W = .972 (p < .001)	W = .986 (p < .001)

Abbreviation: SD, standard deviation.

FIGURE 8.

Scatterplot with LOWESS curve and marginal histograms of covariate‐adjusted counterproductive work behaviours and sleep disturbances.

FIGURE 7.

A flowchart for applying the two‐step algorithm in practice.

TABLE 2.

Results of the two causally competing regression models to explain the relationship between counterproductive work behaviours and sleep disturbances while adjusting for covariates (n = 486).

Target model: Counterproductive work behaviours → sleep disturbances					Alternative model: Sleep disturbances → counterproductive work behaviours
Sleep on	b	95% CI		p‐value	CWB on	b	95%CI		p‐Value
		Lower	Upper				Lower	Upper
CWB	.087	.03	.14	.002	Sleep	.234	.09	.38	.002
Aggression	.030	−.01	.07	.154	Aggression	.187	.12	.25	<.001
Stress	.088	.03	.14	.002	Stress	.112	.02	.20	.017
Satisfaction	.048	.01	.09	.020	Satisfaction	−.184	−.25	−.12	<.001
Big 5 – O	−.038	−.09	.02	.182	Big 5 – O	.061	−.03	.15	.196
Big 5 – C	.108	.05	.17	<.001	Big 5 – C	−.030	−.13	.07	.549
Big 5 – E	−.015	−.07	.04	.603	Big 5 – E	.046	−.04	.14	.321
Big 5 – A	−.001	−.06	.06	.970	Big 5 – A	−.255	−.35	−.16	<.001
Big 5 – N	.048	−.01	.11	.601	Big 5 – N	−.104	−.21	.00	.048
$R^2$	.163				$R^2$	.379

Note: CWB = counterproductive work behaviours; sleep = sleep disturbances; aggression = aggression; stress = perceived stress; satisfaction = global job satisfaction; Big 5 – O = openness to experience; Big 5 –C = conscientiousness; Big 5 – E = extraversion; Big 5 – A = agreeableness; Big 5 – N = neuroticism.

Next, we explored the causal direction between counterproductive work behaviours and sleep disturbances using the proposed dCor testing framework. Results are summarized in Table 3. First, we conducted permutation tests for dCors across the two competing causal mechanisms with α = .05 and observed that p[dCor (sleep disturbances → counterproductive work behaviours)] < .05 and p[dCor (counterproductive work behaviours → sleep disturbances)] > .05. These results suggest that the potential confounders were adequately controlled, and that employees' counterproductive work behaviours caused sleep disturbances.

TABLE 3.

Results for causal discovery using the two‐step algorithm.

Tests	Target causal model	Alternative causal model
	Counterproductive work behaviours → sleep disturbances	Sleep disturbances → counterproductive work behaviours
dCor	dCor = .097, p = .144	dCor = .125, p = .015
Δ dCor	Δ dCor = .028, 95% BCa CI = [.015, .062]

Note: Separate tests for dCor (sleep disturbances → counterproductive work behaviours) and dCor (counterproductive work behaviours → sleep disturbances) are conducted using permutation tests with 5000 resamples. ΔdCor = dCor (sleep disturbances → counterproductive work behaviours) – dCor (counterproductive work behaviours → sleep disturbances). The values of the confidence intervals (CIs) are provided using the 95% BCa bootstrap CI, based on 5000 resamples.

For demonstration purposes, we also examined the differences in dCors between the two competing models (i.e., ΔdCor = dCor[sleep disturbances → counterproductive work behaviours] – dCor[counterproductive work behaviours → sleep disturbances]). The results indicated that ΔdCor >0, and the 95% BCa bootstrap confidence interval (based on 5000 resamples) did not include zero. The ΔdCors test further supports the causal model of the form counterproductive work behaviours → sleep disturbances. For the final causal model, the distance correlation dCor(counterproductive work behaviours → sleep disturbances) obtained from analysis was small and fell below the .10 cut‐off. This provides additional support that the findings from the current analysis are robust to unobserved confounders. In summary, by applying the proposed two‐step algorithm to empirical data, we add to the evidence supporting that counterproductive work behaviours are more likely to cause sleep disturbances than vice versa. Our causal finding is in line with substantive theories and results from previous literature (Yuan et al., 2018).

8. DISCUSSION AND CONCLUSION

8.1. Summary of major findings

This study introduced an independence‐based approach for causal discovery within linear non‐Gaussian models for psychologists. We proposed a two‐step algorithm based on the distance correlation (dCor) for detecting the causal direction between two variables using cross‐sectional data. The performance of the proposed algorithm was evaluated under realistic data conditions (i.e., when unobserved confounders are present) using Monte‐Carlo simulation studies.

Results from Study I showed that the proposed algorithm can effectively detect the true causal direction between two variables, even under conditions of unobserved confounding. The algorithm's performance improved as the effect size of the true causal path increased, the sample size increased and the level of the unobserved confounder became minor. In general, when the true causal effect between the two variables of interest is of medium or large size, the proposed algorithm can be safely applied to determine its causal direction, achieving high hit rates >80%, except in cases where a major confounder is omitted.

When the causal path of interest is small in magnitude, high hit rates for discovering the true causal direction can be achieved only when the sample size is moderate to large (i.e., N ≥ 500), and unobserved confounding effects are minor. Researchers are advised to exercise caution when dealing with a small‐sized causal path, as it may result in high error rates (e.g., >50%) when a strong confounding variable is omitted (especially when confounding is negative). When drawing causal conclusions with the proposed algorithm, applied researchers need to consider and justify the potential presence and degree of unobserved confounding.

In Study II, we further investigated potential indicators of the magnitude of hidden confounding effects. By examining the relationship between the distance correlation coefficients and the level of hidden confounding effect, we proposed additional model selection criteria with empirical cut‐off values to improve the accuracy of the proposed algorithm (i.e., min[dCor] ≤ .10). Study results indicated that using the additional model selection criteria, researchers can draw more cautious conclusions when encountering most undesirable data conditions, such as small‐sized causal paths together with strong hidden confounders. The trade‐off of using the additional cut‐off criterion is a reduced hit rate for detecting true causal directions under favourable conditions, such as when the x → y path has a medium‐to‐large effect size, leading to more inconclusive decisions. Therefore, the additional selection criteria should be considered a conservative approach that is useful in scenarios (1) where researchers do not have any theory or prior knowledge regarding the presence or the level of hidden confounders, and/or (2) where researchers aim to minimize the likelihood of drawing incorrect conclusions based on the purpose of their studies.

In Study III, we assessed the generalizability of findings through additional simulations using a random factor design. The simulation conditions were randomly generated based on the following factors: (1) the shape of the distribution and the level of non‐normality of the confounder, (2) the shape of the distribution and the level of non‐normality of the error term, (3) the effect size of the true causal path and (4) the levels of the hidden confounding effect. Results revealed that the proposed algorithm can be applied under a wide range of plausibly encountered data scenarios.

Finally, we demonstrated the usage of the proposed algorithm in a psychological study. In a real‐world data example, the proposed algorithm confirmed that counterproductive work behaviours can be conceptualized as a causal precursor of sleep disturbances, a finding that aligns with key theories in organizational psychology (Yuan et al., 2018).

8.2. Recommendations for practical applications

Based on the simulation results, the proposed two‐step algorithm can provide useful information about the causal direction between two variables of interest. The performance of the proposed algorithm was validated under conditions applicable to empirical studies in psychology, such as small (N = 200) to large (N = 1000) sample sizes and in the presence of weak to strong hidden confounders. Although the proposed algorithm shows promising results, there are several important caveats when applying it in practice.

First, the proposed algorithm relies on several key assumptions. These assumptions fall into two categories: (1) general assumptions inherent in linear non‐Gaussian models (as listed in Table 4 below), and (2) additional assumptions necessary for making robust conclusions at certain steps of the procedure. Although some assumptions are directly testable, others are challenging to evaluate statistically. All assumptions require justification based on prior knowledge or substantive theories and, when possible, empirical evaluation. In Figure 8, we provide a flowchart with recommended steps and necessary assumptions, guiding applied researchers through the process of implementing the proposed algorithm effectively. We highlight the essential steps together with their key assumptions below.

TABLE 4.

Assumptions for the proposed algorithm.

	Assumptions	Example method(s) of evaluation
General assumptions	1. Non‐normally distributed variables	Tests for skewness and kurtosis; omnibus normality tests (e.g., Kolmogorov–Smirnov test)
	2. Linear relationships	Visual inspection (e.g., scatterplots with locally weighted scatterplot smoothing curves); significance tests for higher‐order polynomials
	3. Homogeneous causal effects	Standard tests for moderation effects (e.g., moderated multiple regression)
	4. No outliers or influential observations	Standard methods for detecting outliers or influential observations in ordinary linear regression (e.g., Cook's distance, leverage)
	5. No measurement error*	Cronbach's alpha/test–retest correlation/inter‐rater agreement
Additional assumptions (for step 2 of the algorithm)	1. A true direct causal path between x and y (i.e., either x → y or y → x)*	Tests based on higher‐order joint cumulants (e.g., Chen et al., 2024)
	2. The influence of the unmeasured confounder(s) is weak*	Examine min[dCor(x → y), dCor(y → x)]

Note: The asterisk indicates assumptions that cannot be tested directly. In such cases, the example methods are useful for providing indirect evidence or rely on tests with additional assumptions.

First of all, researchers need to carefully consider the causal structure between the two variables of interest (i.e., x and y) and try to identify all possible confounders of the variable relation. All observed confounding variables (z*) should be included in, and properly adjusted for, in subsequent analyses to improve the chance of detecting the true causal direction between x and y. Ideally, researchers should include and control only for confounders between x and y. However, in practice, incorrectly including certain covariates, such as a collider,⁹ can introduce bias when assessing the causal relationship between x and y and may also compromise the performance of the proposed algorithm. Therefore, when implementing the proposed algorithm, the selection of covariates should be guided by substantive knowledge and supported by careful causal justification (e.g., Cinelli et al., 2024; Wysocki et al., 2022). Additionally, under the framework of linear non‐Gaussian models, methodologists have developed algorithms that enable researchers to empirically distinguish between confounders and colliders (Entner et al., 2012; Zhang & Wiedermann, 2024).

As suggested by the reviewers, we further examined the impact of collider bias on determining causal directionality and conducted additional simulations to evaluate the robustness of the proposed algorithm when a collider is controlled for. More detailed discussions and additional simulations are provided in Supplementary Material D. Results indicated that the performance of the proposed algorithm is generally robust under simulated conditions where a collider (c) is included in the analysis as a covariate, except when the collider bias was strong (i.e., when the causal paths from x to c and y to c had large effect sizes, particularly the path from x to c). Furthermore, the additional cut‐off of min[dCors] ≤ .10 significantly reduced error rates under undesirable conditions with strong collider bias.

Next, before applying the proposed algorithm, it is crucial to evaluate the following five general assumptions that are fundamental to linear non‐Gaussian models. Specifically, we assume that the (covariate‐adjusted) variables x and y: (1) are non‐normally distributed, (2) are correlated and exhibit a linear relationship, (3) have a homogeneous effect across subjects, (4) have no outliers or influential observations that can distort higher‐order moments and (5) are free of measurement error (for the ‘true’ outcome, measurement error is subsumed in the corresponding model error). Assumptions 1–4 can be empirically tested, and assumptions 2–5 are often implicitly made for ordinary linear regression models. In Table 4, we provide examples of methods that can be used to test the above assumptions.

If the general assumptions required for linear non‐Gaussian models are met, researchers can then proceed to separately test the distance correlations (dCors) obtained from the two competing causal models. Tests of significance using dCors can be conducted through permutation tests or non‐parametric bootstrapping. Decisions regarding the causal direction between x and y can be made based on specific patterns of significance observed in the separate tests. Results from separate tests can also detect scenarios where unmeasured confounder(s) exist between x and y. Therefore, when researchers have adequate statistical power and conclude that the final causal model is either x → y or y → x, there is no need for additional assumptions regarding influential confounders.

On the contrary, if the results from separate tests (i.e., step 1) fail to determine the causal direction between x and y, it implies that researchers either (1) detect unmeasured confounder(s), leaving the causal direction between x and y undetermined, or (2) reach an inconclusive decision, likely due to a lack of statistical power. To draw more informative causal direction decisions, we recommend that researchers then proceed to step 2, which involves conducting a difference test on ΔdCor = dCor (x → y) − dCor (y → x). Tests on ΔdCor can be conducted using 95% bootstrap confidence intervals. Based on the current simulation results, as well as those from previous studies (Pornprasertmanit & Little, 2012; Wiedermann & von Eye, 2015), the difference tests (ΔdCor) in step 2 allow researchers to determine the causal direction between x and y when (1) separate tests lack statistical power and (2) in the presence of unmeasured confounders, especially if the level of the hidden confounder is weak.

However, as a trade‐off, additional assumptions are required to draw valid conclusions in step 2. Specifically, researchers must assume that if any unmeasured confounder(s) exist, then (1) there is a true causal relationship between x and y beyond the association introduced by the hidden confounder (i.e., either x → y or y → x), and (2) the influence of the unmeasured confounder(s) is weak. It is important to note that these assumptions are not directly testable, as researchers do not have access to the hidden confounding variable(s). In practice, when drawing conclusions from tests in step 2, researchers can base their assumptions on substantive theories, prior knowledge or expert judgement.

We also recognized that, with recent advancements in causal discovery within the framework of linear non‐Gaussian models, the assumption of a direct causal path between x and y can be tested statistically using methods based on higher‐order joint cumulants (Chen et al., 2024). However, the validity of these tests relies on additional assumptions (e.g., the hidden confounder u being non‐normally distributed). Moreover, given the use of higher‐order joint information (e.g., fifth‐order joint cumulants), this statistical test requires very large sample sizes (e.g., N > 10,000).

Regarding the second assumption (i.e., weak confounding effects), we found that in selecting the proper causal model, the smaller dCor value, whether dCor(x → y) or dCor(y → x), provides useful additional insights into the level of unmeasured confounding variable(s). For a more conservative decision aimed at minimizing the chance of erroneous conclusions, researchers can apply additional selection criteria by comparing the values of dCor(x → y) or dCor(y → x) to a pre‐defined cut‐off value, in addition to the decision rule for step 2. Using additional criteria with an empirical cut‐off of .10, conclusions about x → y or y → x become more reliable when dCor(x → y) ≤ .10 or dCor(y → x) ≤ .10, respectively.

In addition, it is crucial to understand that the proposed algorithm is not a cure‐all for establishing the causal relation between variables. Instead, this algorithm is better suited to confirm, supplement, enhance or refine existing substantive theories. When there is an absence of theoretical underpinning, this method may also be used as a preliminary investigative tool to aid in the process of theory development. Regardless of the application scenario, the vital role of integrating human discernment in decision‐making should never be overlooked, especially when dealing with complex causal frameworks.

Finally, two interesting questions were raised during the review process. First, what is the relationship between the proposed algorithm and alternative causal inference methods that can handle unobserved confounders? Second, how would the proposed algorithm perform under reciprocal causal effects? We believe these questions are important and practical for applied researchers to better understand and effectively utilize the proposed method. We discuss these two questions further below.

8.3. Comparison of the proposed approach with other causal inference techniques

We proposed an algorithm that enables researchers to detect the causal direction between two variables in the presence of weak hidden confounders. As pointed out by the anonymous reviewers, methodologists in the field of causal inference have developed various approaches to address the issue of hidden or unobserved confounders, including the traditional instrumental variable method (i.e., the IV method, Angrist et al., 1996; Bastardoz et al., 2023; Maydeu‐Olivares et al., 2020) and instrument‐free methods, such as the Gaussian copula method (Becker et al., 2022; Falkenström et al., 2023; Park & Gupta, 2012).

Notably, the proposed algorithm and the IV/Gaussian copula methods aim to address related but distinct causal questions. Specifically, IV and Gaussian copula methods focus on estimating the magnitude of causal effects. In these methods, the causal directionality between variables of interest is typically assumed known, and the objective is to estimate the causal magnitude of the effect. In contrast, the proposed algorithm falls within the field of causal discovery or causal structure learning (Shimizu, 2019), which aims to identify the causal structure or the directionality of causal relationships between variables.

It is worth noting that estimating the true causal effect (i.e., determining ${\beta }_{\mathit{yx}}$) is sufficient but not necessary to detect the presence of a causal path (i.e., whether ${\beta }_{\mathit{yx}}=0$). In this sense, the IV and Gaussian copula methods can also be applied to detect causal directionality. For example, with the IV method, by estimating the true causal effects of both x → y and y → x, researchers can recover the causal structure between x and y, even allowing for reciprocal causal effects. However, the performance of the proposed algorithm and the IV method are not directly comparable, as the IV method addresses more general causal questions (e.g., determining ${\beta }_{\mathit{yx}}$ and ${\beta }_{\mathit{xy}}$) and relies on a different, and arguably stronger, set of assumptions (i.e., two valid IVs: one for estimating x → y and one for y → x). Future studies are expected to further elucidate the differences among various causal methods, examine their aims and assumptions in detail and offer practical guidelines for their application across diverse research scenarios.

8.4. Performance of the proposed algorithm in the presence of reciprocal causal effects

In this study, we only consider the scenario where the causal relationship between the two variables of interest is not reciprocal. Given our focus on cross‐sectional data, this perspective appears reasonable, as methodologists have argued that testing a reciprocal relationship with cross‐sectional data could be considered inappropriate (Hunter & Gerbing, 1982; Wong & Law, 1999). However, testing a reciprocal relationship with cross‐sectional data is mathematically possible, and some researchers have highlighted its merits (Berry, 1984; Finkel, 1995; Wong & Law, 1999). We further discuss the behaviour of the proposed algorithm in the presence of reciprocal causal effects.

As discussed earlier, mathematically, ignoring a reciprocal causal path can be viewed as a special case of omitting a confounder. Thus, when fitting linear non‐Gaussian models in the presence of a reciprocal causal effect, the errors and predictors of both causally competing models (x → y and y → x) are expected to be asymptotically non‐independent. This implies that by applying the proposed algorithm, no conclusion regarding the causal directionality can be drawn from step 1 (i.e., separate tests). Moreover, during the difference test at step 2, the proposed algorithm concentrates on selecting between the two causally competing models. By design, the proposed algorithm is not intended to detect the coexistence of both causal paths (x → y and y → x). Instead, by comparing the difference between dCor(x → y) and dCor(y → x), the proposed algorithm can be expected to identify the causal directionality with the more dominant causal effect. Even though the proposed algorithm is not designed to detect both causal paths in the presence of a reciprocal relationship, conceptually, identifying the existence of a causal path with a more dominant causal effect is not an incorrect conclusion. For this reason, we did not formally include the absence of reciprocal causal effects as an assumption for the validity of the proposed algorithm.

To further verify this conjecture, we conducted additional simulation experiments to evaluate the performance of the proposed algorithm in the presence of reciprocal causal effects. More details and results of these additional simulations are reported in Supplementary Material E in Data S1. The results showed that in the presence of reciprocal causal paths, the proposed algorithm tended to draw conclusions about causal directionality based on the more dominant causal path (i.e., the path with a greater standardized causal effect).

8.5. Limitations and future directions

There are several promising directions that future researchers are encouraged to work on in this area. First, regarding measures of independence, we concentrated on the dCor measure as a natural extension of the linear Pearson correlation coefficient to higher‐order associations that can be interpreted as a standardized measure between 0 and 1. However, many other indices of non‐linear dependency exist, including measures of mutual information (Shimizu, 2022), non‐linear correlation tests (see Hyvärinen et al., 2001) and the Hilbert–Schmidt independence criterion (HSIC, Gretton et al., 2008). It would be interesting to extend the current algorithm by replacing dCor with one of these alternative measures of dependency to examine and compare their performance.

Second, within the framework of linear non‐Gaussian models, we examined the detection of causal directionality between two continuous non‐normally distributed variables. The performance of the proposed algorithm was evaluated under various continuous non‐normal distributions (e.g., Gamma and Johnson distributions). Future studies may expand these findings to other families of distributions (von Eye & Wiedermann, 2021) and investigate approaches to causal discovery involving categorical variables. For example, with binary data, the proposed two‐step algorithm can be applied under linear probability models. However, the independent patterns between predictors and error terms become more complex. Because error terms in linear probability models are inherently heteroscedastic (Aldrich, 1984), independence tests reflect not only the mis‐specification of causal directionality but also the presence of heteroscedasticity. Further research should explore the theoretical underpinnings and focus on developing and evaluating algorithms for causal discovery under categorical data.

Finally, when determining the direction of causality from the difference test (i.e., step 2), it is important to assume that any hidden confounding effects are weak. Although this assumption cannot be directly tested, the extent of hidden confounding may be inferred from the values of min[dCors] derived from the two competing causal models. To reduce the likelihood of erroneous decisions, we recommend incorporating an additional selection criterion with a heuristic cut‐off of min[dCors] ≤ .10. The use of such cut‐off values proved useful in our simulation experiments. Because dCors are standardized between 0 and 1, this conventional cut‐off may be applicable under more general conditions. Moreover, researchers can adjust the level of conservatism to align with their specific research goals. For instance, thresholds <.10 may be adopted, though this approach comes at the expense of reduced statistical power. Future studies should explore the relationship between the extent of hidden confounding and dCor values to provide more robust and informative recommendations for applied researchers.

AUTHOR CONTRIBUTIONS

Dexin Shi: Conceptualization; investigation; writing – original draft; writing – review and editing; methodology; formal analysis; project administration; software; visualization; validation. Bo Zhang: Writing – review and editing; data curation; investigation; resources; validation; methodology. Wolfgang Wiedermann: Software; writing – review and editing; investigation; methodology; conceptualization. Amanda J. Fairchild: Investigation; writing – review and editing; methodology; validation.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

DISCLOSURE OF ARTIFICIAL INTELLIGENCE–GENERATED CONTENT(AIGC) TOOLS

The authors produced, and take full responsibility for, all written content. Note that an AIGC tool (ChatGPT‐4o) was used for several grammatical checks of language.

Supporting information

Data S1

Shi, D. , Zhang, B. , Wiedermann, W. , & Fairchild, A. J. (2025). Distinguishing cause from effect in psychological research: An independence‐based approach under linear non‐Gaussian models. British Journal of Mathematical and Statistical Psychology, 78, 965–995. 10.1111/bmsp.12391

DATA AVAILABILITY STATEMENT

The original dataset can be accessed at https://osf.io/bjfdc/?view_only=5942590d234d458aba06eaff75167955 (Zhang et al., 2022). Data and analysis script used for the empirical example are available at https://osf.io/ef4z7/?view_only=e2af3692cb1c4fcfaa34231e8eb252f5.