Summary:
Autism Spectrum Disorder (ASD) is a neurodevelopmental condition associated with difficulties with social interactions, communication, and restricted or repetitive behaviors. To characterize ASD, investigators often use functional connectivity derived from resting-state functional magnetic resonance imaging of the brain. However, participants’ head motion during the scanning session can induce motion artifacts. Many studies remove participants with excessive motion, and then estimate the effect of diagnosis on functional connectivity using linear regression. However, participant exclusions and linearity assumptions can cause biases. We propose an estimand that quantifies the difference in average functional connectivity in autistic and non-ASD children while standardizing motion relative to the low motion distribution in scans that pass motion quality control. We introduce a nonparametric estimator for motion control, called MoCo, that uses all participants and flexibly models the impacts of motion and other relevant features using an ensemble of machine learning methods. We establish large-sample efficiency and multiple robustness of our proposed estimator. The framework is applied to estimate the difference in functional connectivity between 132 autistic and 245 non-ASD children, of which 34 and 126 pass motion quality control, respectively. MoCo appears to dramatically reduce motion artifacts compared to a standard approach with no participant removal, while more efficiently utilizing participant data and accounting for possible selection biases compared to participant removal.
Keywords: neuroimaging, nonparametric efficiency theory, resting-state fMRI, selection bias, stochastic intervention
1. Introduction
Early studies on neurodevelopment using functional magnetic resonance imaging found that short-range brain connections weakened and long-range brain connections strengthened during development. However, the validity of these findings was undermined by the discovery that motion during imaging can lead to these same patterns (Van Dijk et al., 2012; Power et al., 2014). This discovery led to the widespread adoption of motion quality control via participant removal, which can result in drastic data loss. A recent study removed 60% of approximately 11,500 children due to excessive motion (Marek et al., 2022). Removal of these children not only greatly decreases sample size, but also may introduce selection bias (Cosgrove et al., 2022). This is especially true for studies of neurodevelopmental conditions, such as autism spectrum disorder (ASD). Nebel et al. (2022) found that 80% of autistic children compared to 60% non-ASD children were removed during quality control, and the removed autistic children had greater social deficits, worse motor control, and lower generalized ability index. The authors concluded that differential removal of scans may significantly bias results. These studies point to a need to develop efficient statistical methods that can avoid selection bias in order to draw unbiased inferences about brain development.
The current study is motivated by studies of ASD, where investigators often use resting-state functional magnetic resonance imaging (rs-fMRI) to derive measures of functional connectivity between regions in the brain. Functional connectivity is commonly defined as the correlation between the blood oxygen level dependent signal of different brain regions across time. Functional connectivity may be atypical in autism (Di Martino et al., 2014). However, obtaining high-quality rs-fMRI data for functional connectivity analysis is challenging. Participants’ head motion during the scanning session can induce motion artifacts. The patterns of correlation induced by motion artifacts mimic the connectivity theory of autism, which predicts increased correlations between nearby brain regions and decreased correlations between distant brain areas (Deen and Pelphrey, 2012). Artifact-driven disruptions in brain networks can arise in comparisons of high and low motion rs-fMRI scans (Power et al., 2014).
Current guidelines for analyzing rs-fMRI involve four steps. First, rigid body motion correction is used to align fMRI volumes across time. Second, it is generally recommended to remove individuals in which motion is deemed unacceptable, e.g., remove individuals if they have less than five minutes of data free from excessive motion (Power et al., 2014). Third, confound regression is applied to the time series, which may include regressing motion alignment parameters, global signal, cerebral spinal fluid signal, and white matter, which may be combined with the removal of high-motion volumes or spike regression (Ciric et al., 2017). Following the confound regression, a single measure of functional connectivity is derived for each pair of brain regions summarizing the connectivity between the two regions. In the fourth and final step, the association of diagnosis with the connectivity between two regions is estimated via a linear regression. This regression model includes all children who pass motion quality control and regresses the derived functional connectivity outcome onto a group indicator (e.g., ASD vs. no ASD) while adjusting for certain participant-level variables possibly including a measure of average motion during the scanning session.
There are several shortcomings of current guidelines for the analysis of rs-fMRI data. First, participant removal may be inefficient and lead to selection bias (Nebel et al., 2022). Second, the reliance on linear models may yield inferences that lack robustness due to model misspecification. Nebel et al. (2022) addressed the issue of selection bias by treating the excluded participants as missing data. They used a doubly robust method from causal inference (Benkeser and van der Laan, 2016) that improves upon inverse propensity weighting (Petersen et al., 2024) to correct for potential bias. However, their approach did not leverage information from the fMRI data in excluded participants.
The objective in this paper is to define an estimand (and estimators thereof) that appropriately quantifies the association between functional connectivity and ASD diagnosis that: (i) minimizes the impact of selection bias from motion quality control, (ii) does not rely on correct specification of a linear model, and (iii) provides guidance on which covariates should be adjusted for and how we should adjust for them. Our proposed estimand has connections to direct standardization (Rothman, 2008) and certain estimands used in causal inference (Díaz et al., 2021); however, we do not require counterfactuals to define our estimand. Nevertheless, by connecting the estimand to these areas, we are able to provide insight into whether and how to adjust for certain participant-level characteristics. Our proposed estimators can utilize nonparametric learning approaches (e.g., based on machine learning), while maintaining standard asymptotic behavior. We also show that the estimators enjoy a desirable multiple robustness property. Our approach to motion control, which we call MoCo, is a novel solution to the significant challenges associated with the analysis of rs-fMRI data. We illustrate the MoCo approach via an analysis of functional connectivity between a seed region in the default mode network and other brain regions in children in the Autism Brain Imaging Data Exchange (Di Martino et al., 2017).
2. Methods
Notation. Let denote the diagnosis group, which is equal to 1 if the participant has ASD and 0 otherwise. Let denote the motion variable. In our data application, we take to be mean framewise displacement (FD). FD quantifies head motion between consecutive fMRI frames; each participant’s scan yields approximately 120–180 frames over 5.5–6.5 minutes, and mean FD is a commonly used summary measure of motion during resting-state fMRI (Di Martino et al., 2014; Power et al., 2014). Let denote an inclusion indicator, which is equal to 1 if the participant meets a pre-specified set of criteria for inclusion in the study, related to the aggregate amount of movement during a child’s scanning session. In our data application, we use the criteria from Power et al. (2014), in which if a child has more than 5 minutes of data after removing frames with FD > 0.2 mm. Let denote the functional connectivity between two locations in the brain. For clarity, we initially define for a pair of regions, but in Section 2.4 we extend to the multivariate case with appropriate family-wise error control. Let denote covariates that are putatively related to functional connectivity and are possibly imbalanced across diagnosis groups. Such covariates could include age, sex, and handedness. Let be variables related to the diagnosis group and the pathophysiology of ASD that could possibly contribute to children moving more/less during a scanning session and that have substantially different distributions with little or no overlapping support in ASD and non-ASD groups.
The distinction between and is an important scientific decision, as these sets of variables serve distinct roles in defining our estimand and deriving estimators thereof. In general, we wish to include in any variables that would be balanced across diagnosis groups in an ideal study, but that may be imbalanced due to imperfect recruitment. Thus, variables such as age, sex, and handedness may be important to consider as components of . Each of these variables may biologically relate to connectivity in the brain; however, these biological impacts are not of direct scientific interest and we simply wish to control for any differences in the distribution of these variables across diagnosis groups. On the other hand, we wish to include in any variables that may be related to ASD diagnosis that may also be associated with motion during a scanning session. Such variables could include, for example, a child’s score on the autism diagnostic observation schedule (ADOS, a measure of social disability) and/or their full-scale intelligence quotient score (FIQ, a measure of intelligence). Note that for these variables either (i) we could not balance them by design (e.g., ADOS) or (ii) we would not wish to balance them by design (e.g., FIQ as balancing intelligence across groups may decrease differences between diagnosis groups). In our analysis, we include age, sex, and handedness in and ADOS score, FIQ score, stimulant medication status, and non-stimulant medication status in . Previous analyses have controlled for a limited set of variables in linear regression, e.g., age, FIQ, site, and mean FD (Di Martino et al., 2014), while we argue in favor of considering a broader collection of variables in order to appropriately control for motion artifacts.
Let represent a random variable with distribution . Denote as i.i.d. observations of , where . We assume , where is a statistical model for probability distributions on the support of that is nonparametric up to certain positivity conditions that will be defined below.
In our notation, an uppercase letter with no subscript denotes a random variable, an uppercase letter with an index, typically , is an observed value of a random variable, and a lowercase letter indicates a typical realization of the random variable. For example, is a random variable, while is non-random.
Let denote the probability distribution of conditional on evaluated at value . is thus the probability distribution of motion given fixed diagnosis status and covariates , among children who meet the inclusion criteria. We use to denote a density with respect to Lebesgue measure. For simplicity, all subsequent densities are also defined with respect to the Lebesgue measure. We define as the conditional density of given and as the conditional density of given .
Let denote the conditional mean functional connectivity given , , , and let denote the probability that given . We use an -subscript to denote an estimate, e.g., is an estimate of .
2.1. Defining a model-agnostic target parameter for group comparisons in fMRI studies
We propose a framework motivated by limitations in previous studies of developmental and neurological disorders. The prevailing approach (e.g., Di Martino et al. 2014) is to remove participants that fail motion quality control , then fit a linear regression with functional connectivity (, correlation between a seed region and another brain region) as the response variable and autism diagnosis , mean framewise displacement , and a limited set of demographic variables as predictors . Our goal is to address two issues with this approach: first, that the linear model does not adequately capture the effects of motion (Power et al., 2014); and second that removing participants that fail motion quality control introduces selection bias (Nebel et al., 2022). To that end, our approach considers nonparametric adjustment for , and as part of an outcome model . Because our choice of model is nonparametric, is an infinite-dimensional function of , and . However, our primary interest lies in assessing differences in functional connectivity outcomes between diagnosis groups . Thus, we choose to standardize over particular distributions to yield a motion- and covariate-controlled estimand.
In an ideal world, we would explicitly set motion equal to zero. The ideal estimand of group differences would be , where
| (1) |
Unfortunately, is never truly equal to zero in data applications, as all participants move at least some during a scanning session. Thus, this estimand is not identifiable. We propose to instead consider standardizing motion with respect to a given “tolerable motion distribution” and suggest that a natural choice is , the -conditional distribution of motion observed in non-ASD children that pass motion quality control. Then we define the Motion Controlled (MoCo) estimand of group differences, , where
| (2) |
In this estimand, we average values over their conditional distribution given and , . We average motion values over their conditional distribution given , , and , . Finally, we average covariates over their marginal distribution . Relative to existing approaches, our approach explicitly (i) averages over a tolerable distribution of motion and (ii) adjusts for . We provide motivation for these choices below.
Remark. The gap between the ideal estimand and target estimand is . In some semiparametric and parametric models, this gap is equal to zero. For example, if , we have . In contrast, if there is an interaction between motion and diagnosis, or between symptom severity and diagnosis, then the gap is non-zero. For example, if , then the gap is equal to . If there is no interaction between diagnosis and motion and no interaction between symptom severity and motion, the ideal estimand and MoCo coincide. Specifically, when the outcome model admits the decomposition for functions and , then the gap is zero. The gap is in general not equal to zero if the motion artifact is modified by diagnosis. See Supplement Section 1 for additional details. It seems biologically plausible to assume , since if two children have identical motion in the scanner, we expect the motion artifacts to be equivalent regardless of their diagnosis. In this sense, the MoCo estimand is a reasonable approximation to the ideal estimand.
Justification for tolerable motion distribution. In our application, the distribution of motion in children that pass motion quality control differs between diagnostic groups (Figure 1), with the non-ASD group demonstrating lower marginal motion than the ASD group. Allowing the tolerable motion distribution to depend on adheres to our principle that differences in across diagnosis groups are not of primary interest in our analysis. Thus, marginalizing both diagnosis groups with respect to the same tolerable motion distribution ensures that any residual artifacts attributable to motion within the tolerable range are appropriately balanced across diagnosis groups.
Figure 1:

Distributions of mean framewise displacement (FD) in the school-age children dataset. Panel A shows the distribution of mean FD over all children. Panel B shows the distribution of mean FD over children who meet the inclusion criteria. The distribution of motion in non-ASD children that pass motion quality control differs from the distribution of motion in children with ASD that pass motion quality control.
Justification for adjusting for . Consider an outcome model that omits from its formulation, while adjusting only for , , and . We argue that the fact that this estimand ignores would lead to undesirable consequences and potentially biased analyses. If we were to adopt the same standardizing approach based on , we might consider the parameter , which by the law of total probability equals
| (3) |
This view indicates that excluding may implicitly yield residual motion artifacts within the tolerable range that are imbalanced within a diagnosis group across levels of . To understand why this may be undesirable, note that in children with neurodevelopmental challenges , more severe symptomatology (as may be indicated by components of ) is associated with higher motion (Cosgrove et al., 2022; Nebel et al., 2022). Thus, considering (3) with , we find that the weights formed from the product of densities will tend to place more weight on children with severe symptoms and high motion than they place on children with severe symptoms and low motion. This is illustrated in Supplement Section 2. On the other hand, our proposed estimand uses to standardize over the distribution of . By removing the dependence of this distribution on motion, we are able to appropriately balance motion artifacts within each diagnosis group across levels of .
The motion-controlled estimand is well-defined and nonparametrically estimable if:
(A1.1) for every such that , we also have for ; for every such that , we also have .
(A1.2) for every such that , we also have that for .
(A1.1) states that, at a population level, there cannot be values of that are observed exclusively in the ASD group or exclusively in the non-ASD group and that there cannot be values of that exclusively lead to non-usable scans in the non-ASD group. In our application, consists of age, sex, and handedness, which do not perfectly predict ASD nor scan usability, and therefore assumption (A1.1) is plausible. (A1.2) stipulates that for both , the conditional mean must be well defined for every term in the integrand (2) that is given non-zero weight by the product of the densities . Thus, we require that within each diagnosis group , for any demographic variable value in the marginal support of , any behavioral variable value in the support of for that diagnosis group, and for any motion on the support of the , it must be possible to observe the motion value for all values of that are observed in with . This assumption would be violated for example if included a measure of social disability and children in the ASD group with the highest levels of social disability never generate motion values comparable to the the motion values observed in the non-ASD group.
In the Supplement Section 3, we further describe connections between our estimand and those used in causal inference and include a causal graph illustrating relationships between the various variables used in our analysis.
2.2. Efficiency theory
A key step in developing our estimator is deriving the efficient influence function (EIF) of regular, asymptotically linear estimators of . See Supplement Section 4.1 for a short review of efficiency theory. To characterize this EIF, we define as the probability of a non-ASD child with covariate value having usable data. We introduce the shorthand as the probability that and conditional on . We denote the indicator function equal to 1 if and zero otherwise; equal to 1 if and and equals zero otherwise. We also define for
| (4) |
| (5) |
| (6) |
In these definitions, we use a subscript notation for the functional parameters and that attempts to make explicit both the integrand in the parameter’s definition, as well as the random variables that are arguments of the function. For example, the definition of (5) involves integrating , while is a function of the random variables appearing in the subscript, , and .
Theorem 1: (Efficient Influence Function). In a nonparametric model, the efficient influence function for evaluated on a typical observation is defined as
| (7) |
A proof is included in the Supplement Section 4.2. Fubini’s theorem allows us to write in terms of either or , , where .
We use the one-step estimation framework to define efficient estimators of (Bickel et al., 1993). Suppose we have an estimate of available, say . An estimate of can be obtained by marginalizing over the empirical distribution of , yielding the plug-in estimate . A one-step estimator of can be constructed as , where is an estimate of . Thus, to construct a one-step estimate of , we require as an intermediate step estimates of the various parameters of that appear in . We refer to these quantities as nuisance parameters, parameters that need to be estimated as an intermediate step in the estimation of .
Examining Theorem 1, we find several nuisance parameters in for which we will require estimates. Estimation of several of these parameters is straightforward. For example, could be estimated using mean regression of on . On the other hand, the and parameters involve integration and conditional densities, which generally present challenges in implementation. Our approach emphasizes two key points: (i) wherever possible mean regression with pseudo-outcomes is used to avoid numeric integration and conditional density estimation and (ii) flexible estimation techniques are used.
We choose to emphasize the use of mean regression because it is a technique familiar to many applied statisticians and there are widely available tools. In our application, we focus on a flexible framework for regression, known as regression stacking or super learning (van der Laan et al., 2007). Super learning uses cross-validation to build a weighted combination of candidate regression estimators, with large sample theory indicating that the ensemble estimator is essentially as good or better than any of the individual candidate regressions considered. Unfortunately, mean regression cannot be used exclusively in the estimation of and we require estimates of conditional motion distributions described below. For this purpose, we utilize a version of the highly adaptive lasso (HAL) specifically tailored for conditional density estimation, as implemented in the haldensify R package (Hejazi et al., 2022). To circumvent numerical integration in our estimation, we make use of a technique proposed by Díaz et al. (2021) that re-casts these estimation problems that involve integrals and densities as an estimation problem that can be solved using mean regression with pseudo-outcomes. A detailed description of the implementation of our estimator is included in Supplement Section 5.2.
2.3. Inference
Below we present two theorems establishing the consistency and asymptotic linearity, respectively, of the one-step estimator . We define to be the norm of a given function defined as . We note that for the purposes of this definition, the function is treated as given, even if it involves estimated quantities. Theorem 2 assumes:
(B1) Boundedness: is bounded below by some , is bounded below by some , and is bounded below by some .
(B2) -convergence of certain combinations of nuisance parameters: certain subsets of the nuisance parameters are consistently estimated, as described in Table 1.
Table 1:
Assumption (B2) of Theorem 3.2 (multiple robustness). Each row indicates a setting for consistency, where check marks indicate the nuisance parameters which, when they converge to true functions combined with assumptions (B1), (B3) and (B4), result in the consistency of .
| (B2.1) | ✓ | ✓ | ✓ | ||||
| (B2.2) | ✓ | ✓ | ✓ | ||||
| (B2.3) | ✓ | ✓ | ✓ | ✓ | |||
| (B2.4) | ✓ | ✓ | ✓ | ||||
| (B2.5) | ✓ | ✓ | ✓ |
(B3) -consistent influence function estimate: , where denotes the in-probability limit of as approaches infinity and is treated as a fixed function of in this expression.
(B4) Glivenko Cantelli influence function estimate: the probability that falls in a -Glivenko Cantelli class tends to one as .
Assumption (B1) guarantees that estimated propensities and motion densities are appropriately bounded so that the one-step estimator is never ill-defined. Assumption (B2) stipulates consistent estimations of the nuisance parameters. Assumptions (B3) and (B4) are necessary to ensure the negligibility of an empirical process term.
Theorem 2: (Multiple robustness). Under (B1) - (B4), .
According to Theorem 2, our one-step estimators will only require some of the nuisance parameters to be consistently estimated to achieve consistency of . For example, (B2.1) implies that obtaining consistent estimates of and , and is sufficient to ensure a consistency of . For a proof, see Supplement Section 5.
Theorem 3: (Asymptotic linearity). Under (B1), (B3), and
(C1) -convergence of second order terms: , and .
(C2) Donsker estimates: falls in a P-Donsker class with probability → 1 as . then , and .
Assumption (C1) states that nuisance estimates converge to their true values at a sufficiently fast rate, while (C2) ensures large-sample negligibility of a certain second-order empirical process term (so-called Donsker conditions.(C2) can be eliminated through the use of cross-fitting (Supplement Section 7). We study the benefits of this approach in our simulation. For further discussion of assumptions and a proof of the theorem, see Supplement Section 8.
When all nuisance regressions are consistently estimated, provides a consistent estimate of . Thus, an asymptotically justified confidence interval for is , where denotes the -quantile of a standard Normal distribution. Similarly, by Theorem 3 implies the limiting distribution of is , with . The estimate will be consistent for and can be used to generate a confidence interval for the association of ASD with brain connectivity in a single brain region, .
2.4. Simultaneous inference for associations
To control family-wise error rate across hundreds of regions, we conduct testing using simultaneous confidence bands. Let index the region. In our application, as we examine the association between a seed region and 399 other regions. Let denote the MoCo estimand in group and region , and let denote its estimate. Let denote the EIF for diagnosis group and region , and let denote the region-specific estimate of the asymptotic variance. By Theorem 3,
| (8) |
An approximate simultaneous confidence interval is , where is the quantile of , which depends on the covariance matrix in (8).
To approximate , Monte-Carlo integration is performed by taking 105 independent draws of a mean-zero -variate normal random variable with covariance equal to the sample correlation matrix of the vectors , . For each of the draws, the maximal absolute value of the components of the vector is calculated. The critical value is approximated by the -quantile. Wald hypothesis tests controlling family-wise error rate at level are conducted by rejecting the null hypothesis of no association between diagnosis group and functional connectivity in the -th region whenever is larger than the estimated value of .
3. Simulation study
To mirror our data analysis, we simulated 1000 datasets with 400 children. Details are in Supplement Section 9.2. Briefly, for , we simulated values for age, sex and handedness with marginal distributions similar to the ABIDE dataset. For , we simulated values for autism diagnostic observation schedule (ADOS), full-scale IQ (FIQ), indicator of stimulant usage and indicator of other medication usage. For given , and , we simulated a conditional log-normal distribution. We simulated functional connectivity between a seed region equal to the default mode network and six other resting-state parcels using linear models with no interactions between diagnosis and motion or between symptoms and motion. In this design, the ideal estimand is equal to MoCo (see Section 2.1). For , we set the coefficients for , and equal to 0. For and , coefficients were selected to result in large and small negative associations existed between the diagnosis group and and , respectively. For and , our data generating process also included quadratic associations between motion and observed functional connectivity to examine the ability of Super Learner to account for possible non-linear relationships. For example, . Other formulas are in the Supplement Section 9.2.
We compared MoCo to four approaches. 1) The naïve approach that removes high-motion participants, which targets the estimand . 2) The naïve approach that does not remove any participants, which targets . 3) Inverse Probability of Treatment Weighting (IPTW), which targets . 4) The method proposed by Nebel et al. (2022), which regresses against and and subsequently uses the residuals as input to doubly robust targeted minimum loss based estimation in which the removed high-motion scans are treated as missing data. With the preprocessing step, it targets . Further details can be found in Supplement Section 9.1. Arguably, the target estimands for both IPTW and Nebel’s method are approximations to the ideal estimand. Our simulation results reflect the gaps between the target and ideal estimands for respective methods in addition to estimation error.
MoCo demonstrated advantages in terms of bias, MSE, type I error, and power relative to other approaches (Table 2). It had the lowest type I error in regions 1 to 4, and lower bias than the naïve methods and IPTW. MoCo achieved the lowest MSE in three of four zero-association regions and, in regions with true associations, demonstrated greater power and lower bias than all other methods. Nebel’s method was accurate in regions 1 to 4 in which motion impacts were linear, but exhibited substantial bias in regions 5 and 6 in which motion effects were nonlinear. In contrast, MoCo effectively captured both linear and nonlinear relationships, making it more robust across scenarios. Figure 2 illustrates the results of MoCo with cross-fitting on one simulated dataset and demonstrates its performance.
Table 2:
Simulation results comparing MoCo, the naïve approach with participant removal, the naïve approach including all participants, IPTW, and Nebel et al. (2022)’s method. Bolded values indicate the lowest bias, standard deviation, MSE, and Type I error, and the highest power across methods.
| Truth | Metric | MoCo | Naïve removal | Naïve | IPTW | Nebel | |
|---|---|---|---|---|---|---|---|
| Region 1 | 0.0000 | Bias | 0.0005 | −0.0190 | −0.0644 | −0.0107 | 0.0002 |
| SD | 0.0372 | 0.0191 | 0.0204 | 0.0232 | 0.0193 | ||
| 1.3839 | 0.7283 | 4.5657 | 0.6441 | 0.3720 | |||
| Type I Error | 0.0110 | 0.1070 | 0.8670 | 0.0610 | 0.0740 | ||
| Region 2 | 0.0000 | Bias | 0.0048 | 0.0176 | 0.0611 | 0.0097 | 0.0004 |
| SD | 0.0238 | 0.0234 | 0.0221 | 0.0282 | 0.0289 | ||
| 0.5894 | 0.8576 | 4.2200 | 0.6456 | 0.8359 | |||
| Type I Error | 0.0100 | 0.0730 | 0.7220 | 0.0610 | 0.0910 | ||
| Region 3 | 0.0000 | Bias | 0.0044 | 0.0153 | 0.0553 | 0.0088 | −0.0002 |
| SD | 0.0183 | 0.0179 | 0.0182 | 0.0223 | 0.0211 | ||
| 0.3554 | 0.5542 | 3.3941 | 0.6464 | 0.4441 | |||
| Type I Error | 0.0080 | 0.0680 | 0.8320 | 0.0690 | 0.0840 | ||
| Region 4 | 0.0000 | Bias | −0.0034 | −0.0179 | −0.0663 | −0.0105 | −0.0001 |
| SD | 0.0204 | 0.0199 | 0.0204 | 0.0234 | 0.0208 | ||
| 0.4275 | 0.7180 | 4.8182 | 0.6456 | 0.4312 | |||
| Type I Error | 0.0100 | 0.1160 | 0.8870 | 0.0870 | 0.0880 | ||
| Region 5 | −0.0484 | Bias | 0.0065 | 0.0213 | 0.0695 | 0.0165 | 0.0312 |
| SD | 0.0214 | 0.0208 | 0.0212 | 0.0250 | 0.0264 | ||
| 0.4990 | 0.8848 | 5.2748 | 1.6316 | 1.6748 | |||
| Power | 0.3790 | 0.1700 | 0.1260 | 0.2410 | 0.2272 | ||
| Region 6 | −0.0682 | Bias | 0.0063 | 0.0241 | 0.0796 | 0.0186 | 0.0517 |
| SD | 0.0203 | 0.0178 | 0.0214 | 0.0220 | 0.0261 | ||
| 0.4523 | 0.8979 | 6.7937 | 3.3802 | 3.3521 | |||
| Power | 0.8690 | 0.5280 | 0.0670 | 0.6000 | 0.2490 |
Figure 2:

Example from a typical simulated dataset. The true association is marked in dark green and purple, while other regions have zero associations. MoCo identified one of the two true associations correctly. However, the naïve method with participant removal failed to detect either of the two regions with true associations. The naïve method with all data had two false positives and failed to recover either of the regions with true associations. IPTW detected one of the true regions, but it was a biased estimate. Nebel’s method was able to detect one of the two true regions, but its estimate was highly biased because it could not capture the nonlinear relationship between motion and functional connectivity.
Additional simulations demonstrate that (i) MoCo has low bias but also lower power for small sample sizes (Supplement Section 9.3); (ii) treating FIQ as instead of tends to increase MoCo’s MSE, but otherwise has minimal impact (Section 9.4); (iii) that MoCo remains accurate when simulating time series motion that causes correlation between two regions (Section 9.5); and (iv) that MoCo is multiply robust as shown in Theorem 2.
Additional simulation results are available in the Supplement. In Supplement Section 9.3, we show MoCo has low bias but also lower power for and . In Supplement Section 9.4, we show that under the same design described above, treating FIQ as instead of tends to increase MoCo’s MSE, but overall does not have a substantial impact. In Supplement Section 9.5, we simulate a time series of motion that causes correlation between two regions, and we demonstrate that MoCo is accurate in this setting. Supplement Section 9.6 presents additional simulations (n = 50 to 4000) demonstrating multiple robustness.
4. Data analysis of functional connectivity in ASD
We conducted a functional connectivity analysis using a seed region in the default mode network. We applied our method to 377 resting-state fMRI data from children ages 8–13 in the ABIDE dataset (Di Martino et al., 2014, 2017) (Supplement Table 8). Details are described in Supplement Section 10. We compared naïve estimates without participant removal, naïve estimates with removal , IPTW, Nebel’s method, and MoCo with cross-fitting. FWER-critical values were determined using simultaneous confidence intervals (Section 2.4) derived from residual correlations for naïve methods, bootstrap replicates for IPTW, and EIFs for Nebel’s method and MoCo. Nebel’s method and MoCo used the same super learner library as the simulations. To handle variability from cross-validation, we generated estimates 50 times and averaged z-statistics across runs. Positivity assumptions were assessed via histograms of inverse probability weights and density ratios (Supplement Section 10.3). All estimated density ratios were below 4, suggesting the assumptions were reasonably satisfied. Results were visualized using the R package ciftiTools (Pham et al., 2022).
MoCo and the naïve approach use the imaging data from 377 participants, including 132 with ASD, while the naïve approach with participant removal, IPTW, and Nebel’s method use the imaging data from 160 participants, and only 34 with ASD. MoCo reveals four regions that differ in connectivity with the posterior default mode seed region in ASD versus non-ASD at FWER=0.05, including three regions of hyperconnectivity with distant frontal-parietal regions (Figure 3). The naïve approach indicates more extensive differences than MoCo, including prominent default mode hypoconnectivity in ASD in long-distance correlations. These are possibly spurious differences due to motion, as long-distance correlations tend to be attenuated in high-motion participants (Ciric et al., 2017). These possible biases are also prominent in the mean connectivity estimates (Supplement Figure 4). The naïve approach also selects some regions of hyperconnectivity in ASD with lateral regions of the frontal lobe. The naïve approach with participant removal produces more conservative results, and it suffers from substantial loss of power due to reduced sample size. IPTW behaves similarly to the naïve approach with participant removal. Nebel’s method yields the fewest discoveries, possibly reflecting low sensitivity from the small number of children that pass motion quality control. Overall, MoCo detects more regions than the naïve with participant removal, IPTW, and Nebel’s method, while remaining more selective than the naïve approach. This may be due to improved sensitivity relative to methods that do not use all imaging data (naïve with participant removal, IPTW, and Nebel’s method) and improved specificity relative to the naïve approach.
Figure 3:

Z-statistics for the group difference (ASD – non-ASD) for a seed in the posterior default mode network (fuchsia point) in the ABIDE dataset.
5. Discussion
We introduce MoCo, a method for controlling motion in fMRI studies to estimate the difference in functional connectivity between two groups that addresses the selection bias caused by motion quality control exclusion criteria. We use flexible machine-learning techniques for parameter estimation with simultaneous confidence intervals for controlling FWER across hundreds of brain connections. MoCo improves statistical power and lowers type I error rate.
In our data application, our findings differ greatly from the naïve approach including all participants, which suggested hypoconnectivity across many DMN regions. The naïve approach with participant removal suggests these differences were due to motion artifacts, but it is difficult to disentangle this from power loss and selection biases, as only 34 ASD children passed motion quality control. MoCo contributes to the ASD literature by flexibly modeling motion artifacts while including all the phenotypic variability in the study sample, providing stronger evidence that the hypoconnectivity differences were due to motion artifacts. MoCo recovered more regions than the naïve approach with motion removal (four versus two at FWER=0.05), although the overall picture suggests minor differences between autistic and non-ASD children in correlations with the default mode network seed region.
An important decision in the modeling process is to designate variables that could or should have been balanced through careful recruitment versus variables biologically related to diagnosis group . In the data analysis, we consider age, sex, and handedness as . We treat FIQ as a diagnosis-specific variable, which on average was lower in the ASD group. However, FIQ is highly variable in autism, and whether or not it should be considered as a part of or as a part of is debatable. In our dataset, the child with the highest FIQ was also diagnosed with autism (Supplement Table 8). Neural diversity in autism is associated with strengths like unique perspectives, problem-solving skills, intense focus, attention to detail, and other traits that extend beyond a single measure of intelligence.
There are a number of limitations and directions for future research. First, we use machine learning to predict functional connectivity from an overall measure of motion, mean FD, which does not use the time series structure of motion. Future research can investigate the use of machine learning to predict the BOLD time series from the motion alignment parameters, followed by the calculation of the functional connectivity, although this would be computationally demanding. Second, our study fits functional connectivity for a seed-based analysis, rather than simultaneously analyzing the functional connectivity matrix. A matrix-variate approach could be designed to exploit low-rank or sparse structure, which may improve efficiency. Efficiency may also be improved in small samples by considering more stable nuisance parameter estimates, such as those based on low-dimensional, working parametric models. This approach may lead to improvements in the smallest sample sizes, where our methods showed considerable under coverage. Finally, while we focus on rs-fMRI, MoCo could may be effective in other neuroimaging modalities, where motion can similarly induce spurious effects in morphometry and diffusion MRI.
Supplementary Material
Web Appendices, Tables, and Figures referenced in Sections 2–5 and simulations code are available with this paper at the Biometrics website on Oxford Academic. The R package MoCo is available on https://github.com/thebrisklab/MoCo.
Acknowledgments.
We thank Xiyan Tan, Liangkang Wang, and Zihang Wang for assistance with processing and quality control of the ABIDE data.
Funding.
This work was supported by R01 MH129855 (BR, DB).
Data Availability Statement.
The Autism Brain Imaging Data Exchange (ABIDE) data are from https://fcon_1000.projects.nitrc.org/indi/abide/.
References
- Benkeser D and van der Laan M. (2016). The highly adaptive lasso estimator. In 2016 IEEE International Conference on DSAA, pages 689–696. IEEE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickel PJ, Klaassen CA, Ritov Y, and Wellner JA. (1993). Efficient and adaptive estimation for semiparametric models, volume 4. Springer. [Google Scholar]
- Ciric R, Wolf DH, Power JD, et al. (2017). Benchmarking of participant-level confound regression strategies for the control of motion artifact in studies of functional connectivity. Neuroimage 154, 174–187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cosgrove KT, McDermott TJ, White EJ, et al. (2022). Limits to the generalizability of resting-state functional magnetic resonance imaging studies of youth: An examination of abcd study® baseline data. Brain imaging and behavior 16, 1919–1925. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Deen B and Pelphrey K. (2012). Perspective: brain scans need a rethink. Nature 491, S20–S20. [DOI] [PubMed] [Google Scholar]
- Di Martino A, O’connor D, Chen B, et al. (2017). Enhancing studies of the connectome in autism using the autism brain imaging data exchange ii. Scientific data 4, 1–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Di Martino A, Yan C-G, Li Q, et al. (2014). The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Molecular psychiatry 19, 659–667. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Díaz I, Hejazi NS, Rudolph KE, and van der Laan MJ. (2021). Nonparametric efficient causal mediation with intermediate confounders. Biometrika 108, 627–641. [Google Scholar]
- Hejazi NS, van der Laan MJ, and Benkeser D. (2022). haldensify: Highly adaptive lasso conditional density estimation in R. Journal of Open Source Software 7, 4522. [Google Scholar]
- Marek S, Tervo-Clemmens B, Calabro FJ, et al. (2022). Reproducible brain-wide association studies require thousands of individuals. Nature 603, 654–660. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nebel MB, Lidstone DE, Wang L, et al. (2022). Accounting for motion in resting-state fmri: What part of the spectrum are we characterizing in autism spectrum disorder? NeuroImage 257, 119296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Petersen GL, Jørgensen TSH, Mathisen J, et al. (2024). Inverse probability weighting for self-selection bias correction in the investigation of social inequality in mortality. International Journal of Epidemiology 53, 1–7. [DOI] [PubMed] [Google Scholar]
- Pham DD, Muschelli J, and Mejia AF. (2022). ciftitools: A package for reading, writing, visualizing, and manipulating cifti files in r. NeuroImage 250, 118877. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Power JD, Mitra A, Laumann TO, et al. (2014). Methods to detect, characterize, and remove motion artifact in resting state fmri. Neuroimage 84, 320–341. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rothman K. (2008). Modern epidemiology. Lippincott Williams & Wilkins. [Google Scholar]
- van der Laan MJ, Polley EC, and Hubbard AE. (2007). Super learner. U.C. Berkeley Division of Biostatistics Working Paper Series. pages 1–22. [Google Scholar]
- Van Dijk KR, Sabuncu MR, and Buckner RL. (2012). The influence of head motion on intrinsic functional connectivity mri. Neuroimage 59, 431–438. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The Autism Brain Imaging Data Exchange (ABIDE) data are from https://fcon_1000.projects.nitrc.org/indi/abide/.
