A dynamic perspective on clinical treatment evaluation using dynamic structural equation modeling

Abstract

Background

Conventional evaluations of psychological treatment effects have primarily focused on changes in average symptom levels, thereby overlooking intra-individual variability. However, evidence from intensive longitudinal data demonstrates that psychological states fluctuate dynamically over time, suggesting that treatment effects may manifest not only in mean levels but also in temporal instability. It is imperative to acknowledge the impact of variability on treatment efficacy, particularly in the context of clinical intervention research. Failure to account for this variability can lead to the misinterpretation of treatment outcomes, potentially compromising the accuracy of research findings.

Methods

The proposed methodology, under a randomized controlled trial design, is intended to facilitate the dynamic assessment of clinical treatment. The present method is based on a dynamic structural equation model framework, which allows individual-specific residual variances to be contingent on treatment allocation, thereby facilitating the direct evaluation of the impact of treatment on psychological instability. The proposed framework is illustrated with an empirical case, and the performance of dynamic treatment assessment parameters is systematically evaluated under different conditions using Monte Carlo simulations.

Results

Empirical evidence has demonstrated the efficacy of the dynamic structural equation model in capturing variations in dynamic treatment effects between experimental and control groups. The simulation results further indicate that this method provides accurate parameter recovery and stable estimates under various conditions, thereby supporting its robustness and flexibility for simulating dynamic treatment effects in intensive longitudinal data settings.

Conclusions

The present study proposes a novel perspective on dynamic treatment evaluation based on a dynamic structural equation model framework. By conceptualizing residual variance as a dynamic outcome, this framework facilitates a more nuanced comprehension of clinical change and offers a significant extension to conventional treatment-evaluation methods in randomized controlled trials.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-026-02921-1.

Background

In recent years, the widespread use of electronic devices and the rapid development of mobile health technologies have made the collection of intensive longitudinal data (ILD) much more convenient in clinical settings [1]. Compared with traditional longitudinal designs, ILD allows for much more frequent assessments over short periods of time [2]. This enables researchers not only to capture trends in symptoms or momentary experiences but also to observe dynamic fluctuations within individuals (Fig. 1). Such within-person dynamics may indicate substantial changes in treatment adherence or increasing physiological instability. Therefore, clinical evaluations that rely primarily on changes in mean symptom levels may be insufficient to fully capture treatment-related processes.

Fig. 1.

Temporal changes in mood based on intensive longitudinal data (ILD). Individual 4 showed lower intraindividual variability in mood compared to Individual 9

When dynamic processes are considered in clinical research, temporal instability has typically been quantified using summary indices such as overall autoregressive parameters [3] and the mean squared successive difference (MSSD) [4]. These indicators have been widely applied in clinical and medical studies based on ILD, including assessments of affective and behavioral instability [5, 6]. However, temporal instability is not a unidimensional construct. Prior research has shown that instability can be decomposed into at least two distinct but complementary components, namely temporal persistence, and fluctuation magnitude [7, 8]. Temporal persistence is captured by autoregressive dynamics and reflects the extent to which deviations from an individual’s typical level carry over across time points. In contrast, fluctuation magnitude, commonly referred to as amplitude, reflects the intensity of short-term deviations around an individual’s typical level that are not predictable from prior states. Although both components relate to instability, they represent conceptually distinct dimensions with different predictive implications for health outcomes. Specifically, temporal dependency is more strongly associated with chronic health conditions, whereas fluctuation amplitude is more closely related to subjective somatic complaints [8]. Accordingly, amplitude can be regarded as a clinically meaningful form of within-individual variability, and reductions in amplitude, such as attenuated emotional or physiological reactivity, may indicate improved treatment response [9].

However, conventional statistical models generally assume constant residual variance, a limitation that curtails their ability to detect dynamic characteristics. When interpreted dynamically, residual variance represents the amplitude of short-term deviations from an individual’s expected level [4], with larger residual variance corresponding to greater fluctuation intensity. Whereas residual variance captures the magnitude of short-term, unpredictable fluctuations around an individual’s expected level, temporal dependency is governed by autoregressive dynamics, which reflect the degree of carryover from prior states. As demonstrated in Fig. 2, increases in residual variance result in larger amplitudes over time, underscoring the importance of modeling residual variance as an informative outcome to capture clinically relevant dynamics.

Fig. 2.

Simulated ILD data under three levels of residual variance (σₑ = 0.2, 0.5, 1.0). As residual variance increases, the range of observed fluctuations becomes larger, indicating greater amplitude and reflecting increased psychological instability

Dynamic structural equation modeling (DSEM) has been introduced in the field of psychology as a Bayesian hierarchical modeling 1framework for analyzing intensive longitudinal data [10–13]. This approach allows both the mean (location) and residual variance (scale) of outcomes to be modeled simultaneously. This methodological framework facilitates the examination of treatment effects not only on mean levels but also on amplitude, defined as residual variance. Whereas conventional mixed models for repeated measures (MMRM), commonly used in confirmatory clinical trials, primarily target mean trajectories across discrete follow-up visits and treat time as a design factor, DSEM is rooted in a time-series perspective in which dynamic process is explicitly modeled as a core feature of the design.

By linking the log-transformed residual variance to predictors through a structural equation, DSEM offers a flexible hierarchical framework for capturing intra-individual variability and clinical. This hierarchical formulation allows residual variance to be modeled as an outcome, rather than as a fixed nuisance parameter, thereby enabling direct inference on dynamic instability. Although this approach has not yet been widely applied in clinical and medical research, its univariate specification is consistent with the mixed-effects location–scale modeling (MELSM) framework [2]. The integration of time-series analysis with multilevel modeling is a methodological approach employed by DSEM, which enables the capture of both within- and between-individual variability. Modeling residual variance as a dynamic outcome provides a clinically meaningful way to capture the amplitude of treatment effects or moment-to-moment fluctuations in psychological states.

Although modeling residual variance as an index of amplitude and capturing dynamic treatment effects are theoretically relevant and practically feasible, their application remains limited in clinical research. Most conventional randomized controlled trials (RCTs), particularly those that employ infrequently spaced follow-up assessments, predominantly emphasize mean-level treatment effects at predefined primary endpoints. These trials frequently neglect to consider intra-individual variability. Conversely, trials that incorporate intensive longitudinal measurements provide an opportunity to examine dynamic or secondary outcomes reflecting within-person processes.

Recent work on distributional treatment effects [14] demonstrates that interventions can alter outcome variances and other distributional properties even when mean-level effects are modest, underscoring the importance of modeling residual variance to fully characterize treatment effects. Even within DSEM applications, emphasis is usually placed on modeling average trajectories and autoregressive effects, with little attention given to fluctuations in residual variance. Most extant work incorporates random residual variance (RRV) to enhance the accuracy of variance estimation or improve overall model fit [15]. However, there is a paucity of research that examines the substantive and clinical meaning of amplitude (i.e., residual variance) itself.

To address this gap, the present study adopts a DSEM framework to evaluate dynamic treatment effects in clinical settings and proposes residual variance (i.e., amplitude) as a clinically meaningful dynamic or secondary outcome metric in trials with intensive longitudinal designs. The DSEM is demonstrated to be a flexible method for estimating time-varying and group-specific dynamic effects. Empirical evidence shows that modeling changes in amplitude can reveal clinically meaningful dynamic treatment effects that may be overlooked in conventional clinical assessments. To assess the efficacy of the proposed model, a Monte Carlo simulation was employed to evaluate its performance under diverse conditions. This approach was undertaken to ascertain the circumstances under which dynamic treatment effects remain robust. This text provides practical guidelines for applied researchers seeking to use this approach in empirical studies.

Methods

By extending the DSEM and MLSM frameworks to include an RCT predictor at the between-individual scale level, this approach allows researchers not only to examine mean-level treatment effects but also to evaluate differences in amplitude variability between treatment and control groups. Such differences in variability serve as dynamic indicators of treatment effects.

Model specification

The clinical outcome of an individual  i(i=1,...,I) at the measurement time point) is denoted as .

At the within-individual level (Level-1), the model is specified as follows:

1

where is the individual-specific mean level, and the term captures the autoregressive effect, reflecting the extent to which an individual’s previous score or state influences the current score or state. The autoregressive parameter is bounded between − 1 and 1. Values closer to 0 indicate weaker carryover from the previous time point, whereas values with larger absolute magnitude indicate stronger temporal dependence. The residual error term is assumed to be normally distributed, , where a larger value of reflects greater amplitude in the ILD, indicating more pronounced moment-to-moment fluctuations in the psychological outcome.

At the between-individual level (Level 2), both and are modeled based on treatment condition and individual random effects, as follows:

2

3

where represents the fixed effect corresponding to the grand mean across individuals, and captures the conventional treatment effect of interest in RCTs—namely the change in overall mean levels between the treatment and control groups. is a dummy variable (0 = control, 1 = treatment). The random effect reflects individual-specific deviations from the mean, where denotes the variance of these deviations, capturing the degree of heterogeneity in baseline levels across individuals.

Similarly, the autoregressive parameter consists of a fixed effect reflecting the mean temporal dependency across individuals, and a random effect , which captures individual-specific deviations in the strength of autoregression. Here, denotes the variance of these deviations, reflecting the degree of heterogeneity in the autoregressive effect across individuals.

In (Eq. 1), the residual variance is not indexed by individual 𝑖, implying that all individuals are assumed to share a common residual variance. This means that everyone’s behavior is considered equally predictable, and that the amplitude of their moment-to-moment fluctuations is assumed to be similar. However, this assumption is overly restrictive. Individuals exhibit variability in sensitivity to external factors over time. While some individuals demonstrate relatively stable clinical responses following treatment, others continue to exhibit pronounced short-term fluctuations and rapid changes over time, reflecting heterogeneity in treatment effects [16]. Therefore, modeling the residual variance as individual-specific, which is a potentially dynamic parameter, is crucial for accurately characterizing within- individual processes and for obtaining valid estimates of treatment effects.

Equation (4) extends the specification of the residual variance in (Eq. 1) by allowing it to vary across individuals. The model can be expressed as

4

where follows a normal distribution with mean zero and variance . This allows for RRV, which captures between-individual differences in amplitude and is defined as the magnitude of moment-to-moment fluctuations in clinical outcomes. This formulation enables the model to represent heterogeneity more accurately in terms of temporal instability across individuals. In the present specification, the random effects governing individual differences in mean level, temporal dependency, and residual variance were assumed to be uncorrelated, resulting in a diagonal covariance structure. This parsimonious assumption was adopted to focus on treatment effects on each dynamic component separately and to avoid additional complexity that is not central to the primary research questions.

In the multilevel location–scale modeling framework [17, 18], this approach extends the prior specifications of individual differences in mean levels (Eq. 2) and autoregressive effects (Eq. 3) by introducing variability in the scale component—that is, residual variance. Importantly, this framework explicitly models amplitude as a treatment-sensitive parameter. This enables the detection of dynamic treatment effects, defined as the extent to which an intervention reduces not only average symptom levels but also intra-individual variability in clinical outcomes.

At the between-individual level, RRV is modeled as follows:

5

where represents the average log residual variance across individuals, and captures the dynamic treatment effects on residual variance, reflecting the degree to which the intervention alters within-individual fluctuation (amplitude). The variable is a dummy indicator (0 = control; 1 = treatment). The term denotes an individual-specific random effect that accounts for unexplained heterogeneity in amplitude across individuals. Residual variances are modeled on a logarithmic scale to ensure they remain strictly positive. This approach prevents Heywood cases, in which variance estimates may become negative owing to sampling errors or model misspecifications [19]. Equation 5 can also be written as:

6

Although the dynamic treatment effect parameter is estimated on the logarithmic scale, it can be transformed to the original scale via the exponential function. As demonstrated in Fig. 3, the log-scale estimate is centered at zero when comparing the treatment and control groups. Values below zero indicate smaller amplitude and greater symptom stability in the treatment group, whereas values above zero indicate smaller amplitude in the control group and increased fluctuation in the treatment group. On the original scale, this parameter is interpreted as , with 1 serving as the reference value. Illustrative numerical examples are provided in Table 1.

Fig. 3.

Interpretation of log residual variance effects

Table 1.

Log-transformation and interpretation of the dynamic treatment effect parameter ()  

Estimate(log)	Exp(log)	Interpretation
0	1	No treatment effect on amplitude (treatment = control)
-0.3	0.74	26% reduction in amplitude in the treatment group relative to the control group
0.3	1.35	35% increase in amplitude in the treatment group relative to the control group

Estimation

Conventional multilevel location-scale models are generally estimated using maximum marginal likelihood [17]. However, when a model includes a substantial number of random effects, maximum likelihood estimation often encounters convergence challenges, estimation instability, and difficulties with model identification, especially with limited sample sizes or high model complexity.

Therefore, this study employs the Bayesian Markov chain Monte Carlo (MCMC) method for model estimation. In comparison with conventional maximum likelihood estimation, Bayesian methods facilitate more stable and flexible inferences, particularly when the model incorporates multiple random effects or faces convergence difficulties [13]. The Bayesian MCMC framework offers a distinct advantage in its capacity to facilitate straightforward interval estimation for variance components. This functionality is frequently hindered by boundary issues and asymmetric sampling distributions, which are prevalent in maximum likelihood estimation. The incorporation of prior distributions into Bayesian MCMC facilitates the regularization of complex models and stabilizes parameter estimation in scenarios with limited sample sizes or elevated complexity [20]. Recent studies have demonstrated that MCMC-based Bayesian estimation is superior in recovering parameter values and provides more robust inference under conditions of non-normality or missing data [21].

All models in the present study were estimated in Mplus using the default Bayesian settings for DSEM [22]. Fixed-effect parameters were assigned diffuse normal priors, , while variance components, including random-effect variances and residual variances, were assigned noninformative inverse-gamma priors, , as implemented by default in Mplus. For each model, two parallel Markov chains with distinct random seeds were executed, with each chain comprising 5,000 iterations. The initial iterations were designated as burn-in, in accordance with the default Bayesian procedure. Convergence was assessed using Mplus default diagnostics, including inspection of the potential scale reduction factor (PSR) and trace plots. All reported models showed satisfactory convergence, with PSR values close to 1 and stable trace behavior across chains.

Results

Simulation study

We conducted a Monte Carlo simulation to examine the performance of dynamic treatment effect modeling in RCT designs with ILD. Both models were then estimated for each simulated dataset, with the primary objective of evaluating their ability to recover the mean and dynamic treatment effects under varying data conditions. The simulation design was informed by previous studies on DSEM frameworks [12, 23] and adapted to the context of ILD and treatment evaluation.

Following these established frameworks, we implemented a fully crossed factorial design involving five manipulated factors (Table 2): sample size, number of time points, the between-group allocation ratio, RRV (), and dynamic treatment effect (). Specifically, sample sizes of N = 80 and N = 160 were selected based on prior methodological findings and the design of the present study. Preliminary studies indicate that DSEM generally necessitates sample sizes that exceed approximately N = 50 to attain sufficient statistical power and stable estimation [12]. Consequently, N = 80 was designated as a reasonable lower-bound condition, considering the RCT design and the possibility of between-group imbalance. To assess sensitivity to increased sample size, a commonly used doubling strategy was adopted, with N = 160 included as a comparison condition. Each participant was assessed at T = 15, 30, or 45 time points, corresponding to increasing measurement density. The minimum condition (T = 15) reflects empirical precedents in the DSEM literature, where dynamic parameters have been examined with short individual time series [7, 12]. The between-group allocation ratio was set to 1:1 and 0.3:0.7, representing balanced and moderately unbalanced group assignments commonly observed in RCTs. To account for inter-individual heterogeneity in within-individual variability, the RRV component was varied across two levels: 0.0 (homogeneous residual variance) and 0.4 (heterogeneity). The true dynamic treatment effect was specified at four levels (0, − 0.2, − 0.5, and − 0.8). These values were defined on the log-variance scale and correspond to no effect and progressively larger proportional reductions in within-individual amplitude (approximately 18%, 39%, and 55%, respectively), thereby capturing null, small, moderate, and substantively large dynamic treatment effects.

Table 2.

Simulation conditions

Factor	Levels
Sample size (N)	80, 160
Time points (T)	15, 30, 45
RCT ratio	1:1, 0.3:0.7
RRV	0.0, 0.4
Dynamic treatment effect	0, − 0.2, − 0.5, − 0.8

Crossing all levels of these factors yielded 96 simulation conditions. Eight hundred datasets were generated for each condition. The initial input data and all simulation outputs are available on the Open Science Framework (OSF) at https://osf.io/r3sbw/.

Evaluation

For each simulation condition, results from 800 replications were summarized from the Monte Carlo output generated by Mplus. For each parameter of interest, model performance was evaluated using five indices. Specifically, these indices capture aspects of parameter estimation accuracy, interval estimation quality, and inferential decision-making. The definitions of all evaluation indices are provided in (Eq. 7).

7

The most critical parameter in this study is the dynamic treatment effect (). The recovery of this parameter was evaluated using five performance indices: relative bias, root mean square error (RMSE), 95% coverage rate, Type I error rate, and power. The relative bias (RB) was evaluated across conditions, with values below 10% considered acceptable and values below 5% considered negligible [24]. RMSE, defined as the square root of the sum of the squared bias and variance, reflects the overall accuracy of the estimation. The magnitude of RMSE is inversely proportional to the precision of the recovery process. This makes it particularly useful for comparing conditions across varying sample sizes and time points.

The 95% coverage rate is defined as the proportion of replications in which the true parameter values fall within the estimated 95% confidence interval. Although Bayesian credible intervals do not guarantee nominal frequentist coverage, empirical coverage was examined as a diagnostic measure of interval performance across repeated samples, consistent with prior Bayesian Monte Carlo studies. A coverage rate ranging from 0.91 to 0.98 is considered acceptable [25]. The Type I error rate is defined as the proportion of replications in which a non-existent treatment effect is incorrectly detected. Finally, statistical power indicates the probability of correctly detecting a true dynamic effect. In accordance with established conventions, power values exceeding 0.80 and Type I error rates within the range [0.025, 0.075] are regarded as indicative of acceptable inference [26, 27].

Results

Relative bias

Figure 4 shows the relative bias of the dynamic treatment effect estimates across different combinations of sample size (N), number of time points (T), treatment effect magnitude, and group allocation ratios. Across all conditions, relative bias remained within the range of − 0.1 to 0.1, indicating negligible bias. The presence of relative bias diminished as the sample size increased and the magnitude of the treatment effects increased. While relative bias showed higher values under unequal allocation ratios than under equal allocation, this discrepancy diminished with increasing sample size. Overall, the model exhibited stable and acceptable bias across the examined conditions.

Fig. 4.

Relative bias of the dynamic treatment effect parameter () across conditions. RRV refers to random residual variance. Note: Conditions with a true effect equal to zero were excluded from the relative bias calculations

RMSE

Figure 5 shows the RMSEs of the dynamic treatment effect estimates across varying levels of true dynamic treatment effects (0, − 0.2, − 0.5, − 0.8), residual variance heterogeneity (RRV = 0, 0.4), and sample designs (N/T). Overall, compared with homogeneous residual variance, higher RMSE was observed under residual variance heterogeneity (RRV = 0.4), indicating reduced estimation precision as within-individual variability increased. In contrast, RMSE declined with increased sample size and elevated measurement frequency, a phenomenon particularly evident at T = 45, underscoring the efficacy of higher information density. In RCT designs, unequal treatment–control allocation ratios increased RMSE, indicating reduced estimation precision due to information imbalance between groups.

Fig. 5.

Root mean squared error (RMSE) of the dynamic treatment effect parameter () across conditions. RRV refers to random residual variance

95% coverage

Figure 6 shows the 95% coverage rates for the dynamic treatment effect estimates across various combinations of sample size (N), time points (T), and residual variance heterogeneity. These results highlight model robustness in maintaining credible interval accuracy across a range of designs and heterogeneity levels.

Fig. 6.

95% coverage of the dynamic treatment effect parameter () across conditions. RRV refers to random residual variance

Type I error rates and power

Figure 7A summarizes the Type I error rates under the null condition (treatment = 0) for different levels of residual variance heterogeneity and sample configurations. Across most conditions, Type I error rates remained close to the nominal level of 0.05, indicating robust control of false positives.

Fig. 7.

Type I error rate (A) and Power (B) of the dynamic treatment effect parameter () across conditions. RRV refers to random residual variance

Figure 7B illustrates the statistical power to detect treatment effects at three levels: small (treatment = − 0.2), moderate (treatment = − 0.5), and strong (treatment = − 0.8). Under strong treatment effects (e.g., − 0.8), statistical power remained consistently high, approaching or equal to 1.00 across all combinations of residual variance heterogeneity and sample design conditions. Conversely, for smaller treatment effects (e.g., − 0.2), statistical power was substantially lower and more susceptible to both residual variance heterogeneity and sample design. In the absence of residual variance heterogeneity (RRV = 0), power increased rapidly with larger sample sizes and higher measurement frequencies, reaching or exceeding the conventional threshold of 0.80 in most design conditions. However, under residual variance heterogeneity (RRV = 0.4), power was noticeably reduced, particularly in designs with smaller sample sizes and fewer measurement occasions. In such cases, increasing the number of measurement occasions (e.g., from 30 to 45) improved statistical power. Overall, strong dynamic treatment effects were reliably detected across designs, whereas adequate power for modest effects required greater information density through larger samples and more frequent measurements, particularly under residual variance heterogeneity.

Application to a health study example

This study draws on ecological momentary assessment (EMA) intervention research [28], which examined the effects of feedback-based interventions on emotional experiences in patients with depression, to illustrate the clinical importance of amplitude as an outcome indicator. The present analysis is secondary, and all affect measures followed the original measurement protocol. At each EMA assessment, participants rated affect-related adjectives on a 7-point Likert scale (1 = “not” to 7 = “very”), yielding polytomous responses. Negative affect was operationalized as a composite score based on the mean of four items (lonely, anxious, guilty, suspicious), which formed a negative affect factor in the original study. Measurement error was not modeled explicitly in the present analysis.

This study used EMA data collected during the five days following treatment completion, with a particular emphasis on changes in negative affect. Participants were permitted to report their emotions at any time, resulting in unequal numbers of observations across individuals. The mean number of reports in the control group (n = 24) was 22.2 (SD = 13.4), whereas the mean number of reports in the treatment group (n = 25) was 19.8 (SD = 11.9). Participants were randomly assigned to either the treatment or control group. At the observation level, the control group exhibited a mean negative affect score of 2.36 (SD = 1.29), while the treatment group demonstrated a lower mean score of 2.13 (SD = 1.38).

DSEM requires data in long format, which is particularly advantageous because it allows the inclusion of participants with unequal measurement intervals and varying numbers of observations. All analyses were conducted using Mplus 8.7 [22]. Missing values were coded as 999 and treated as unknown quantities within the Bayesian estimation framework; they were sampled using a Gibbs sampler with Metropolis–Hastings steps [10].

Results

Table 3 presents parameter estimates from the DSEM, including fixed, random, and treatment effects. The dynamic treatment effect on residual variance () reflects the extent to which the intervention affects within-individual amplitude in negative affect over time. The parameter was estimated on a log scale, representing the proportional change in within-individual amplitude between groups. The estimate of log () = − 1.504 (95%CI: -2.296 to -0.743), corresponds to a proportional reduction of approximately 77.8% (1 - exp (-1.504) = 0.778) in residual variance for the intervention group compared to the control group, suggesting greater negative affect stability. Standardized coefficients were obtained using the STDYX standardization option in Mplus. The corresponding standardized estimate ( = −0.489) indicates a moderate-to-large effect on within-individual variability, based on conventional effect size benchmarks [29].

Table 3.

Estimates for dynamic structural equation modeling

Effect	Parameter	Estimate ()	95% credible interval
Mean (Fixed effect)
NA		2.461(1.913) *	[1.937, 2.998]
Autoregression		0.273(1.131) *	[0.191, 0.353]
Residual variance		-1.185(-0.763) *	[-1.717, -0.628]
Variance (Random effect)
NA		1.611(0.975) *	[1.067, 2.541]
Autoregression		0.058(1) *	[0.033, 0.106]
Residual variance		1.807(0.761) *	[1.216,2.541]
Treatment Effects
Dynamic group		-1.504(-0.489) *	[-2.296, -0.743]
Mean group		-0.392(-0.152)	[-1.124, 0.363]

NA: Negative affect, *: statistical significance (i.e., the 95% credible interval does not include zero).: standardized results

In addition to estimating the dynamic treatment effect, the model also included the mean treatment effect (), yielding an estimated between-group difference of − 0.392, 95% CI [-1.124, 0.363]. The corresponding standardized estimate ( = −0.152) indicates a small effect size. Although this mean treatment effect was not statistically significant, this result should be interpreted with caution, as the absence of significance may reflect limited statistical power rather than a substantively negligible effect.

To further evaluate the robustness of these findings, a sensitivity analysis was conducted, examining the influence of prior specification on the dynamic treatment effect. Specifically, a comparison was made between the default diffuse priors in Mplus and two increasingly restrictive weakly informative priors (i.e., N (0, 1) and N (0, 10). The findings indicated that the estimated dynamic treatment effect exhibited remarkable stability across all prior specifications, with only negligible changes in posterior means and consistently significant 95% credible intervals (See Appendix A Table 1). This finding indicates that the observed treatment effect is resilient to prior assumptions, even under conditions with relatively small sample sizes.

Notwithstanding this limitation, the dynamic treatment effect on within-individual amplitude was statistically significant, indicating a reliable treatment-related difference in symptom variability over time. Instead of suggesting that mean-level effects are inconsequential, this pattern indicates that evaluations based exclusively on mean symptom changes may offer a limited representation of treatment-related processes. Consequently, the integration of dynamic assessments with mean-level effects may provide a more comprehensive understanding of the manifestation and evolution of intervention effects over time.

Supplementary simulation analysis

To further evaluate the performance of the proposed DSEM in estimating dynamic treatment effects under conditions comparable to the empirical application, we conducted a supplementary simulation study informed by the characteristics of the health study data. The sample size was set at N = 50. The simulation design was consistent with the model specification of the primary simulation study, with a particular emphasis on evaluating the estimation accuracy and robustness of the mean treatment effect and the dynamic treatment effect on within-individual amplitude.

The results of this supplementary simulation are summarized in Appendix B. The findings indicate that, except for conditions characterized by a minimal number of measurement occasions (T = 15) and the most negligible dynamic treatment effect (– 0.2), the proposed model yielded stable parameter estimates and reliable inference. It is important to note that as the magnitude of the dynamic treatment effect increased, adequate statistical power was achieved even under small-sample conditions (N = 50) and a limited number of measurement point (T = 15). These results provide further support for the applicability of the DSEM framework in practical clinical research settings.

Discussion

This study extends the DSEM to advance the evaluation of psychological dynamic treatment effects in RCTs. While traditional approaches to treatment evaluation have predominantly focused on mean-level changes over time, the DSEM offers a more comprehensive framework by simultaneously modeling within-individual variability (i.e., amplitude) as an indicator of dynamic change. This method allows us to examine not only whether a treatment reduces symptom severity but also whether it contributes to symptom or psychological stability at the individual level.

The DSEM incorporates individual-specific residual variance into the hierarchical structure of time-series data, providing a novel avenue for investigating treatment-related changes in temporal fluctuations. Changes in within-individual variability, such as reduced short-term fluctuations, may reflect clinically meaningful improvements in disease stability. For example, these changes may indicate more consistent symptom control or reduced physiological lability over time. This perspective acknowledges that treatment efficacy is not solely defined by reductions in symptom means. It is also defined by increased predictability and stability of clinical processes over time. By explicitly modeling dynamic treatment effects on variability, the present study enables the examination of whether interventions attenuate the amplitude of fluctuations rather than merely shifting average outcome levels. In this way, the proposed approach provides a more nuanced and ecologically valid assessment of treatment effects within the DSEM framework, particularly for ILD observed in clinical research.

Practical recommendations

Data sufficiency

When a new model is proposed, researchers are often concerned with the amount of data required to use it. The simulation results emphasize the importance of sample size (N) and measurement frequency (T) in accurately estimating and detecting dynamic treatment effects. Across all evaluation criteria—including model selection accuracy, parameter recovery, and statistical power—larger N and T values consistently improved performance. Specifically, increasing N and T reduces estimation error (RMSE) and increases the power to detect moderate treatment effects. Based on the results of the simulation study, we suggest that more than 1,200 data points are needed in equal-sized RCT groups to detect moderate treatment efficacy and individual heterogeneity (i.e., N = 80, T = 15).

In addition, in contrast to balanced RCT group allocations, unequal group sizes resulted in higher RMSE and lower power. However, increasing the overall sample size partially mitigated these adverse effects.

Asymmetric compensation between N and T

The simulation results indicated an asymmetric compensatory relationship between sample size (N) and the number of time points (T) in recovering the dynamic treatment effect parameter. In the context of residual variance heterogeneity (RRV = 0.4), the findings indicated that an increase in T resulted in a consistent reduction in RMSE, particularly in scenarios where sample size was constrained. To illustrate, the RMSE decreased from approximately 0.166 at N = 80, T = 15 to 0.156 and 0.154 at T = 30 and T = 45, respectively. Furthermore, upon attaining a moderate sample size (N = 160), subsequent increases in T resulted in enhanced estimation precision, as evidenced by a decline in RMSE from approximately 0.114 at T = 30 to 0.105 at T = 45.

A similar pattern was observed in the statistical power analysis. In a limited number of instances (T = 15), increasing the sample size from N = 80 to N = 160 modestly enhanced power. However, when a constant sample size of N = 80 was maintained, an increase in T from 15 to 30 and 45 resulted in substantial increases in power. The findings indicate that enhancing temporal density can significantly improve estimation precision and statistical power in detecting dynamic treatment effects, potentially compensating for reduced sample sizes.

Heterogeneity in dynamic processes

The increasing availability of time-series data has enabled researchers to examine dynamic processes in unprecedented detail, thereby attracting renewed attention to individual-level heterogeneity [30, 31]. However, in much of the clinical medicine literature, discussions of individual differences in treatment response remain largely focused on average treatment effects [16, 32], with limited attention paid to heterogeneity in intra-individual variability or changes in within-individual amplitude over time. In our study, we modeled residual variance as a random effect (), which allowed for variability in intra-individual fluctuations across individuals. This approach not only relaxes the assumption of strict homogeneity but also enables the examination of potential issues that may arise when individual heterogeneity is ignored. Our findings indicate that the degree of individual variability moderates the treatment effect, supporting the argument that dynamic processes often differ across individuals and, therefore, require appropriate modeling strategies. We further hope that this approach can be more widely adopted in clinical treatment evaluation, thereby enabling a more comprehensive understanding of treatment effects.

Limitations and future directions

While this study contributes methodologically to modeling dynamic treatment effects, several limitations remain. First, the current modeling approach relies on between-group comparisons within an RCT framework. Although randomization is increasingly used in intervention studies, non-randomized single-arm trials remain common in many fields [33]. For instance, Rupp et al. (2019) employed EMA to assess treatment sensitivity by comparing pre- and post-intervention symptom levels in individuals with obsessive-compulsive disorder [34]. Future research could extend dynamic treatment effect modeling to non-randomized single-arm designs by incorporating within-individual changes over time. Moreover, hybrid designs that integrate RCTs with single-arm trials may provide additional flexibility for applying dynamic treatment effect modeling in clinical settings.

Second, the Bayesian estimation in this study relies on the Mplus platform, which introduces certain limitations owing to its closed-source nature. Although Mplus provides stable estimation and user-friendly model specifications, its proprietary structure restricts transparency and extensibility. To enhance transparency and replicability, future research may consider migrating the model implementation to open-source platforms such as RStan [35], which allows greater flexibility in model customization and broader evaluation approaches, including posterior predictive p-values [20] and Leave-one-out cross-validation [36].

Third, a significant methodological consideration pertains to the identifiability and interpretation of residual variance. Residual variance may be indicative of true within-person process variability, as well as measurement error, alterations in reporting behavior, or unmodeled temporal structure. The interpretability of residual variance as an indicator of symptom stability is contingent upon the accuracy of model specification and the availability of sufficient observations per individual. In scenarios where per-subject time series are constrained, permitting heterogeneity in residual variance can enhance the complexity of the parameters and potentially diminish estimation precision, as evidenced by the simulation outcomes.

Furthermore, changes in residual variance do not exclusively reflect increased or decreased symptom stability. Such effects may arise from altered sensitivity to situational influences, differential reactivity to unmodeled time-varying factors, or heterogeneous treatment effects across individuals or covariates. Consequently, even in randomized controlled trials, variance effects should be interpreted with caution and not as definitive evidence of improved stability. The contribution of this study lies not in treating reduced variance as a stand-alone clinical conclusion, but in demonstrating that within-person variability represents a distinct and empirically testable dimension of treatment effects that is not captured by mean-based analyses alone. Variance effects are best regarded as complementary dynamic indicators that may signal changes in within-person fluctuation patterns. This suggests that further investigation of potential moderators and alternative model specifications is warranted in future research. More broadly, this perspective aligns with a growing methodological shift that moves beyond average treatment effects to examine structural or dynamic features of psychological processes. For instance, recent work in network psychometrics highlights that treatment may alter symptom connectivity or system organization rather than solely mean symptom levels [37–39].

Abbreviations

ILD

intensive longitudinal data

RCT

randomized controlled trials

DSEM

dynamic structural equation modeling

MSSD

mean squared successive difference

MELSM

mixed-effects location–scale modeling

RRV

random residual variance

MCMC

Markov chain Monte Carlo

EMA

ecological momentary assessment

RMSE

root mean square error

Authors’ contributions

SH initially conceived the study, designed and conducted the simulation study, and drafted the initial manuscript. WL supervised the research, contributed to the simulation design, and manuscript revisions. All authors read and approved the final manuscript.

Funding

No funding was received for this study.

Data availability

The data that support the findings of this study are openly available in OSF at https://osf.io/r3sbw/.

Declarations

Ethics approval and consent to participate

Not applicable. This study was based solely on simulation data; any example data used were obtained from publicly available datasets and did not involve human participants or identifiable personal data.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.