Abstract
Ayudhaya et al. examined the effect of Behavioral Activation on daily step count and heart rate variability among older adults with depression in a study labeled a cluster randomized controlled trial (cRCT). However, only one cluster was assigned to either of the study conditions. Such a design would have zero degrees of freedom for inferential testing, because the variation due to cluster membership cannot be estimated apart from the variation due to treatment assignment. Thus, the intervention effect is completely confounded with the cluster effect. The study should be labeled a quasi-experimental study, not a cRCT. Accordingly, the numerical results should be interpreted as associations but not evidence for causal relationships.
Keywords: Cluster randomized trial, Degrees of freedom, Inferential testing, Nesting
Ayudhaya et al. [1] conducted a study to examine the effect of Behavioral Activation (BA) on daily step count (as a physical activity indicator) and heart rate variability (as a depression-related autonomic function indicator) among community-dwelling older adults with mild to moderate depression. The article is labeled as a cluster randomized controlled trial (cRCT). However, there is only one cluster (i.e., hospitals) assigned to either of the study conditions (BA plus usual care or usual care only), which makes the study design invalid for estimating the causal effect of treatment assignment. Forty-one participants were recruited in each hospital after they were assigned to study conditions.
In cRCTs, there can be and typically are correlated data within each cluster (i.e., unit of randomization) so “errors” (loosely speaking, model residuals) cannot be assumed to be independent across study participants within clusters. Moreover, when clusters (rather than individuals) are assigned to study conditions, the study has a hierarchical structure (individuals nested in clusters) [2]. Thus, the degrees of freedom for testing the intervention effect depends both on the number of individual participants and on the number of clusters per condition [3]. A purported cRCT with two conditions and only one cluster assigned to each condition, such as Ayudhaya et al.’s study [1], would have zero degrees of freedom (n1 + n2–2) for inferential testing. That is, the variation due to cluster membership cannot be estimated apart from the variation due to treatment assignment, and a valid analysis of the intervention effect could not exist [4].
We commend Ayudhaya et al. for addressing an important question about depression among the older population, and for their collegiality in providing clarification about their study design. The authors’ clarification confirmed our understanding from reading the published methods.
Because in Ayudhaya et al.’s study [1] the intervention effect is completely confounded with the cluster effect, it should be labeled a quasi-experimental study, not a cRCT. Accordingly, the numerical results should be interpreted as associations between BA and the outcomes but not evidence for causal relationships [5].
Acknowledgements
Not applicable.
Abbreviations
- BA
Behavioral Activation
- cRCT
Cluster Randomized Controlled Trial
Authors’ contributions
Conceptualization: DBA, YJ-N, LG-A, ABS, CJV; Investigation: YJ-N; Writing original draft: YJ-N; Review and editing: DBA, LG-A, ABS, CJV. All authors approved the final draft.
Funding
Authors supported in part by NIH grants R25DK099080, R25HL124208, U24AG056053, P30AG050886, and the Gordon and Betty Moore Foundation. The opinions expressed are those of the authors and do not necessarily represent those of the NIH or any other organization.
Availability of data and materials
Not applicable.
Declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
Dr. Allison and his institution (Indiana University) have received payments for consultation, grants, contracts, in-kind donations, and contributions from multiple for-profit and not-for-profit entities interested in statistical design and analysis of experiments, but not directly related to the research questions addressed in the paper question.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Ayudhaya WSN, Pityaratstian N, Eungpinichpong W, et al. Effect of behavioral activation on time and frequency domain heart rate variability in older adults with subthreshold depression: a cluster randomized controlled trial in Thailand. BMC Psychiatry. 2022;22:319. doi: 10.1186/s12888-022-03962-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Murray DM, Varnell SP, Blitstein JL. Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health. 2004;94:423–32. doi: 10.2105/AJPH.94.3.423. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Jamshidi-Naeini Y, Brown AW, Mehta T, et al. A practical decision tree to support editorial adjudication of submitted parallel cluster randomized controlled trials. Obesity. 2022;30:565–70. doi: 10.1002/oby.23373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Varnell SP, Murray DM, Baker WL. An evaluation of analysis options for the one-group-per-condition design: can any of the alternatives overcome the problems inherent in this design? Eval Rev. 2001;25:440–53. doi: 10.1177/0193841X0102500402. [DOI] [PubMed] [Google Scholar]
- 5.Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin. 2002.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Not applicable.