Limitations of CBA study: Controlled before after study

Sir,

Apropos the article “Evaluation of short-term use of nocturnal nasal continuous positive airway pressure for a clinical profile and exercise capacity in adult patients with obstructive sleep apnea-hypopnea syndrome” published in your journal.[1] By conducting this study the authors have done a commendable job. However, I have few concerns about the study design chosen by the authors for the study.

The authors have made use of non-randomized quasi-experimental, pre-test/post-test study design with a contemporaneous control group. In a controlled before-after study two or more groups are compared with each other, usually comprising one group in which an intervention is carried out (experimental group) and one group where no or an alternative intervention is conducted (control group) as was done in the present study. Although group allocation is not random, the pre-test measurements tell us whether the two groups are similar. Therefore, if a change between the pre- and the post-test measurements is observed only for the intervention group, the assumption of causal inference is more robust than it would be had there been no control group for comparison.[2] A control group provides some evidence that changes occurring over time were not the result of natural temporal trends or of unmeasured events that occurred contemporaneously with the exposure under study. Although adding a pretest or a control group can help to overcome the problem of confounding in quasi experimental studies, there is always a risk of unidentified confounders in studies that lack randomization. Because such control groups are not selected randomly, associations between intervention and outcomes are at some risk for confounding or bias. Another potential limitation of quasi-experimental study designs is regression to the mean, a common statistical phenomenon in which extreme values on any measure at a point in time will probably be less extreme the next time it is measured. This is particularly a problem if the groups selected for study are identified by their initial “outlier” status with respect to the outcome.[3] Its limitations notwithstanding, the quasi experimental study design is useful in situations in which randomization is not possible because of (i) ethical considerations, (ii) the inability to randomize individual patients or locations, or (iii) a need to intervene quickly (hence, the intervention is undertaken in the context of clinical care and only retrospectively evaluated as research).