Methodological issues in conducting yoga- and meditation-based research: A narrative review and research implications

Abstract

Yoga and meditation-based interventions have been extensively utilized in the field of contemporary complementary and alternative medicine for various physical and mental health conditions. Ongoing COVID-19 pandemic has rekindled the interest of researchers in yoga and meditation for its preventive and therapeutic utilities. However, the available literature in this area has several methodological concerns, limiting formers’ clinical utility. A comprehensive literature on this topic would stimulate researchers and guide them to conduct research on this topic with robust methodologies. The current review highlights the methodological issues with the yoga and meditation-based Research (henceforth, MBR), discusses some of the contentious issues, and provides future directions. The PubMed, Medline, and google scholar databases were searched to screen records dealing with the methodological issues on MBR. The search yielded 299 records, upon screening, only 24 articles were found suitable for the current study. Common methodological issues with MBR: lack of the consensus definitions of the yoga and meditations, interventions lacking theoretical framework of meditation; inadequate description of the study design; difficulty with participants recruitment, setting up the control groups, and blinding; difficulty in assessing the baseline characteristics of the participants, and validity issues with the outcome measures. A few research, however, have also highlighted the potential measures to overcome these methodological challenges. Yoga and meditation-based interventions are promising for several health conditions. However, literature suffers from considerable methodological issues, thus, limiting its utility in modern clinical practice. The study findings can stimulate and guide future research on this topic.

1. Introduction

Meditation and yoga (defined as a combination of meditation and its context) [1], have gained popularity in contemporary scientific research and have been used for several mental health (stress, anxiety, depressive disorders, etc.) [2,3] and physical conditions (pain, etc.) [4,5]. Worldwide, yoga and meditation are explored as an alternative and complementary approach to the treatment for both psychological and physical disorders, and to attain a better quality of life [[6], [7], [8], [9]]. Apart from the clinical population, it has also gained popularity among the non-clinical population as a means to attain a state of general wellbeing. It is seen as a relatively safe, inexpensive, and sustainable measure that can be used as a standalone or as an adjunct to the standard treatment, in achieving good health or for a specific health condition [10,11]. Ongoing COVID-19 pandemic has rekindled the interest of researchers on yoga and meditation for its preventive and therapeutic utilities. Both yoga and meditation (meditation, henceforth) have been utilized for various mental health problems and to boost immunity to fight against the physical impact of the SARS-CoV-2 (COVID-19) infection [12,13].

However, the available research on the yoga and meditation-based interventions suffered from several methodological limitations: non-incorporation of the theoretical framework of the yoga/meditation practices while developing the interventions, lack of validated instruments to measure the practitioners’ experience, issue related to participant recruitment, and setting up effective control arm, concern over monitoring of meditation-based clinical trials, difficulty in assessing the role of various overt and covert factors on the outcome, etc. [[14], [15], [16], [17], [18]]. This has also been reflected in the ongoing research on the efficacy of yoga and meditation-based intervention for mental and physical consequences of COVID19 pandemic [19].

Despite literature highlighting several methodological issues with yoga and meditation-based Research (henceforth, MBR), available literature is largely restricted to mindfulness meditation, while literature concerning other common forms of yoga and meditation practices is elusive, which have distinct methodological concerns. Furthermore, a comprehensive account on the ontological aspects of the yoga and meditation practices and their adaptation in the modern contemplative neurosciences, and associated challenges, particularly those centering around the meditation-based intervention, is still lacking. An inclusive literature on this topic can stimulate researchers and guide them to conduct research on this area with robust methodologies, which this paper intends to provide.

Hence, the current review is aimed to highlight various methodological issues in conducting research based on meditation-based interventions, discuss some of the contentious issues, and provides future directions.

2. Methods

2.1. PubMed, Medline, and Google Scholar databases were searched with the search terms

‘Yoga’ OR “meditation” (for yoga and meditation), ‘research’ and ‘methodologic issues’ or ‘research implications’ (for methodological concerns) were used to identify the relevant records (Supplementary file 1). The inclusion criteria for the studies to be considered for the current review were articles primarily dealing with the methodological issues on MBR or research implications of these limitations (e.g., issues with the definition, study design, participants recruitment, intervention delivery, outcome assessment, statistical analysis, etc.), irrespective of the type of yoga and meditation practices, from the inception till 14th May 2020. However, articles not directly dealing with the methodological issues, those papers whose full text was not available (despite making efforts to contact the authors), and those not in the English language were excluded. Additional relevant records were obtained through the bibliographic search.

3. Result

A total of 299 records were obtained. Upon title and abstract screening, only 24 records were found suitable for the current review. Among the 24 articles selected for the review, most were related to methodological issues in researching MM followed by those with transcendental meditation and yoga-based therapies (described in Table 1).

Table 1.

Summary of the relevant research/papers on the methodological issues with the yoga and meditation.

Author (year)	Title of the study	Methodology	Findings	Remarks
Vieten et al. (2018) [56]	Future directions in meditation research: Recommendations for expanding the field of contemplative science.	Design: cross-sectional Aims: to investigate the prevalence of a wide range of experiences. n = 1120 meditators Inclusion Criteria: current or past meditation practice Outcome: MEQ30, Extraordinary experiences	•Gender: 59% female •Average age of 47 ± 16 •Comorbid psychiatric illness: 25% of participants •Majority of respondents report having had many of these anomalous & extraordinary experiences	•These aspects of meditation could be crucial to practitioners' psychological & spiritual development •Could also serve as mediators/mechanisms for conferring benefits. •They can be subjected to rigorous scientific investigation
Munaz et al. (2017) [57]	The importance of research literacy for yoga therapists.	Design: perspective	•Research literacy & evidence-informed practice are essential skills for yoga therapists. •They should be systematically included in yoga therapy programs. •Many yoga therapists have limited training in these skills, which negatively impacts inter-professional communication & advancement of yoga research	•Should be introduced early in the course curriculayoga therapy programs •Include qualified faculty to teach these competencies
Van Dam NT (2018) [17]	Mind The Hype: A Critical Evaluation and Prescriptive Agenda for Research on Mindfulness and Meditation	Design: review mindfulness meditation Objectives: to highlight difficulties of defining mindfulness, delineates scope of research into mindfulness practices, & illuminates crucial methodological issues for interpreting results from research of mindfulness	•Various possible meanings of ‘mindfulness’ need to be clarified •Researchers must adopt nuanced, precisely focused, terminology for referring to various states/behaviours a/w mindfulness. •MBR must work on construct validity of the practice. •Sound methodology e.g., adequate outcome measures, good psychometric properties, trial registration would aid in research •MBI research needs to have uniform & better control; participants need to be sensitized about the potential ADRs with it. •Advances in contemplative neuroscience should explore mental processes & brain mechanisms of mindfulness practices & report them properly.	•Greater research is required toward improving the rigor of methods used, coupled with the accuracy of news-media publicity & eliminating public misunderstandings aroused by past undue ‘Mindfulness Hype’
Davidson & Dahl (2017) [25]	Outstanding Challenges in Scientific Research on Mindfulness and Meditation	Design: commentary	•Methodology issue with the operational definition of the MM or meditation is not unique to MBR but applied to other area of behavioral sciences as well. •Both self-report measures & implicit behavioral aspects need to be considered in MBR •Future research needs to expand their horizon to include other family of contemplative practices rather limiting to MBR •Mindfulness practices have been conceived for attaining a general wellbeing; their utility for various disorders need to be researched contextually •Greater research is required on the frequency, duration, & spacing of the mindfulness practices & how it can be piggybacked with ADLs for its wider dissemination •Power of mobile technology can be harnessed in both disseminating contemplative training & its assessment	•These issues are germane to both basic & clinical research studies. •Also, it has important bearing on the future scientific investigation of mindfulness & meditation
Davidson & Kaszniak (2015) [26]	Conceptual & Methodological Issues in Research on Mindfulness & Meditation.	Design: review	•State & trait effect of the MM & their interactions along with practitioner's personality & expectations, compliance with the practices must be studied as they are crucial determinants of efficacy •Many practices are subsumed under MM though the extent of practice differ •The experiential account of the practitioner can be strengthened by incorporating second & third persons' account •Challenges are also posed by research designs where double-blinding is not possible: active control, e.g., dual blinding, could evade this limitations. •Adequate description of the intervention, comparable control arm, credence of the instructors, & fidelity of intervention delivery need to be ensured. •Similarly, the question of measurability of the mindfulness, what can & cannot be inferred from self-report measures; & utility of contemplative neurosciences are essential. •Robust study design & data analyses are also crucial.	•Mindfulness-based research is still in infancy •The methodological limitations can be overcome by robust study design. •Mindfulness-based intervention should be tested both for the clinical and non-clinical population with giving due consideration to its conceptual framework
Van Dam NT (2018) [58]	Reiterated Concerns and Further Challenges for Mindfulness and Meditation Research: A Reply to Davidson and Dahl	Design: commentary	•Nascent Scientific Fields, incl. MBR, are especially susceptible to methodological Issues, thus initial findings, or mis-findings, can have huge long-term impacts. •Meditation can cause adverse S/E; even in a small subset of participants, it can matter a lot •Impact of MBIs on clinical population should be investigated more thoroughly, for former is not traditionally meant for medical conditions, but general well-being. •Assessing & establishing dose–response curves of the meditation is crucial •Its impact must be ascertained based on outcome (e.g., alleviation of anxiety, attaining wellbeing) •For mobile-based system to deliver MBIs & for their dissemination, much research is warranted to establish fidelity of intervention delivery, analyse participants characteristics, etc.	•The use of MBIs, including its delivery, for various conditions need to be established under the background of certain unresolved methodological and impact issues.
Patwardhan AR (2017) [59]	Aligning Yoga With Its Evolving Role in Health Care: Comments on Yoga Practice, Policy, Research	Design: commentary	•Yoga therapists need to calibrate their model of yoga by reducing emphasis on postures & increasing it on meditation & breathing exercises while dealing with clients with chronic health issues. •Yoga research should be more critical in evaluating yoga's fundamental framework & have reductionist approach. •Regulatory bodies must extricate injury prone postures from the practice for regulatory purposes, than regulate yoga summarily. •Insurers should pay for yoga akin to vaccination (paying for initial dose & then booster sessions), this would widen the coverage of yoga & prevent future ill health	•There is certain practice, policy, & research related issues concerning yoga, hence, it should be promptly & adequately addressed for yoga to flourish and bring fruition to the population
Field T. (2016) [2]	Yoga Research Review	Design: review (Of empirical research, systematic reviews and metanalyses)	•Overall, yoga has been more effective than control, including waitlist control, for several physical & psychological problems, although results are mixed when compared with other treatment arms e.g., exercise •Having established the benefits of yoga, conducting a research without active control group, thereby not maintaining research equipoise, raises ethical concerns. •Shorter sessions should be investigated for cost-effectiveness & for ensuring daily practice. •Multiple physical & physiological measures, particularly objective measures, need to be added to the self-report research protocols. •Potential underlying mechanisms of effectiveness of the yoga to be explored.	•There is literature support for yoga as an effective strategy for several medical and psychological problems •Such interventions van be termed as yoga therapy •Such interventions van be termed as yoga therapy
Uebelacker et al. (2012) [60]	Yoga for Depression and Anxiety: A Review of Published Research and Implications for Healthcare Providers	Design: review	•Preliminary evidence suggests the effectiveness of yoga for anxiety, depression, & PTSD. •With evidence being strongest for unipolar depression. •However, there are risks with engaging with yoga as well.	•HCPs can play a crucial role in evaluating the safety & utility of community-based yoga class in patient population.
Jeter et al. (2015) [29]	Yoga as a Therapeutic Intervention: A Bibliometric Analysis of Published Research Studies from 1967 to 2013.	Design: bibliometric analysis of clinical trials based on yoga	•A three-fold increase in yoga publications in the decade. •45% are RCTs f/b controlled (18%) & uncontrolled trails (37%) •Top three disease conditions are mental health, CVDs, & respiratory disease. •Research suffers from the methodological limitations (e.g., insufficient sample size). •The research also suffers from publication biases with uncontrolled & poor quality studies getting published in traditional journals.	•The use of yoga as a complementary Tt in clinical practice may lead to health benefits beyond traditional Tt alone. •Determining the suitability & content of yoga for a specific condition/patient poses challenge for evidence-based integrative medical practice •However, to effect changes in health care policy, greater high-quality, evidence-based research is required.
Cook-cottone et al. (2013) [61]	Dosage as a critical variable in yoga therapy research.	Design: perspective	•Considerable variability exists about the optimum dosage/duration of yoga to be effective (range 4–32wks, median 8wks) •Until empirically meaningful dosages, incl. length & frequency of sessions & longitudinal duration of practice are found, clinical guidelines would be arbitrary •Home practice contributes substantially to the benefits of yoga practice, which often remains unaccounted	•Therapeutic dosage is a critical component of yoga intervention research. •Empirically supported guidelines are required to establish best practices for research & to create clinical recommendations for effective use
Josipovic Z (2010) [22]	Duality and nonduality in meditation research	Design: commentary	•There is a taxonomical concern with the classification of the meditation: FA, OM, & AT types •A dualistic & non-dualistic classification of meditation for cognitive neurosciences & research is more pragmatic, albeit with some research-related limitations. •A multidimensional construct involving targeted states of consciousness; duality to nonduality scale; stages of expertise; cognitive processes; & objects of meditation is better •There is an overlap among the different meditation types concerning EEG changes. •Hence, formers' utility in meditation categorization needs to be further studied	•Expanding the current taxonomy of meditation and defining the characteristic neurophysiological signatures of various meditation •Categories are important issues in meditation research
Park et al. (2015) [62]	Comparison groups in yoga research: a systematic review and critical evaluation of the literature	Design: systematic review	•n = 128 RCTs •65 included only a passive control while 63 included at least one active comparator •Primary comparison groups: physical exercise (43%), meditation/relaxation (20%), & general education (16%) •Literature rarely provides the rationale for the choice of the a particular comparator •Notably, use of active controls in yoga research seems to be gradually increasing over time	•Considering that yoga has been established as a potentially effective intervention, future research should endeavour having active control groups •Furthermore, care is needed to select a robust comparison conditions that help to identify the specific mechanisms underlying yoga's effects
Goyal et al. (2014) [3]	Meditation Programs for Psychological Stress and Well-being: A Systematic Review and Meta-analysis	Design: systematic review & Meta-analysis	•n = 47 trials with 3515 participants •MM programs had moderate level of evidence for improving anxiety, depression, & pain at 8 weeks to 3–6 months. •Low evidence of improved stress/distress & mental health–related quality of life •No effect or insufficient evidence of any effect on positive mood, attention, substance use, eating habits, sleep, & weight •Meditation programs did not fare better than drugs, exercises, & other behaviour Therapy	•Clinicians ought be prepared to convey their patients about the potential role of the meditation program could have in addressing their psychological stress. •Robust study designs are required to elucidate the impact of meditation programs in improving the positive dimensions of psychological health and measures of stress.
Sedlmeier et al. (2012) [16]	The psychological effects of meditation: a meta-analysis	Design: systematic review & Meta-analysis	•Majority of the studies were excluded d/t methodological issues •Most studies were conducted without theoretical background •Mediation-based interventions had moderate effect sizes •Results were strongest (medium-large) for changes in emotionality & relationship problems, less strong (medium) for measures of attention, & weakest (small-medium) for cognitive measures.	•A comprehensive understanding of why & how meditation works, prominence should be placed on the development of more accurate theories & measurement tools.
Awasthi B. (2013) [14]	Issues and Perspectives in Meditation Research: In Search for a Definition	Design: commentary	•A wide array of techniques & methods are collectively termed as “meditation” with mixed and at times conflicting findings reported within the contemplative neurosciences •There is the lack of philosophical grounding for the neuroscience of meditation. •For instance, modern neurosciences considers it in light of mind-body dichotomy, however, traditional meditation ontology sees it as non-dual entity. •Lack of sound definition results in methodological limitations, including replication of the studies. •Also, results in non-elucidation of its mechanism of action for neuro-cognitive-behavioural changes.	•Paying due attention to the contextual & inception of meditation traditions is vital •1st person methodologies should be integrated with the 3rd person methodologies of neurosciences •Specific outcome measures should be assessed for the different phases & types of the meditation •An integration of traditional ontology, 1st person phenomenological accounts & neuroscientific findings will facilitate the development of more comprehensive models of the mind for neurocognitive research with the contemplative traditions.
Davidson RJ (2010) [15]	Empirical explorations of mindfulness: Conceptual and methodological conundrums	Design: commentary	•The use of the term “mindfulness” has varied substantially across the articles, ranging being a state to trait to an independent variable •Measures of the mindfulness, duration, and adequate comparison group related issues have been the consistent methodological concerns	•Additional research with focus on the potential targets within the emotion domain of contemplative practices is warranted
Larkey L. (2021) [40]	Meditative Movement as a Category of Exercise: Implications for Research	Design: review (on Meditative Movement e.g., Tai-chi, Qigong, etc.)	•Relevant dimensions of Meditative Movement, e.g., frequency, duration, type of movement, degree of exertion, breathing type, etc. are recommended to be succinctly described & measured. •To consistently define the category across studies & illuminate how Meditative Movement may affect health outcomes in similar, & perhaps different, ways than conventional exercise.	•As Meditative Movement is studied as a category of exercise, research may progress more efficiently to define the domains of physiological and psychological benefit
Lustyk et al. (2009) [37]	Mindfulness meditation research: issues of participant screening, safety procedures, and researcher training	Design: review	•Mental health- followed by physical & spiritual health consequences are common S/E with MM. •Despite this, most research do not describe, or even consider them. •Research participants/patients should be thoroughly medically examined before enrolling in MBI	•A cautious approach to using meditation in research is warranted in the face of the limited and •Preliminary nature of current evidence to maximize participant safety.
Grossman P. (2008) [41]	On measuring mindfulness in psychosomatic and psychological research	Design: commentary	•Definition of mindfulness has been varied across the studies. •Denominator of “present-moment awareness of perceptible experience” might, at first consideration, seem feasible; but suffers from reductionism •The inventories used to measure mindfulness vary substantially, therefore, there is a poor correlation among them •The measuring instruments also suffer from semantic confusion •Valid & reliable instrument are lacking that would objectively measure the mindfulness. •Mindfulness inventories may be hampered by serious limitations of construct validity	•Qualitative analysis of the experiences of the Buddhist & western practitioners of mindfulness & putative consequences (e.g., general wellbeing) of MM can assist in measuring the level of mindfulness •It must be appreciated that Buddhist & Western approaches to mind & body differ substantially, thus study of mindfulness- & based intervention should be understood in its traditional framework.
Travis et al. (2010) [21]	Focused attention, open monitoring and automatic self-transcending: Categories to organize meditations from Vedic, Buddhist and Chinese traditions	Study design:perspective	•Each of the three meditation-categories-focused attention, open monitoring, & aut. self-transcending- represents practices with different degrees of attention control, subject/object relations, & different procedures •However, TM technique, the most researched of the aut. self-transcending procedures, is often placed under FA	•These explicit differences between meditation techniques must be regarded when studying physiological patterns or clinical outcomes of meditation practices.

AT: autonomic transcendence, a/w: associated with, CVDs: cardiovascular disease, d/t: due to, EEG: electroencephalogram, FA: focused attention, HCPs: Healthcare providers, MBR: mindfulness-based research, med.: medium, MEQ30: Revised Mystical Experience Questionnaire, MM: mindfulness meditation, MBI: mindfulness-based interventions, OM: open monitoring, PTSD: post-traumatic stress disorders, RCTs: randomized controlled trials, S/E: side-effects, TM: transcendental Meditation Tt: treatment.

4. Major methodological issues with the available MBR

Literature suggests the common methodological issues with MBR to be lack of the consensus definitions of the meditations, meditation-based interventions lacking theoretical framework of meditation; inadequate description of the study design; difficulty with participants recruitment, setting up the control groups, and blinding; difficulty in assessing the baseline characteristics of the participants, and validity issues with the outcome measures.

4.1. Defining meditation

One of the major methodological issues with MBR is the operational definition of the meditation practice under consideration and to explicitly describe its core component(s). There have been substantial variations in the definition/framework used across the studies, despite a large number of studies on meditation, including the MM, they lack an operational definition and that varies across the studies, as a result, the effect size of the intervention of a particular kind cannot be measured. Meditation has been classified under two broad categories: 1) Focussed attention (FA) (concentrative type), where the practitioner voluntarily focuses ones' attention within oneself (breathe, thoughts, imagery, etc.) or outside (candlewick, an image of the deity, etc.) to the exclusion of all other experiences e.g. Zen meditation, hath yoga, and Sudarshan kriya yoga (SKY) and 2) Open monitoring (OM), where the attention is not directed to a particular thing, but to remain aware in a non-reactive manner about whatever arises in the mental continuum of oneself e.g. MM, Vipassana meditation, etc. [20] However, this dual classification is limited because of considerable overlaps between the practices, resulting in ambiguity and inconsistencies in the definitions in the literature. Furthermore, a third category of meditation the automatic self-transcendence (AST)-which involves transcending the practice of meditation and harnessing the natural tendency of the body to attain pure consciousness (transcendental meditation [TM], Sahaj Samadhi Meditation)-has also been proposed. It is a state of complete letting go and settling into oneself [21]. Although AST-based meditations begin with the FA, the very practice of transcending the subject-object duality is effortless from the beginning itself as compared to FA-based meditations. For instance, in TM, the mantra chanting though begins with the FA, the practitioner from the very beginning learns to allow oneself to lose focus from the mantra and transcend the subject-object duality to attain self-awareness [22]. In fact, of the eight limbs of Yoga described by the ancient sage Maharishi Patanjali, one limb is Pratyahara or focussing inside e.g. focussing attention on the heart region, followed by Dhyana or meditation where thoughts/feelings/body sensations are observed and one is in a state of ‘witness consciousness’ followed by transcendence which is a deeper state where one has completely let go [21,23]. Yet, there is another proposed category, the guided-meditation (GM) (e.g. Love and kindnesses meditation), where the content of meditation takes precedence over other aspects of the meditation, and a practitioner is guided through a set of images, or chants to engage in a particular aspect of self (empathy, kindness, etc.) mindfully [24].

4.2. Characteristics and components of the yoga and meditation

The available research on yoga and meditation-based interventions has been criticized for not accounting for and not explicitly describing various components of the meditation under study including their key elements [2,25]. Likewise, these baseline differences among the participants of different interventions group can confound the research findings. Therefore, simply focussing on a particular aspect of meditation (like the degree of mindfulness achieved during the MM or level of physical exertion in the hath yoga or Qigong) and excluding other components (the practice of mindfulness in day-to-day activities including exercise, or routine work, etc.) could lead to inaccurate and inconsistent study findings [17].

Likewise, randomly dismantling the process of the meditative practice (e.g. eliminating the integral moral, or lifestyle changes associated with a particular mediation) as per convenience while developing an intervention, without considering its ontological significance, would lead to the development of a distorted or flawed intervention; the findings of such interventions would not be representative of the original contextual meditation. Consequently, the findings would be inconsistent and less-interpretable [14,26].

4.3. The theoretical framework of yoga and meditation

Yoga and meditations have been practiced since antiquity across different cultures. For instance, the concept of meditation in Hinduism is a practice that helps in attaining the pure-consciousness (enlightenment or self-connection) [27] while Buddhism considers it away to achieve emptiness (Shunyata, skill to disentangle oneself from the defilement and dissatisfactions; state of no-attachment) [28]. Neither of them has developed meditation merely for prevention or treatment of a particular health condition. The researchers involved in contemplative sciences have argued that simply adapting the traditional meditative practices for developing an intervention to treat a particular health condition raises validity concerns. For instance, there are now several MM-based interventions (incorporating the principles of mindfulness to varying degrees) such as mindfulness-based cognitive behavioral therapy (MB-CBT), dialectic behavioral therapy (DBT), acceptance and commitment therapy (ACT), etc. However, research analyzing the construct validity of these interventions is still lacking; similarly, the outcomes and their measures used to assess the effectiveness of these interventions also have validity concerns [17,25]. Such contextual issues act as a prominent hindrance in the field of the MBR. Although picking up these practices without the associated philosophical framework has increased its acceptability in the west and the medical and scientific community worldwide, however, researchers need to be mindful about the fact that such derivations, viewed only microscopically through the lens of modern science, could lead to concern over the time-tested practices that have benefitted people for time immemorial. Be that as it may, the current approach can be pursued for serving the purpose of research and science but with a clear understanding of its limitations.

4.4. Study design

Most of the studies were commentaries or perspectives of the experts followed by the review papers, including systematic review and metanalysis. In addition, one of the studies has supplemented its review paper with the findings of the cross-sectional online survey [56]. The nature of the current review, and methodological issues with the yoga and meditation-based research, however, restricted the incorporation of the original articles.

4.5. Sample size of the studies

Researchers have also suggested that MBR, especially those on MBI, suffers from the limited sample size of the participants across the comparison groups [26,29]. Moreover, baseline differences between the comparison groups, which often remain unaccounted, could act as confounders in assessing the effectiveness of the meditation-based interventions [16,26].

4.6. Assessment of the effect of the yoga/meditation as a state versus a trait phenomenon or an interaction between them

The existing evidence on the effectiveness of the meditation-based interventions has been criticized for studying and reporting the effect of the meditation only during the meditative state (a state phenomenon) i.e. trying to assess its impact merely based on the neuro-psychological findings during the practice of meditation or by comparing the mental state immediately before and after practice [14,26].

The impact of the meditations is also influenced by the duration of the practice (adept practitioner versus novice), both formal and informal, and the constellation of pre-existing traits of the practitioner (their level of motivation, baseline understanding and expectancies from the meditation, adherence to the practice, and the preparatory steps including the lifestyle modifications associated with the practice) [20,30,31]. For instance, in Buddhism, certain prerequisites are there before enrolling an individual into the MM which includes ones' life's goals and the baseline understanding about the Buddhist philosophy; such features determine at which level of meditation the participants to be enrolled in [32]. In contrast, simply studying the state effect of meditation without considering its long-term effect (trait effect) would limit its clinical utility in the real-world scenario. Thus, it would be prudent to study meditatation in the context of a state and trait interaction.

4.7. Participant recruitment process

Literature suggests that MBR often suffers from selection bias [26]. Research shows that most of the participants getting involved in the MBR have an inherently higher level of enthusiasm and expectancies towards the meditation than their less enthusiastic counterparts who either do not participate or exhibit lesser compliance with the instructions. Moreover, it is not uncommon for the participants to have a prior association with a particular meditation group, this may be a potential source of bias [33,34]. Lastly, participants’ association with the investigator, if both are part of the same yoga/meditation group, may also lead to a personal relationship effect [16]. Such samples are not representative of the population from which they are drawn and can also act as a major confounder, therefore, the results of such research have serious generalizability issues and a possibility of biased results (unduly higher effect size of the interventions) could not be ruled out [35].

4.8. Participant safety

The MBR has been criticized for not adequately accounting for and describing the potential adverse consequences of the meditation especially in the vulnerable population (those with pre-existing physical or mental illnesses) [26]. Further, MBR has been criticized for not providing adequate information to the participants while obtaining consent. This issue is particularly important as a certain vulnerable population are at risk of experiencing adverse effects which include psychological (depersonalization, psychosis, dysphoria, initial worsening of the anxiety symptoms, etc.), physical (worsening of the somatic symptoms and risk of epilepsy, et.), and spiritual-adverse effects (conflict between the philosophy behind the given meditation and the religious belief of a participant, etc.) [36,37]. This methodological limitation is to a certain extent has to do with the poor knowledge of investigators about the potential meditation-related adverse consequences [17].

4.9. Randomization, allocation concealment, and blinding

The randomization of the participants adds to the vigor of the study. However, the literature shows that MBR often inadequately describes the process of randomization of the participants. Another methodological limitation is the participants' selection for the study. For instance, if a given intervention is expected to affect the cognitive ability (e.g. multitasking) of the participants, then it is prudent to recruit participants for whom multitasking forms an important part of their work or life [18,38]. This criticism, however, stems from the belief that meditation may affect a specific cognitive ability. However, based on the neurobiological findings that show involvement of multiple areas of the brain during meditation, it is unlikely that the benefits of meditation are so specifically limited to a specific cognitive ability as would happen during a psychological intervention designed to enhance a specific cognitive skill.

The greater issue is related to allocation concealment and blinding. Lack of allocation concealment can introduce a bias in assigning participants to a particular group. This can stem from the association between the participants and the investigators and a potential tendency towards allocating the more enthusiastic participants to the active group than their less interested counterparts. Usually, the MBR follows a wait-list control design or control arm receiving the non-specific interventions like general exercise, dietary advice, self-help books, etc.; consequently, the participants are liable to exhibit demand characteristics (behaving or experiencing in a manner that is expected of their placement in a particular group) [26,39]. Such a bias can also occur at the level of the instructor who may have an association with the meditation group or is aware of which arm they are training to, consequently, their level of effort and commitment may vary accordingly [26], though this potential bias needs further exploration.

Similarly, the findings of the effectiveness studies are vulnerable to getting biased if the participants, the assessor, or the analyzer/statistician are aware of the group to which a particular participant belongs. Thus, for MBR, blinding at multiple levels (at the level of the participant, assessor, and analyzer) becomes essential.

4.10. Fidelity in delivering the meditation-based interventions

For MBR to progress, it becomes vital that the intervention is delivered in a way it was traditionally intended to. For this to happen, the instructor must have adequate knowledge and experience (including the theoretical framework of meditation) about meditation and its delivery [40]. However, it has often been argued that researchers often do not provide an adequate description of the characteristics of the instructor (experience, certification, etc.), and the method adopted by them to ensure the fidelity of intervention delivery. These raise concerns about the methodological robustness of the research and also act as a roadblock in replicating the research findings [18,26].

4.11. Outcomes of the yoga and MBR

Although researchers have shown keen interest in studying the positive effects of yoga and meditation as alternative and complementary medicine for various psychological or physical health issues and have tried to develop various interventions based on them, the outcome measures used are still in its infancy and lack validations [41,42]. Without understanding the ontological context of yoga/meditation and the set goals with which it was practiced, which is quite fluid (enlightenment, self-actualization, or to attain pure-consciousness), the outcomes assessed with a reductionist approach (e.g., present-moment-awareness erroneously considered to be a reflection of the psychological construct of the traditional Buddhist concept of mindfulness or any meditation) could act as a major methodological limitation in the ongoing research on meditation [43]. One major observation is that a large part of the meditation research emanates from the West rather than from the eastern part of the world from where these practices originated and where there is an inherent better understanding of its philosophy and the context. The meditation research seems to have picked up only a minuscule part of the innumerable practices available in eastern culture for research.

4.12. Assessment measures

One of the major methodological issues with MBR is to accurately measure the subjective experiences (mindfulness, relaxation, attaining consciousness, etc.) of the practitioners with any self-report or objective measure [14,17,26]. Further, the outcome measures often being used in the research are unspecific, hence cannot delineate the effects of various subtypes of meditative practices and their variations and differentiate the effect of one practice from another. Probably, the reason for this may be that the researcher may expect that the outcomes of such variations among the meditations would differ qualitatively, but not quantitatively. However, this may not be true. For instance, the Sahaj Samadhi Dhyana Yoga is specifically expected to provide relief from past traumatic memories and so, may be studied for its potential benefit in Post-Traumatic Stress Disorders [44]. Further, relying on the third persons' account for the first-persons’ experience is vulnerable to depict a distorted picture unless the former has a deep knowledge and experience in recognizing and differentiating the various subjective meditative experiences and their outward manifestations [41]. Similarly, the objective measures (neuro-psychological or electrophysiological findings) used to infer the subjective experience may not accurately capture the real change brought about by the meditation. For instance, it is challenging to accurately and reliably tap the mind wandering, or non-judgemental approaches towards ones' thoughts or other mental states with the help of neuro-imaging. Overenthusiastic and undue attempts to quantify or measure the psychological state of an individual (e.g. mindfulness, non-judgemental, awareness, mind in the present moment) through reverse-inference (a neuro-cognitive sciences' approach in which based on the functional neuroimaging findings [brain activation] the mental state of an individual is inferred) is liable to suffer from biases, a common pitfall in neuro-imaging based study, especially when the pieces of evidence are still evolving [45]. The construct validity of the measuring instruments has also been debated especially in the absence of any gold-standard reference instrument [14]. Further, the self-report questionnaires are also likely to get biased depending on the participants' expectancies and prior experience with any meditation (e.g. an experienced practitioner may be better able to delineate and describe ones’ meditative experience as compared to a novice) [16]. Thus, first-person qualitative research needs to be used more often in meditation research, especially for hypothesis generation.

4.13. Effect of yoga and meditation and sustainability of their benefits

As has been discussed, the effect of the meditation is also determined by several associated factors (such as preparatory measures and lifestyle modifications) [14,32]. Similarly, there are several latent factors such as duration of the informal practice of the meditation (e.g. being mindfulness), ones' personality traits, or expertise in the meditation, etc., and their interrelation, which may influence the overall effect of the meditation on the practitioners [17]. Moreover, an experienced practitioner of the meditation is vulnerable to exhibit the ‘Hawthorne effect’ (a tendency to perform better or report in an exaggerated positive manner about the effect of the meditation one has been involved in) [16]. Many studies do not account for, or at least adequately describe these latent variables, thus their results should be interpreted with some caution [17]. Some of these latent variables may be quite difficult to assess, such as depth of meditation and level of expertise over it, while others such as personality traits may be more easily measurable.

Further, in the absence of long-term follow-up studies assessing the effects of meditation on the practitioner, its sustainable effects are yet to be established. Factors like duration, quality of meditation, compliance with the recommended practice, and practice of more than one type of meditation during the same period are difficult to be ascertained, hence, could act as a hurdle in conducting longitudinal studies on meditation. Moreover, self-report measures are often subjected to recall bias.

4.14. Statistical analysis-related issues

MBR is also limited by factors concerning the statistical analysis. Literature shows that studies often do not employ an intention to treat (ITT) analysis while analysing data rather only show findings of the per-protocol analysis (those participants who followed the interventions as per the protocol), resulting in inaccurate results. This also overlooks the feasibility and acceptability aspects of a meditation-based intervention for various health conditions [3,26]. Hence, the issue of dropouts needs to be addressed more comprehensively. Furthermore, as several factors act parallelly, mediator and moderator analysis, though may be difficult to perform, should be attempted, whenever feasible, to explain and understand the benefits of the interventions [26,46]. Finally, baseline differences between the participants (within the groups or between active and control groups) are often not analyzed.

5. Measures to overcome the methodological limitations with the MBR and the road ahead

5.1. Operationalizing the definition of yoga and meditation

It is highly recommended that the MBR should operationally define the meditation under study (e.g. mindfulness, transcendental meditation), giving due consideration to its ontological definition and psycho-philosophical context [14,17,26]. This would ensure research fidelity and replicability of studies across different populations and study designs. Furthermore, the theoretical framework of the meditation should be taken into account while developing any meditation-based interventions and generating a testable hypothesis; this would facilitate the progress of MBR. Most of the research is focused on a specific kind of meditation practice and the research base needs to be widened and be more inclusive.

5.2. Component analysis of the yoga/meditative practice

Meditation involves many preparatory steps (breathing, postures, lifestyles and philosophical changes, etc.) and components (e.g., Qigong meditation includes self-awareness, stilling of the mind, and also raising the Qi energy/prana through a concentrative focus on breath and posture etc.). Hence, it is worthwhile to systematically analyze various components of the meditation (breathing exercises, asanas, chanting, meditation-proper [dhyana], etc.) and also to differentiate between otherwise seemingly similar activities (e.g. exertion due to body moments in Qigong from the exertion during the aerobic exercise), to be able to find out the mechanism of its action and its uniqueness [2,3,23,40]. Similarly, different modifications of a particular meditation type (e.g. mindfulness-based practices: ACT, MBSR program, MB-CBT, etc.) should be compared among themselves to be able to identify the key component of the modified meditations bringing about the desired benefit. Furthermore, exploration of the non-core component of meditation in various permutations and combinations with the core element of meditation could bring about significant insight into the effect of the individual components of meditations and the most effective combinations among them. For instance, the non-attentional component of mindfulness meditation such as breathing pattern (chaotic Vs rhythmic), sensory involvement (deprivation Vs enhancement), moral discipline, and other contextual effects, etc. should be studied alongside its core component (e.g., attentional and non-reactive engagement with the subjective experiences). One of the potential study designs to realize these objectives are a single-case experimental design where a single participant, usually an expert in meditation, is assessed on multiple occasions after performing different elements of the meditation or different types of meditations [47]. A similar methodology could also be employed on a small number of participants in an experimental design.

5.3. Participants’ recruitment using the opt-out approach

To overcome the inherent flaw with the participants' recruitment in the MBR and issues with its generalization, some researchers have suggested an opt-out approach to participants' recruitment to be more suitable. This approach has been found to be associated with a higher recruitment rate, adherence rate, and better compliance with the intended meditation-based interventions. Since the opt-out approach require a lesser activation level (required confidence over ones’ ability to change the behavior) and reasonable baseline expectation and interest from the intervention, such participants are more representative of the population, thus making the research findings more generalizable and closer to real-world scenario [33,34].

5.4. Setting up an effective control/comparison group

To decipher the benefits of meditation and compare its impact with other practices or placebo, we need to conduct research having effective control arm(s). Some of the limitations of the wait-list control design (demand characteristics) could be surmounted by employing the dual blinding method (in which the participants are unaware as to which arm of the study is the intervention arm) [48]. One such example is the health enhancement program, which has been used to serve as a control arm for MBSR [49]. The latter differed from the MBSR in comprising of music therapy (Vs body scan), nutrition education (Vs sitting meditation), and just walking (vs mindful walking). Other useful strategies could be dismantling strategies (where the practice is systematically broken down into various parts and the key component is replaced by a neutral activity) and by having several comparison groups [50].

5.5. Adequate description of the study

For the MBR to progress, the researchers must provide adequate descriptions of the study design including the process of obtaining the informed consent; characteristics of the participants, trainers, and the investigator (including their affiliation with any meditation-based organization, experience in meditation, knowledge, and expertise in the field of contemplative sciences, etc.) and most importantly about the meditative practice under study (FA, OM, GM, ASD or mixed; kind of breathing; the level of exertion; modification of the practice based on age and culture of the participants, preparatory steps, etc.) [18,26]. It is also prudent to inform participants beforehand about the meditation-based intervention under study and its potential effect in a neutral yet specific manner. For instance, telling the participants that the technique would ‘train one in stabilizing one's mind’ in place of telling them they are going perform some meditation, would prevent unreasonable expectancies among them, which otherwise could act as a confounder. The researcher can use certain instruments such as credibility and the expectancies questionnaire (CEQ) to assess the baseline expectations of the participants on the effectiveness of the meditation [51]. Similarly, the fidelity of delivering meditation-based interventions by the instructor could be examined by videotaping the training sessions.

5.6. Ensuring fidelity of intervention delivery

To overcome the issues of the fidelity of delivering meditation-based interventions by the instructors, study protocols must adequately describe the mode of delivery of the intervention, who would provide the intervention, and how?; similarly, pilot testing of the intervention delivery, video-recording of the therapy sessions, feedback from the experts and necessary course corrections become crucial.

5.7. Validating outcomes and measures

The outcomes used to assess the effectiveness of the MBR must be specific and have adequate construct and content validity. Similarly, the outcome measures (questionnaire: self-report or interview-based) developed to assess the effectiveness of the interventions should have criterion validity. It can be achieved by incorporating the theoretical framework of meditation while developing the instruments.

Further, to precisely measure the outcome of meditation, a multi-model approach would be more robust. For example, one may couple the subjective experience of the practitioner (e.g. current level of mind-wandering, affective state, etc.) with an objective measure. One of the useful methods is an ‘Ecological Momentary Assessment’ (EMA), where the practitioner while meditating is asked about his/her current state of mind-wandering and simultaneously given a cognitive task (breath counting or a series of words, etc.) to assess their cognitive performance; such objective measures could also be complemented by a neuro-imaging or electrophysiological studies [52]. Further, the first persons' account about the meditative experience should be assessed by an experienced third person rather than someone lacking sufficient insights and expertise into the meditation and its effect. This would help in assessing the nuances of the meditative experience of the practitioner. Additionally, the experience of the first person, especially in a novice, can be indirectly assessed by enquiring about his/hers' behavioral change by the second-persons’ account (a family member or an instructor who is also in close terms with the practitioner). This could avoid recall bias and subjective biases while reporting ones' experience, thus provide an accurate result. It should be possible in at least some of the research studies though may be difficult in many others.

5.8. Trait-state interaction and longitudinal assessment of the effect of the yoga/meditation

The effectiveness of the meditation to a larger extent is determined by both the quantity and quality of the meditation practice. Future research must endeavor to capture these aspects, at least over some time, in a practitioners’ routine life. This should also account for the duration of each meditation practices separately if one is involved in more than one type of meditative practices; duration of both the home-based practice and the retreat course; and informal practice during the day to day life (e.g. practicing mindfulness or breathing exercises during the period of stress and cognitive task) [26]. Moreover, the personality characteristics of the practitioners and other psychological factors that facilitate their persistency in the practice should be assessed. Furthermore, factors related to compliance with the meditation need to be assessed.

The impact of meditation should be longitudinally assessed at different time points. Although it might be difficult, but not insurmountable. To avoid the recall bias, a daily reconstruction strategy could be utilized in which the participants systematically reconstruct their activities and experience of the preceding day [53]. More robust research design and assessment measures that would be less burdensome and effortless for the participants should be developed to capture the experiences of the practitioners.

5.9. Monitoring for any adverse effects of the yoga and meditation-based interventions

It is not uncommon that various meditative practices are inadvertently advertised as a technique that is free of any potential side-effects and contra-indications, which may not be true for all the meditations and all the participants [37]. Hence, a robust research methodology should include careful participant selection, explicitly describing the exclusion criterion of the study, and adequately informing the participants about the potential side-effects or adverse effects of the practice while obtaining their consent. Having a mental health professional and yoga expert on board could address this issue to a larger extent [26]. Till the sound evidence for all meditation-based interventions in mental health problems is established, it is imperative that the individuals, particularly those with mental health problems, should also be adequately informed about the more conventional and evidence-based interventions (e.g., CBT for depressive disorders or aerobic exercise/physiotherapy for the pains).

5.10. Statistical measures

Despite randomization, considerable baseline differences may exist among the participants, such cofactors influencing the response of meditation in the real world should be analyzed by the appropriate statistical tests [54]. In certain cases, especially when sample sizes are small, it is advisable to provide findings of both the ITT and per-protocol analysis to bring about greater insights into the effectiveness of the interventions. Furthermore, to be able to accurately measure the impact of meditation (e.g. attainment of the mindfulness or level of cognitive performances) on the participants, mediator (e.g. duration of practice, participants’ baseline interest for the meditation, etc.) and moderator analysis (e.g. change in the lifestyles, association with a meditation related organization) should be performed [55].

6. Limitations

The current review has a few significant limitations. Firstly, we have only included records that are available in English; therefore, we might have missed some of the crucial literature available on native/regional languages, which otherwise could have strengthened the current review. Secondly, most of the included papers were commentaries/perspectives, which were personal accounts of the researchers that might not have undergone strict scientific scrutiny; therefore, the inferences drawn from them might not be conclusive. Lastly, our review was focused mainly on the qualitative aspect of the MBR, including methodological issues, adverse effects with the meditation practice, the effectiveness of the MBI, statistical conundrums, etc. Hence, it could not comment upon quantitative data on the MBR and associated methodological concerns, which otherwise would have been valuable for the readers.

7. Conclusion

MBR has grown exponentially over the last few decades. Meditation-based interventions have been tried extensively for various psychological and medical conditions with some are backed by high-quality research. The ongoing COVID-19 pandemic and its psychological impact have opened a new research avenue to explore the effect of meditation (and meditation-based interventions) on general well-being and for its positive physio-psychological effect in the general public and clinical populations, respectively.

Despite some of the promising findings, MBR suffers from important methodological limitations. Major methodological limitations with the MBR include ambiguity about the definitions of the meditations, lack of testable hypothesis, difficulty with participants’ recruitment, issues with allocation concealment and blinding, difficulties in setting-up the comparative groups, assessing the characteristics of the participants, trainers, and investigators, lack of the validated outcome measures, statistical analysis, etc. However, these limitations are not insurmountable and can be overcome by robust research designs. Future research must take into account these findings while trying to generate evidence for yoga and meditation and interventions.

Authors’ credit statement

SG: Conceptualization ideas, development or design of methodology, maintaining research data, writing initial Draft.

AD: Conceptualization ideas, reviewing draft.

Authors’ disclosure

Dr. Anju Dhawan is a certified teacher of the Art of Living foundation with its head quarter based in Bengaluru, India. She conducts worskshop on Sudarshan Kriya Yoga and Sajah Samadhi Meditation. However, none of the authors report any financial or other incentives from any organization in writing this paper. Nothing to disclose.

Declaration of competing interest

None.

Appendix A. Supplementary data

The following is the Supplementary data to this article: