A protocol for systematic reviews of Ayurveda treatments

Abstract

This protocol is intended primarily for Ayurveda doctors who wish to take up systematic reviews along with an expert who has experience in doing such reviews. We have structured this protocol by incorporating the principles of patient treatment in Ayurveda, within the Cochrane framework, using Vitiligo as a model. The treatment section provides a comprehensive list of classical medicines used in the treatment of the disease. This will help in increasing the search terms. Such a list also helps to determine the needs of individualized treatment principles used in the trial and to assess the confounding factors. The search strategy includes an extensive listing of eastern data bases and hand searching. In Ayurveda, the titles of articles are not in the Population, Intervention, Control, and Outcome (PICO) pattern and sometimes the title and methodology do not tally. Therefore, a search of all types of studies is necessary to pool all the relevant publications. A data extraction form is proposed for use in assessing the quality of Ayurvedic studies. The form provides a template for performing evidence reviews of Ayurvedic interventions.

INTRODUCTION

Systematic reviews (SR) generally combine information from many relevant studies using meta-analysis, to provide more precise estimates of the treatment effects. They are considered superior to narrative reviews.[1] Cochrane reviews search journal publications and synthesize evidence in a structured format. Therefore, SR are the best evidence in biomedical health care. Ideally SR are done when multiple, similar studies are available. Meta-analysis, as part of SR, uses statistical methods to summarize the results of independent studies that are similar in interventions and outcomes. By combining information from all the relevant studies, a meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review. The largest collection of such reviews is the Cochrane Database of Systematic Reviews (CDSR), published quarterly as part of the Cochrane Library.[2] SR also appears in other journals. A protocol is mandatory before undertaking SR, as it helps to reduce the bias during the review. SR of Ayurveda treatments are occasionally published.[3–5] However, we have found that their protocols are not comprehensive in their search, their description of Ayurvedic etiology, interventions, or principles. This protocol is intended for Ayurveda doctors who wish to take up systematic reviews along with an expert who has experience in doing such reviews. We have structured these protocol incorporating principles of patient treatment in Ayurveda within the Cochrane framework,[1] using vitiligo (called Switra in Ayurveda) as a model.

Performing SR on Ayurveda interventions presents unique sets of methodological challenges. We have addressed them stepwise. We suggest that the protocol comprise of all subtitles beginning at the ‘background’ till ‘sources of internal support’ in the same sequence as discussed a little later in the text.

BACKGROUND

Switra, [6] known as Vitiligo in biomedicine, is a common problem associated with an intense social stigma in Indian society. The disease is more than a simple cosmetic problem. It is difficult to find alliances for unmarried youth in India because of its familial transmission.[7] There are instances of divorce if a wife develops the disease after marriage. Ideas such as, antisocial deeds committed in previous birth (life)[8] are linked to this disease; therefore, patients sometimes live in social isolation in India.

According to Ayurveda, the skin is an essential sense organ. Basic energy principles like ‘mobile natured energy’ (vata) and one of the five heat-producing activities similar to metabolism (bhrajaka pitta)[9] reside in the skin (called twak in sanskrit). As the skin covers the whole body bhrajaka pitta should be maintained in a proper state and it needs continuous care. Therefore, Ayurveda probably has multiple treatments available for skin care and dermatoses.

Definition and types (Nirukthi and Bheda)

Ayurvedic descriptions of Switra and Kilasa are synonymous.[10] However, some authors consider these two as subtypes.[11] Sushruta explains that Kilasa is a variety that affects only the skin.[12] Supporting his opinion, Madhava and Vagbhata view that Kilasa is stable (nirdista) as it remains at skin level (called the twakgata condition in Ayurveda). Charaka explains that Switra becomes firmly established when the basic energy principles (dosha) enter different levels of basic body tissues (dhatu), indicated by change in color of the lesions.[13] When dosha enters dhatu, the disease shows poor prognosis (asadhya).[14] Bhoja identified more subcategories on the basis of etiology (nidana); originating from the basic energy principles (doshaja) and from wounds (vranaja).[15] Sub-classifications in the doshaja variety are Daruna, Aruna, and Kilasa.[16] There are three types of energy principles (Doshas) involved in the balance or imbalance of the physiology, according to Ayurvedic theory: motion (Vata in Sanskrit — or ‘mobile natured energy’ indicating xerosis, light, cold, subtle, mobile aspects of physiology); metabolism,[17] (Pitta in Sanskrit — or ‘combustion, heat production’ indicating oily, corrosive, heat, foul smell, liquid aspects of physiology), and structure (Kapha in Sanskrit — or water, indicating the slow, cooling, binding, stabile aspects of physiology). When predominant physiological disturbance is of ‘mobile natured energy’, it is called vataja Switra. Similarly, the pittaja and kaphaja varieties are recognized. When the vataja variety is observed, the physiological disturbance is usually embedded at the blood tissue level (rakta dhatu). In the same way pittaja Switra spreads to the muscle tissue (mamsa dhatu) and kaphaja to the adipose tissue level (medo dhatu level).[18]

The stage of the disease and selection of medicine are based on the above Ayurvedic principles. We have explained these principles on the basis of clinical features, using biomedical terminology[19]

Etiology (Nidana)

The causative factors for skin diseases (kusta) and Vitiligo (Switra) are the same[20] and affect the same basic body tissue (dhatu) levels. Switra differs from other skin disorders by the normal functioning of all but the ‘skin tissue’ (twak)[21] resulting in discoloration of the skin (twak vaivarnyata), without discharge (aparisravi).[22–24] The dietary factors are also discussed in the causation of vitiligo. Seventeen ways of dietary intake are labeled as incompatible (virudha) diet.[25] Such incompatibility with the normal physiology of humans are determined on the basis of the habitat of the patient, food versus season, different combinations that are incompatible, and so on, which might trigger Vitiligo lesions. Excessive consumption of food that takes a longer time to digest (guru) and non-vegetarian and spicy foods are also an associated etiology.

Sinful actions (papa karma) done in the previous birth and current life and fighting with teacher (gurugharshana)[26] are frequently reported by Indian patients as a possible cause. Customarily Hindu religion recommends several compensatory rituals for the same. Many patients undergo these rituals before reporting to or during the treatment course.

Prognosis (Sadhya-asadhyata)[27]

Good prognosis (sadhya) patients have lesions of recent onset (nava), are not thickened (thanu), and show pigmented hairs (ashukla roma).[28] However, patients who are likely to have poor prognosis exhibit lesions on the external genitalia, lips, and tips of fingers (Anthejatam),[29] coalescing (samslista), and / or cover large areas of body (bahala)[30] or those appeared following burns (agnidagdhaja).

Treatment (Chikitsa)

Vitiligo (Switra) is chronic (deergha roga)[31] and should be treated immediately.[32] Classical textbooks of Ayurveda recommend four steps of treatment. The first step is purification therapies (shodhana karma) using herbal decoction of Psoralia corylifolia (bakuci kwatha) and Eurphorbia nerifolia (snuhi). This medicine is expected to induce multiple bouts of purgation.[33,34] In the next step oil massage using oil selected on the basis of patient examination (rogi pariksa) and disease (roga). Step 3 is exposure of lesions to the sun rays as long as the patient can tolerate (Soorya pada santhapam in Ayurveda). Patient receives Gruel (peya)[35] for the following three days. Ifbullae arise after a sun bath they must be punctured using a sterile needle. For the last three days, the patient receives decoction (kwatha) made of Ficus hispida (malayu), Pterocarpus marsupium (asana), Calllicarpa macrophylla (priyangu), Peusedanum graveolens (satapuspa), Coleus vettiveroides (ambhasa), and alkaline extract of Butea monosperma (palasa ksara), along with an alcoholic preparation of jaggery (the preparation is called phanitha in Ayurveda). On these three days the diet should be salt-free and should contain buttermilk.[36]

Ayurveda recommends any of the following medicines alone or in combination, following the four steps of initial therapy mentioned earlier.

Preparations that may or may not contain minerals

Topical applications (Lepa)

Ankollakadi lepa,[37] Avalgujadi lepa,[38] Bakucyadi lepa,[39] Balyadi lepa,[40] Bhallatakadi lepa,[41] Bhringarajadi lepa,[42] Gandhakadi lepa,[43] Girikarnika yoga,[44] Grhadhumadi lepa,[45] Gunjadi lepa,[46] Gunjaphaladi lepa,[47] Katukalabvadi lepa,[48] Manasiladi lepa,[49] Maricadi lepa,[50] Panca Nimbava Lepa,[51] Pathyadi Lepa,[52] Patrakadi lepa,[53] Putikadi lepa,[54] Talakadi lepa,[55] Triphaladi lepa,[56] and Vayasyadi lepa[57]

Herbal preparations similar to mixtures (Kashaya)

Bakuci beeja yoga,[58] Bakuci prayoga,[59] Bhadrodumbarikadi yoga,[60] Dhatryadi kwata,[61] Kakodumbarika kasaya,[62] and Khadiradi kashaya.[63]

Simple or compound powders made up of medicinal plants (Curna)

Bakucyadya curna, [64] Kakodumbarikadi yoga,[65] Khadira saradi curna,[66] and Pancanimba curna[67]

Herbalized ghee preparations similar to paste (Ghrita)

Dantyadi ghrita, [68] Mahamarkara ghrita,[69] Mahaneela ghrita,[70] Mahatiktaka ghrita,[71] Mahavajraka ghrita,[72] Neelaka ghrita,[73] Neeli ghrita,[74] Neelinyadi ghrita,[75] Somaraji ghrita,[76] and Tiktaka ghrita.[77]

Semi-solid preparations taken orally, by licking (Avaleha)

Bhallatakavaleha [78] and Vidangadi leha.[79]

Herbalized oil preparations (Thaila)

Aragwadhayadya thaila, [80] Citrakadya thaila,[81] Jyotismati thaila,[82] Kusta Kalanala thaila,[83] Kusta raksasa thaila,[84] Laghu maricadya thaila,[85] MahaVajraka thaila,[86] Manasiladya thaila,[87] Maricadya thaila,[88] and Visha thaila.[89]

When treating skin disorders Thaila may be applied topically with a massage (abhyanga), instilled into the nostrils in combination with an oil massage of the face, or could be given orally (pana)

Fermented solutions (Asava-Arista)

Kanakabindvarista[90] and Madhwasava[91]

Tablets (Vati / Gutika)

Swayambhuva Guggulu,[92] Thriphaladi gutika,[93] and Brhat Swayambhuva Guggulu.[94]

Preparations that always contain minerals

Formulations containing processed minerals and metallic salts (Rasousadha)

Candraprabha vati,[95] Galitakustari rasa,[96] Khageswara rasa,[97] Kustebhakesari rasa,[98] Medani sara rasa,[99] Pittala rasayana,[100] Talakeshwara rasa,[101] and Vijayeswara rasa.102

Combined formulations and single herbs that do not have any specific name

In addition to the above-mentioned treatment options, classical books list many more combined formulations and single herbs that do not have any specific names. These may be topical or oral medications; or herbal or herbomineral. As such preparations are described under the section vitiligo, many practitioners do prescribe them.

The classical teachings of Ayurveda do not specify explicit indications for the list of medicines given above. Overlap and ambiguity remain in the treatment guidelines.[103]

OBJECTIVES

When sufficient numbers of studies are available the objective could be:

To compare the purification therapies (shodhana karma) with the suppressive therapies (shamanoushadha) for treating people with vitiligo (Switra).

TYPES OF INTERVENTION

Intervention

Shodhana with or without suppressive therapies (shamanoushadha)

Any or all of following purification therapies (shodhana)[104] should be initiated: Controllable induced emesis (Vamana), controllable induced purgation (Virechana), nasal instillation of drugs (Shiro virechana), and blood letting (Rakta mokshana). These procedure-based therapies are used to expel the vitiated primary life forces (dushita dosha)[105] through the nearest outlet.[106] These procedures will be documented as practiced if the treatment procedures are within the broad outline of Ayurvedic fundamentals. As part of the intervention, the patients may receive additional suppressive therapies (shamanoushadha) to restore the equilibrium of thridosha\, sun exposure at any time of the day, and may be asked to follow diet restrictions.[107]

Control

Shamanousadha

Shamanoushadha is an oral administration of Ayurvedic medicines to keep the vitiated dosha in a normal state,[108] to achieve equilibrium of the thridosha appropriately selected, without attempting to expel them as in the shodhana therapy. This should be documented as practiced if the treatment procedures are within the broad outline of Ayurvedic principles. As part of the control, the patients may receive additional sun exposure at any time of the day and may be asked to follow diet restrictions.

TYPES OF OUTCOME MEASURES

There are three types of outcomes: primary, secondary, and adverse events. Most of the studies focus on multiple outcomes. The authors should identify the different categories of outcomes, to find the suitability of study for the review.

Primary clinical outcome

It is most important to look for a single primary outcome measure if possible, which will denote the ‘success’ or otherwise of the trial objectives. Other secondary outcome measures can be recorded if they are supported with evidence.

Examples in the case of vitiligo are

• Measures of Quality of life (using Skindex,[109] GHQ,[110] DLQI[111] or other) to assess the end result of any treatment, that is, patient satisfaction with their response to therapy[112]

• Partial or near total repigmentation of the exposed area

  Quantitative measurements using ′rule of 9′[113] or VASI[114] score

  Repigmentation described using digital photographs subjected to morphometric computer analysis[115]

• Reduction in the activity of the disease using the VIDA score[116] or as defined by authors

• Any objective measures if employed

Secondary outcome measures may be

• Repigmentation of non-hairy areas

• VETF assessment,[117] a tool that combines the analysis of the extent, stage of disease, and disease progression

• Repigmentation of depigmented hairs

Adverse events

In general non-RCT designs are favored for recording adverse events. Therefore, a review should look into the possible adverse events. If not mentioned this possibility should be discussed under the heading, on the basis of duration and power of the trial. The examples relevant in case of Switra are:

• Adverse events that required discontinuation of treatment

• Sudden increase in depigmentation attributed by the patients to the drugs administered

• Number of participants developing any other side effects during follow-up

• Number of patients developing complications of purification therapies (shodhana), for example,symptoms showing excessive and inadequate administration of oleation therapy (atiyoga and ayoga of snehana,[118,119], sudation therapy (swedana),[120,121] controllable induced emesis (vamana),[122,123] controllable induced purgation (virechana),[124,125] symptoms due to improper administration of vamana and virechana (vamana and virechana vyapath[126), nasal instillation (nasya), and bloodletting (dustavyadha).[127]

• Cochrane protocols[1] always mention serious adverse events leading to death, requiring hospitalization or prolongation of the existing hospitalization that are life threatening, or those that will result in persistent or significant disability or incapacity. Ayurveda reviews should look for serious adverse events that might have occurred during the clinical trial, and relevant review authors should correspond to explore the same.

Timing of outcome assessment

Vitiligo (Switra) is a chronic disease, sometimes considered to be life long, with varied response to therapy. Therefore, review authors should consider the short-term data in order to detect the rapid onset of improvement, which may be important for good adherence to medication (compliance). The longer term data should be considered for primary end points.

CRITERIA FOR CONSIDERING THIS REVIEW

Types of studies

• Randomized Controlled Trial (RCT)

• Non-randomized Controlled Trial (NRCT)

• Quasi Randomized Controlled Trial (QRCT)

• Controlled Trial (CT)

• Controlled Before and After Study (CBA)

• Before and after comparison

• Case series

• Non-inferiority trial[128]

The definition for studies one to seven mentioned above should be based on Cochrane hand book 5[1]

Type of participants

Adults and children with Switra, as defined by Charaka[10] and with Kilasa as defined by Madhava[24] and Sushruta[12]

SEARCH METHODS FOR IDENTIFICATION OF STUDIES

At least two authors should attempt to find all relevant studies regardless of language and publication status (published, unpublished, in press, and in progress). Following are the general guidelines.

Electronic searches

Following databases should be searched

• Complementary and alternative medicine (CAM) databases HerbMed[129]

  Cochrane Complementary Medicine trial register[129]

  Annotated Bibliography of Indian Medicine[3]

• Open access data bases

• Scholarly exchange is an open access movement that provides more than 17 links for data bases.[130] Its directory lists over 3200 journals

• Free Medical Journals data base[131]

• More than 58 official publications (journals) of various Indian societies / associations[132]

•Open Access and Institutional Repositories, with E Prints ((http://www.eprints.org/), which has over seven important links[133]

•Main stream Allopathic databases

PubMed[134] and CAM on Pub Med

Cochrane library[2]

•Indian databases

National Institute of Science Communication and Information Resources (NISCAIR) and the NISCAIR project register[135]

IndMED[136]

•Other sources

Hand Searching

Bhagel's thesis list — researches in Ayurveda[137]

Ayurvedic journals published from India[138]

Conference proceedings — Search the proceedings of Ayurvedic / allopathic conferences.

Hand searching identifies studies in journals that may not have been indexed in any electronic database or may have been indexed in such a way that database searching is impractical. Comprehensive hand searching requires knowledge of the subject area and of the journals most likely to publish the research articles. Apart from the published literature, hand searching also helps to find the ongoing studies that are not published due to negative results. Our article on search methodology elaborately explains hand searching.[139]

Personal communication

The following groups may be contacted to get leads on publication and ongoing research publications

• The ISM Collaboration e group (TheISMCollaboration@yahoogroups.com)

• Contact individual researchers — for unpublished and ongoing studies

• The opinion from Ayurvedic luminaries and experienced senior teachers who are respected by one and all

• Contacting Ayurvedic associations and professional bodies[140]

Mining the reference lists

Search Strategy

Cochrane hand book version 5 gives the outline for developing a search strategy,[1] for example, terms to search for health condition of interest, for example, Psoriasis, AND intervention of interest, for example, herb *plant* AND type of study, for example, case report. Maximum key words should be identified and listed based on the PICO analysis[141] to search the electronic databases. A table of controlled vocabulary of CAM terms[142] as available in various electronic databases should be developed on the basis of PICO analysis.

Searching the databases through PICO (Population, Intervention, Control, and Outcome) helps to narrow the search item. Population stands for disease. In case of Switra (Vitiligo), synonymous words are identified and used to find the relevant articles (e.g., in Ayurveda switra, kilasa, sweta kusta are the synonymous words for vitiligo). The subtypes of switra are also added as keywords (daruna and aruna). As the word switra is written with different spellings in English, possible ways are also mentioned as keywords for search (switra, swithra, shwitra, shwithra, etc.). In a similar manner the other components of PICO should be identified and listed [Table 1].

Table 1.

Population, Intervention, Control, and Outcome table showing arrangement of keywords for Switra (Vitiligo)

METHODS OF REVIEW

Trial selection

Two authors should scan the results of the search strategy and retrieve the full articles that are potentially relevant. Studies that do not meet the eligibility criteria should be excluded, and the reason stated under ′characteristics of excluded studies′. They could be scrutinized for possible multiple publications using http://spore.swmed.edu/dejavu and http://invention.swmed.edu/etblast.).[143] none Both authors should independently assess each potentially relevant trial for inclusion in the review. Disagreements, if any, should be solved by discussing with a senior person experienced in SR. Those not meeting the eligibility criteria should be excluded and the reasons stated under ′characteristics of excluded studies’.

Assessment of methodological quality

The methodological quality should be evaluated independently by at least two authors. In case of disagreement between the authors; such issues should be discussed with a senior member who is experienced in conducting clinical studies. We recommend the methodology article by Narahari and Ryan et al., as the standard.[19]

Assessing ‘black box design’[144]

Ideally and wherever practicable, authors should look for components of the CONSORT model.[19] This article has discussed the general structure for a black box design in Ayurveda as applicable to dermatological disorders.

Assessing patient selection

In each study the patient selection criteria is an important issue that would influence the outcome. Therefore, Ayurvedic principles of treatment-guided patient selection should be discussed in the methods section. Inclusion and exclusion criteria should be supported by classical references. The generalizability[145] component is difficult to conclude because Ayurveda emphasizes on individualized treatment. However, an attempt must be made to relate the outcome, on the basis of the CONSORT model of Ayurveda.[19]

Assessing for blinding of evaluators

i) Assessing for blinding of evaluators should be on the basis of evaluating multimodal patient-specific approaches as narrated in JACM[19] and Chopra A et al.[146] When blinding of investigators and / or patients is not possible, methodological quality should be assessed, as narrated by Chaudhury RR, Thatte U, and Liu Jianping.[147] This is the same as, ‘one administers the medicine and other assesses the response’ narrated by J Vas, Mιndez C, Perea-Milla E et al., for acupuncture.[148]

ii) Studies should be assessed on the basis of:

1. Was follow-up complete?[145] What was the attrition rate and reasons for the same?

2. Description of withdrawals, dropouts[145]

3. Power (sample size) of the study[149]

4. Was the recruitment rate realistic?

5. A multiprofessional clinical contribution to the trial is an additional value

Data extraction

Two authors should independently extract the data from the trials using an ‘Integrated Data Extraction form’ (IDE) [Table 2]. In case the data in the published article do not suit the IDE, it may be documented separately. Contacting authors of trials with insufficient or missing data should be attempted, to get more information. Disagreements, if any, must be solved by referring to the trial report and through mutual discussions or with Seniors experienced in trials.

Table 2.

Integrated data extraction form

Data analysis

Two authors including a statistician should analyze data by using appropriate software, if necessary. Dichotomous data should be presented and combined using relative risks, continuous data summarized by arithmetic means and standard deviations data should be combined by using weighted mean differences; both should be accompanied by 95% confidence intervals. Where continuous data were summarized using geometric means, authors should combine them on the log scale using the generic inverse variance method and report them on the natural scale. Medians and ranges should only be reported in tables. Arithmetic means and standard deviations should be used to summarize continuous data, when the data are assumed to be normally distributed; however, sometimes these summary statistics are incorrectly used when the data are not normally distributed. Therefore, when arithmetic means are reported, and the scale is naturally bound at zero (measurements of the outcome cannot be negative numbers); authors should check the normality of the data by calculating the ratio of the mean over the standard deviation. If this ratio (mean / standard) is less than two, then it is likely that the data are skewed and so should not be combined in a meta-analysis. Once the quality is assessed, the influence of the quality on the effect estimates (summary odds ratio, etc.) could be evaluated by a subgroup (stratified) analysis. Separate summary effect estimates could also be generated for studies that meet and do not meet the individual quality criterion. Heterogeneity among trials should be assessed by inspecting forest plots, to look for overlapping confidence intervals, applying the chi-squared test, with a P value of 0.05, indicating statistical significance, and using the I 2 test with a value of 50%, used to denote moderate levels of heterogeneity. If heterogeneity is detected and it is still considered clinically meaningful to combine studies, then a random-effects model should be used. Provided there are sufficient trials, authors should conduct a sensitivity analysis, to investigate the robustness of the results to the quality components. A funnel plot should be utilized to indicate publication bias, heterogeneity of results or differences in the methodological quality.[1]

Data synthesis

Heterogeneity in studies should be explored, and if sufficient studies are identified and included in this review, a meta-analysis based on the methodological quality of the studies is possible. For individual trials, wherever possible, mean differences (and 95% confidence intervals) should be reported.[1]

DISCUSSION

Systematic reviews synthesize the highest advancements in a given topic that are updated periodically. Therefore, it is customary not to undertake any SR if the Cochrane review already exists on that subject. Maxine Whitton's Cochrane Skin Group review on interventions for Vitiligo[150] was published in 2007. Szczurko and Boon listed 15 articles in their systematic review of natural health product treatment for Vitiligo, published in May 2008. However, we found that both these reviews did not include Ayurvedic trials. Furthermore, their search methods did not mention the data bases on the Indian subcontinent that generally index CAM journals published from India. Hence, we developed and focused this protocol to review trials in Ayurveda for vitiligo treatments.

The SR of Ayurveda treatments previously discussed only randomized controlled trials (RCT). They found little evidence to support their use. This was true in the context of biomedical approaches to treatment and assessment. Many advocate that RCT was not always possible in procedure-based therapies of Ayurveda. When possible it would be expensive to conduct RCT and such funding may not always be available in the clinical settings of Ayurveda. Furthermore, the current Ayurvedic practice of patient treatment is widely observed in India and her neighboring countries and must work at some level. Therefore, we developed this protocol to incorporate lower levels of evidence, but those built on the Cochrane model.

In this protocol we have listed 66 medicines under the treatment section. The main intervention shamanoushadha is compared with shodhana karma. Although these interventions when described in this manner sound very variable and difficult to standardize, it is the manner in which Ayurveda treats patients. The intervention ‘shamanoushadha’ has multiple components and any one of them can be used under the category depending on the presentation of the patient (roga and rogi pariksha). Ayurvedic principle recommends shodhana karma along with shamanoushadha as an ideal treatment. However, many centers currently practice only Shamanoushadha. Therefore, by comparing we are looking at the additional benefit of Shodhana Karma. Another option is to keep the two active treatments of Shamanoushadha in both arms of the SR. An additional advantage of listing these treatment options described in the ancient texts is to increase the number of search terms. Using a large number of search terms is essential to locate publications, particularly when they are scattered. An attempt made to create a comprehensive list of medicines helps to assess if the trial is inconsistent in using an intervention. Some reviewers[138] have considered the use of a different medicine during the trial as beyond the general guideline of the clinical trial, and are tempted to exclude the study from review. However, addition of a different drug during the trial may be a ′confound′ used to manage the patient in a better manner, because there is an emphasis on an individualized treatment mode in Ayurveda. In such instances the investigator should have used the one in the group of medicines recommended in the classical texts. Furthermore, the listing of drugs in this section helps to explore if different centers have used the guidelines of different classical texts for treatment of the given disease.

We suggest that a review authors’ main desire should be to identify the RCT, as this offers the greatest potential for reducing bias. However, there is a case for including good quality case series and other observational studies listed in the type of study section mentioned earlier in the text. There are sophisticated statistical techniques for combining such studies with RCTs, such as, multilevel modeling. The meta-analysis of observational studies is as common as the meta-analysis of controlled trials.[151] SR on Ayurveda have found only a small number of studies that are heterogeneous in nature.[3–5] Therefore, other types of studies will also have to be explored for conducting SR in Ayurveda. Case reports and expert opinions can be used for background information and interpretation, but must not be included in the main analysis.

Sir Michael Rawlins, Chairman of NICE, the UK Advisory Body on Efficacy of Therapy, delivering the Harverian Oration before Fellows of the Royal College of Physicians, in October 2008, discussed the pros and cons of the levels of evidence and RCT.[152] Where the effects of treatment are large and dramatic, RCTs may be inappropriate.[153] RCT also has limitations to generalize the outcome to patients belonging to different ethnic backgrounds, the severity of the disease, and when associated with comorbidities. Observational studies tend to provide larger treatment effects than RCTs.[151] According to the Cochrane hand book-5 it may sometimes be appropriate to conduct an SR of non-randomized studies, of the effects of healthcare. In case the course of a disease is uniform or the effects of an intervention are dramatic then it is unnecessary and unethical to conduct RCTs.[1] We have also found that in evaluating surgical treatments and the efficacy of therapeutic devices and procedures in biomedicine, non-randomized studies are considered in SR.[154] Non-randomized studies include experimental studies (such as quasi-randomized trials) and observational studies with controls (such as controlled before–after studies, concurrent cohort studies, and case–control studies) or without concurrent controls (such as before-after studies, cross-sectional studies, and case series).[149] The most published articles (68 to 87% of feature articles and brief communications in Annals of Internal Medicine, BMJ, and The New England Journal of Medicine) are non-randomized studies.[129] Furthermore, the proponents of the World Health Organization (WHO) recommendation argued that RCTs were designed, and continue to be used, for single intervention drug studies. Insistence on the unmodified use of RCTs to establish an evidence base of Ayurveda fails to comprehend the complexity of Ayurvedic treatment methods.[144] Nahin RL, Senior Advisor for Scientific Coordination and Outreach of NCCAM advocates a full spectrum of studies, without identifying the underlying mechanism of action for each intervention, provided there is a clear, clinically relevant end point.[153]

Searching for all types of studies is essential because the title or abstract of Ayurvedic articles do not always mention the types of studies. Wootton[155] listed electronic databases containing reliable experimental data. However, they do not necessarily contain research publications relevant to Ayurvedic SR. SR on Ayurvedic topics produced by Park and Ernst,[4] Singh et al.,[3] and Agarwal et al.,[5] showed a poor yield of articles in most of these sites. Szczurko and Boon in their systematic review of natural health product treatment for Vitiligo listed only 15 articles, none from Ayurveda.[156] They also searched ‘fee’ databases to identify articles.[150] Considering these inadequacies we developed a search strategy exclusively for Ayurvedic articles (www.systematicreviewinayurveda.org).[139] Considering the contemporary publications of Ayurvedic clinical studies, every subject may not have a sufficient number when classified under ‘type of study’. Therefore, we suggest including all types of studies during the search. Most of the Ayurvedic study titles are not in PICO pattern. Occasionally quazi randomized studies are described with no indication of the study type in the title. The number of studies is small compared to biomedicine. Sub-classification of studies on the basis of study type, therefore, can be done only after pooling the literature. If there a sufficient number of studies is available SR can be undertaken of the basis of the type of study. If the numbers are not adequate a scoping review could be performed.

This protocol is essentially intended to evaluate research publications on Ayurveda that are at a lower level than RCT. A debate continues on whether RCT is applicable to all treatment methods of Ayurveda. Therefore, non-RCT study designs have a role in clinical studies. As only a small number of RCTs are available, SR should be conducted on other types of studies, to synthesize the evidence available. A recommendation out of such an SR would help to undertake new research, to upgrade the evidence level. Pointing a finger toward a medicinal system that has existed for thousands of years as having no evidence may not always be correct. Unless the medicine had some beneficial effect it is unlikely that over 700,000 practitioners are still using it in their private practice, when competitive options are freely available. Hence, SR based on the protocol discussed in this article has the scope to bring to light the current level of evidence to these practitioners and thereby motivate them to conduct better quality studies.