Abstract
Background
Acupuncture has been widely utilized for irritable bowel syndrome (IBS). However, heterogeneity is large among therapeutic strategies and protocols. The aim of this study was to propose some down-to-earth recommendations and establish an optimized protocol for acupuncture practice in IBS.
Methods
A panel of 74 traditional Chinese medicine (TCM) acupuncturists participated in clinical issue investigation. Subsequently, systematic reviews concerning acupuncture for IBS were screened within 3 databases. An initial consensus questionnaire was formed from the results of clinical issue investigation and literature review. Ultimately, a Delphi vote was carried out to determine these issues. 30 authoritative experts with extensive experience were requested to respond with agreement, neutrality, or disagreement for the items. Consensus achievement on a given item was defined as greater than 80% agreement.
Results
Following a 2-round Delphi survey, there were 19 items reaching consensus; of which 5 items (26.32%) achieved thorough consensus, and significant agreement was reached for the other 14 items. These items can be classified into the 3 major domains: 1) clinical outcomes that acupuncture can bring for favorable intervention population (5 items), 2) suitable therapeutic principles and parameters of acupuncture (13 items), 3) possible adverse events in the treatment (1 item).
Conclusion
Without any ready-made guidelines and lacking of homogeneity in the published literatures, such expert consensus could be valuable for TCM acupuncturists in daily practice and patients with IBS to obtain appropriate and standardized acupuncture treatment. In addition, it also points out the clinical focus which need to be further explored in future trials.
Keywords: Acupuncture, Abdominal pain, Defecation, Expert consensus, Functional gastrointestinal disease, Traditional Chinese medicine
1. Introduction
Irritable bowel syndrome (IBS) is a highly prevalent functional gastrointestinal disorder worldwide, affecting approximately 7~15% of the global population,1,2 and more common among females and young people.3 Patients with IBS agonize over chronic recurrent abdominal pain and bowel disturbance, leading to a prominently negative impact on quality of daily life and work productivity.4 Due to its high prevalence, IBS wreaks substantial medical and financial havoc. It has been estimated conservatively that there are 3.1 million healthcare visits and over $1 billion direct costs associated with IBS annually in the United States.5,6 Hence, effective treatment plays an important role in quelling the unfavourable social and economic impacts of this disease.
Due to the heterogeneity among diverse subtypes of IBS, there is still a lack of widely adopted algorithms.4 Nowadays, antispasmodics, laxatives, and antidiarrheals are the three main kinds of first-line medications for ameliorating IBS symptoms, which are recommended by the guidelines.7,8 However, they usually exert temporary effects and may lead to several potential side effects, especially after long-term administration.4,9 In addition, several other agents, such as 5-HT3 antagonists and 5-HT4 agonists, have beneficial effects for severe IBS, but may cause a high risk of drug-associated serious adverse events including ischemic colitis and cardiovascular events, which restricts their utility respectively.10,11 Given the existing treatment gaps, plenty of suffering patients would like to seek for alternative treatments.12 Acupuncture, as a major non-drug modality of traditional Chinese medicine (TCM),13 has been extensively applied in treating functional gastrointestinal diseases including IBS.14,15 In clinical practice, acupuncture is regarded as a promising physical therapy that can relieve chronic painful states,16 which may be especially helpful for IBS patients, in light of that recurrent abdominal pain is the most universal complaint and frequent reason that drives healthcare assistance.17 As a result, numerous IBS outpatients have received acupuncture treatment or ever consulted acupuncture practitioners for help not only in China but also in western countries.18
In spite of its common application in clinic, previous research evidences on acupuncture for IBS are still relatively scarce and in conflict. On the one hand, a previous Cochrane systematic review (SR) composed of 17 randomized clinical trials (RCTs) indicated that acupuncture exerted more beneficial effects than pharmacological interventions,19 which was validated again in the latest published study.20 On the other hand, the sham acupuncture-controlled trials suggested that acupuncture achieved favourable efficacy but did not show therapeutic advantage over sham acupuncture.21,22 At present, it is still controversial on whether acupuncture can bring ideal clinical outcomes for IBS patients. Furthermore, the parameters of acupuncture and acupoint selection also vary greatly in diverse literatures, which may confound the evaluation of the actual efficacy of acupuncture. Distinct from other pharmacological therapies, acupuncture is a kind of intricate intervention whose therapeutic effects are influenced by a battery of factors.23 But up to now, there is no endeavour to explore an optimal and standardized acupuncture therapeutic protocol for IBS. Meanwhile, clinical acupuncturists also have their own individual accumulated experience in daily practice, which may be worthwhile to summarize and analyse.24 It is essential to utilize all the information for better designing new trials to evaluate the true acupuncture efficacy for IBS. Because it is difficult to reach the corroborative conclusions on the basis of the current RCTs and SRs, an expert consensus study is likely to give more comprehensive and direct solutions for intricate clinical issues.25
On this account, aiming to provide some practical recommendations for acupuncture in treating IBS which may be taken into consideration in future clinical trials as well, we invited a group of Chinese TCM acupuncture experts to carry out a multi-round consensus study via the Delphi technique for gaining agreement on the specific issues that acupuncture practitioners really concern about.
2. Methods
We took the following 3 major steps to develop this expert consensus. The first step was to conduct a 2-round clinical issue investigation for generating the initial expert consensus voting list. The second step was to retrieve literatures for the research evidence in this field. The third step was to perform a 2-round Delphi consensus vote to determine the final expert consensus statements. A summarized work flow diagram of the study is shown in Fig. 1.
Fig. 1.
Work flow diagram of consensus development process.
A steering group was built beforehand for designing the concrete schedule, providing operational guidance, and coordinating the entire process. The steering group was made up of one senior acupuncture expert (Cun-Zhi Liu), two clinical acupuncturists (Xin-Tong Su and Ling-Yu Qi), two methodological scholars (Wei Chen and Li-Qiong Wang), and two assistants (Na Zhang and Jin-Ling Li).
2.1. Clinical issue investigation
2.1.1. Development of the clinical questionnaire
The steering group put forward the initial clinical questions on this topic via a brainstorming method. Several online video conferences within the steering group were held for further discussion on these questions with the help of Tencent Meeting version 1.4.6 (Tencent®). After multiple revisions by 3 acupuncture experts (Hui Hu, Ying Li, and Lin Du) and a methodological expert (Wei Chen), the modified list of clinical questions was sent to the acupuncturists for clinical issue investigation.
2.1.2. Selection of the clinical acupuncturist panel
To gain the specific issues of acupuncture in treating IBS which should be provided viable recommendations on, a clinical panel of Chinese TCM acupuncturists who had the professional background and clinical experience were invited. The clinical panel not only participated in the 2-round clinical issue investigation but also made suggestions during the process. The individuals within this panel were clinicians in the field and members of an academic association, namely China Association of Acupuncture-Moxibustion. The invitation letters were sent by e-mail or WeChat (a universal Chinese instant messaging app) in advance, to make sure that they were willing and had enough time to join in. Those who replied and consented to take part were included in the clinical panel list.
2.1.3. Process of the clinical issue investigation
The clinical issue investigation was divided into 2 rounds and conducted during March 3rd to 27th, 2020. A questionnaire comprised of semi-open clinical questions was distributed via the online survey program (www.wjx.cn) in round 1. After analyzing and summarizing the results, the steering group sent the other questionnaire which included the feedbacks from the first round and the questions for the second round to the same clinical panel again. Similar to round 1, the results of round 2 were also analyzed and summarized. On the basis of these results, the items within the expert consensus voting list were developed according to the PICO (patient, intervention, control, and outcome) principle finally.
2.2. Evidence in the field
An electronic literature retrieval within PubMed, EMBASE, and Cochrane Library databases was conducted on April 2nd, 2020 using the following terms: (“acupuncture” OR “electroacupuncture” OR “scalp acupuncture”) AND (“irritable bowel syndrome” OR “irritable bowel” OR “irritable colon” OR “functional bowel disease”) AND (“systematic review” OR “meta-analysis”). The detailed search strategy applied in PubMed is shown in Supplementary material S1. Based on the search strategy established by the steering group, systematic reviews (SRs) on acupuncture therapy in treating IBS, restricted to English-published, with full-text obtainable, were included to provide relevant research evidence to the expert panel for better making judgements during the Delphi vote stage. Literatures were cross-searched by 2 researchers (Xin-Tong Su and Na Zhang) independently to make sure that all eligible articles could be identified. Using a pre-designed information summary table, two assessors (Jin-Ling Li and Ling-Yu Qi) extracted information from each paper independently. Grading of Recommendations Assessment, Development and Evaluation (GRADE) system was applied to assess the quality of evidence extracted from the SRs by the 2 methodological scholars (Wei Chen and Li-Qiong Wang). The quality of evidence could be divided into 4 levels, namely "high", "moderate", "low" or "very low".26,27 Inter-assessor discrepancies could be resolved through an arbitration by a third researcher (Cun-Zhi Liu). Then the relevant research evidences and their corresponding GRADE ratings were presented on the expert consensus voting list as the reference for making judgements. In addition, all the experts were encouraged to indicate any papers they thought were missing.
2.3. Delphi consensus vote
2.3.1. Generation of the expert consensus voting list
Integrating the results of clinical issue investigation with the relevant research evidences extracted from the included SRs, an initial expert consensus voting list for the Delphi survey was prepared. Aiming to help the experts better acquire the current information of acupuncture in treating IBS and making an accurate and objective judgment, each item within the voting list would be attached with the results of clinical issue investigation or the corresponding contents of research evidence.
2.3.2. Selection of the expert consensus panel
On the basis of a comprehensive review of published literatures and acupuncture textbooks in this field, the lead authors or editors were searched for and taken into consideration as potential candidates. Suitable authoritative experts in this part should possess a deputy senior title at least and exceeding 10-year acupuncture practical experience, and be fairly veteran with acupuncture treatment in IBS. After corroboration of their participation, the panel of experts would take part in the multi-round Delphi survey. Prior to the formal Delphi consensus vote, several remote video teleconferences between the expert panel and the steering group were held to introduce the entire workflow in detail and respond to the experts’ queries. To ensure confidentiality and independence, both the identities of the experts and their individual choices were kept in secret between each other so that they did not need to worry about whether they made contrary selections with others. Meanwhile, the experts were requested to hold an impartial attitude and make the objective judgement toward each item.
2.3.3. Process of the Delphi consensus vote
The consensus vote was implemented with the assistance of the Delphi method, which is a widely adopted structured process including 3 main distinguishing features: not requiring face-to-face contact, controlled feedback, and statistical expert response.28, 29, 30 As such, this method has been utilized successfully in previous expert consensus studies of acupuncture treatment.31, 32, 33 To make sure that the responses were collected entirely, experts were requested to fill in their real names in each round. The experts were instructed to make a judgement for each item by integrating the evidence with their individual knowledge and clinical experience. Comment boxes were also attached to the voting list to collect the reasons for their selections and provide experts with the opportunity to share their suggestions. Panelists could fill in the explanations for their choices on each item. These feedbacks contributed to further modifying the items within the process. The voting results of the last round would be shown in the voting list for the next round. The same online questionnaire tool (www.wjx.cn) was applied for producing the expert consensus voting list and collecting the responses from the expert panel. The consensus vote was conducted during April 5th to 30th, 2020.
A 3-option question including ‘Agree’, ‘Neutral’ and ‘Disagree’, was used to measure the experts’ attitudes toward each item. The results of Delphi consensus vote were displayed as statistic percentages. According to previous studies of this type, consensus achievement was defined as that the threshold of agreement amongst respondents should be more than 80%.34,35 If the level of agreement was ≤ 80%, it indicated that the item was still lack of agreement and needed the next round to either reach a higher level of agreement or no consensus achievement.28 In an iterative mode, the identical process was implemented among the same experts again. The general voting results, experts’ own choices, and anonymous qualitative comments of the last round were presented on the new voting list. This allowed the participants another opportunity to dig over the items in view of the whole expert panel’s response. To minimize experts’ fatigue and workload, items for which the level of agreement was < 50% in the last round would not be discussed in the next round, and the votes should not be more than 3 rounds.
2.3.4. Formulation of the final expert consensus statement
Based on the results of the Delphi consensus vote, the steering group took charge of drafting the initial manuscript at first. Subsequently, this manuscript was sent to all the experts who participated in the Delphi consensus vote for scrutiny. After further discussion and revision through several online meetings and e-mails, the final expert consensus statement was built.
3. Results
3.1. Clinical issue investigation
A sum of 100 individuals from different administrative regions of China were invited initially. Ultimately, there were 74 TCM acupuncturists (response rate: 74%) who expressed interest and participated in the first-round clinical issue investigation and 70 of them completed both rounds. The dropout reason of the 4 participants was lack of time due to their personal arrangements. Over half of the clinical practitioners owned a Doctor’s degree and more than 10-year acupuncture practical experience. The detailed demographics of the acupuncturist clinical panel are shown in Table 1.
Table 1.
Characteristics of the participants.
| Clinical issue investigation |
|||
|---|---|---|---|
| Variable | Round 1, n (% of 74 acupuncturists) | Round 2, n (% of 70 acupuncturists) | Expert consensus vote n (% of 30 experts) |
| Female | 46 (62.16) | 43 (61.43) | 11 (36.67) |
| Highest education background | |||
| Bachelor’s degree | 7 (9.46) | 7 (10) | 2 (6.67) |
| Master’s degree | 22 (29.73) | 20 (28.57) | 7 (23.33) |
| Doctor’s degree | 45 (60.81) | 43 (61.43) | 21 (70) |
| Professional title | |||
| Intermediate title | 20 (27.03) | 17 (24.29) | 0 (0) |
| Deputy senior title | 25 (33.78) | 24 (34.29) | 2 (6.67) |
| Senior title | 29 (39.19) | 29 (41.43) | 28 (93.33) |
| Acupuncture practical experience (years) | |||
| 3-10 | 23 (31.08) | 19 (27.14) | 0 (0) |
| 11-20 | 37 (50) | 37 (52.86) | 9 (30) |
| 21-30 | 10 (13.51) | 10 (14.29) | 9 (30) |
| > 30 | 4 (5.41) | 4 (5.71) | 12 (40) |
| Geographical distribution | |||
| East China | 24 (32.43) | 24 (34.29) | 9 (30) |
| North China | 21 (28.38) | 19 (27.14) | 10 (33.3) |
| South China | 4 (5.41) | 4 (5.71) | 1 (3.3) |
| Central China | 5 (6.76) | 5 (7.14) | 4 (13.3) |
| Northeast China | 12 (16.22) | 10 (14.29) | 3 (10) |
| Southwest China | 5 (6.76) | 5 (7.14) | 3 (10) |
| Northwest China | 3 (4.05) | 3 (4.29) | 0 (0) |
Following iterative discussion and modification within the steering group, the initial questionnaire of clinical issue investigation consisting of 23 semi-open questions was generated and sent out in round 1. The predominant contents of the questionnaire were made up of the following 4 aspects: 1) suitable intervention population (2 questions), 2) acupuncture principles and parameters (17 questions), 3) clinical outcomes (2 questions), and 4) adverse events (2 questions). The detailed information of these questions is shown in Supplementary material S2. After analyzing and summarizing the results of the first round, 9 semi-open questions got exact answers, and the other 14 questions were further explained in detail to the participants and asked again in round 2. Information from the 2-round clinical issue investigation was collected and used for developing the items within the Delphi consensus voting list.
3.2. Evidence in the field
The initial database search yielded 24 records in total. Among these, 17 articles were excluded by title and abstract screening. After whole-text detailed screening, we found that one Cochrane SR had an updated version and the same version was published in Cochrane Database of Systematic Reviews and the other journal simultaneously.19,36,37 Therefore, 5 eligible SRs were included in our study finally, including 3 traditional SRs37, 38, 39 and 2 network meta-analysis SRs.18,40 The contained results of interest were extracted out as research evidence and attached to the corresponding items on the initial expert consensus voting list. However, according to the GRADE system, the quality of many research evidences was rated “low” or “very low”. On the other hand, only a few items (5/19 items) could be affiliated with the corresponding evidence from these SRs. The other 14 items on the voting list without available evidence had to be provided with the results from the 2-round clinical issue investigation as reference. The detailed results extracted from the included SRs and their GRADE ratings are shown in Supplementary material S3.
3.3. Delphi consensus vote
In aggregate, there were 30 well-known Chinese TCM acupuncture experts (response rate: 100%) who accepted our invitation and fulfilled all the 2-round Delphi survey votes without dropout during the process. 28/30 experts (93.33%) owned a senior title and 21/30 (70%) had at least 20-year acupuncture practical experience. The detailed demographics of the expert consensus panel are shown in Table 1 and Supplementary material S4.
The initial expert consensus voting list included 19 items which were built on the basis of clinical issue investigation. After the 2-round Delphi vote, there were 5 of 19 items (26.32%) achieving complete consensus, and more than 80% expert agreement was reached for the other 14 items. Among these, expert consensus was achieved on 17 of 19 items in just one round (Fig. 2), and the other 2 controversial items near consensus in the first round were voted again among the same experts in the second round (Fig. 3), which also met the standard for consensus ultimately. According to the panelists’ feedback in the online questionnaires, no new item was put forward by the expert panel during the 2-round vote. Therefore, there were 19 statements in total compiled into the final version of expert consensus.
Fig. 2.
Items achieved consensus in the first round Delphi expert consensus vote.
Fig. 3.
Items achieved consensus in the second round Delphi expert consensus vote.
The 19 statements addressed the most relevant and debatable topics on acupuncture in treating IBS, which could be roughly categorized into 3 major domains: (1) therapeutic effects, (2) therapeutic principles and protocols, (3) possible adverse events. The detailed contents of the final expert consensus statements are shown in Table 2. The specific contents of the consensus reached can be summarized as below:
Table 2.
Summary of the statements from the expert consensus.
| No. | Statements | References | GRADE ratings | Delphi agreement |
|---|---|---|---|---|
| 1 | For patients with mild and moderate IBS, acupuncture is recommended to relieve clinical symptoms (recurrent abdominal pain or abdominal distension related to defecation, frequency and urgency of stool, fecal appearance, etc.). | 18,37, 38, 39, 40 | Moderate/Low/Very low | Round 1 |
| 2 | For patients with mild and moderate IBS, acupuncture is recommended to improve quality of daily life. | 37,38 | Moderate/Very low | Round 1 |
| 3 | For patients with mild and moderate IBS, acupuncture is recommended to ameliorate psychological and mental conditions. | Round 1 | ||
| 4 | For patients with mild and moderate IBS, relief of clinical symptoms can be maintained for 1 to 6 months after one course of acupuncture treatment. | 38 | Moderate/Very low | Round 2 |
| 5 | Based on the Roman Ⅳ Criteria, acupuncture is recommended for the 3 IBS subtypes (IBC-D, IBS-C and IBS-M). | See also in Item 1 | See also in Item 1 | Round 1 |
| 6 | It is recommended to conduct acupuncture treatment on the basis of syndrome differentiation from liver stagnation and spleen deficiency syndrome, spleen deficiency and damp obstruction syndrome, spleen and kidney yang deficiency syndrome, and damp-heat in spleen-stomach syndrome. | Round 1 | ||
| 7 | It is recommended to select acupoints on Spleen Meridian, Stomach Meridian, Large Intestine Meridian and Liver Meridian. | Round 1 | ||
| 8 | Acupoint selection is a crucial factor for the achievement of favorable therapeutic efficacy. It is recommended to lay stress on the application of specific acupoints. The recommended specific acupoints should be Tianshu (ST25), Zusanli (ST36) and Zhongwan (CV12). | Round 1 | ||
| 9 | It is recommended to choose Mu acupoints and Xiahe acupoints as the common species of applied specific acupoints. | Round 1 | ||
| 10 | It is recommended to choose Shu-Mu combination and Xiahe-Mu combination as the common applied specific acupoint combinations. | Round 1 | ||
| 11 | De qi is a crucial factor for the achievement of favorable therapeutic efficacy. | Round 1 | ||
| 12 | Course of treatment is a crucial factor for the achievement of favorable therapeutic efficacy. The recommended course of treatment should be 4 weeks. | Round 1 | ||
| 13 | It is recommended to choose uniform replenishing-reducing method as the common applied acupuncture manipulation. | Round 2 | ||
| 14 | It is recommended to select 4~6 acupoints per session. | Round 1 | ||
| 15 | The recommended duration time of needle retention should be 30 min per session. | Round 1 | ||
| 16 | The recommended treatment frequency should be 3 times per week. | Round 1 | ||
| 17 | It is recommended to combine acupuncture with other TCM therapies, such as moxibustion or Chinese herbal medicine, so as to improve the clinical efficacy. | Round 1 | ||
| 18 | It is recommended that acupuncture practitioners eligible for IBS treatment should own the TCM license and at least 3-year medical practice experience. | Round 1 | ||
| 19 | Adverse events are uncommon in the treatment of IBS with acupuncture. The possible adverse events include subcutaneous hematoma, abnormal post-acupuncture sensation (such as pain, numbness, etc.). | 37 | Low | Round 1 |
Abbreviations: IBS: irritable bowel syndrome; IBC-D: IBS with diarrhea; IBS-C: IBS with constipation; IBS-M: IBS with mixed symptoms of constipation and diarrhea; TCM: Traditional Chinese Medicine.
Statements 105 refer to the clinical outcomes that acupuncture may bring for the suitable intervention population. The consensus reached by the experts in this part is: Acupuncture can be recommended for mild and moderate IBS patients to relieve clinical symptoms, improve quality of daily life, and ameliorate psychological and mental conditions. Additionally, acupuncture is also recommended to be used in the treatment of 3 IBS subtypes based on the Roman Ⅳ Criteria. The efficacy produced by one course of acupuncture treatment can be maintained for 1 to 6 months.
Statements 6018, as the key part, primarily analyze the therapeutic principles and concrete protocols of acupuncture, including TCM theory based on, specific acupoint selection, acupoint combination, treatment frequency, etc. The experts recommend that acupuncture treatment should be conducted based on syndrome differentiation from 4 types of TCM syndromes. The specific acupuncture protocols recommended for IBS in this consensus are: Acupuncturists can select acupoints on the 4 meridians, namely Spleen Meridian, Stomach Meridian, Large Intestine Meridian, and Liver Meridian. The application of specific acupoints should be laid stress on. Tianshu (ST25), Zusanli (ST36), and Zhongwan (CV12) can be chosen as the main specific acupoints. Shu-Mu combination and Xiahe-Mu combination are the recommended specific acupoint combination methods. 406 acupoints are preferable to be applied per session. After routine skin disinfection, the needles should be slowly and vertically/horizontally inserted into the corresponding acupoints. The uniform reinforcing-reducing acupuncture manipulation can be conducted to elicit De qi for patients. Needles can be retained for 30 min per session. The ideal treatment frequency should be 3 times per week and the whole course of treatment would be better as long as 4 weeks. Other TCM therapies, such as moxibustion or Chinese herbal medicine are recommended to be in combination with acupuncture if necessary.
Statement 19 considers the general incidence of possible adverse events in the treatment of IBS with acupuncture. Even though there are some possible adverse events, the expert panel believes that acupuncture is a safe therapy and these adverse events are uncommon and tolerable.
4. Discussion
The purpose of this study was to achieve consensus among a panel of prestigious TCM acupuncture experts on specific clinical issues and develop a relatively optimized and standardized acupuncture protocol in the treatment of IBS via a Delphi consensus study. Ultimately, 19 items categorized into 3 main domains reached consensus. In view of the theoretical background of acupuncture, some statements in the final expert consensus are specific to the TCM meridian terms. Nevertheless, this expert consensus may still provide some practical and generalized recommendations for clinical acupuncturists.
According to the final consensus statements, most of the experts (>90%) agreed that acupuncture could be used to relieve clinical symptoms and improve the quality of daily life in mild and moderate IBS. The recommendations are consistent with the results of a previous clinical trial that compared acupuncture plus usual care with usual care alone. This study concluded that acupuncture could reduce IBS Symptom Severity Score and improve the proportion of successful treatment.41 Two SRs indicated that acupuncture produced a preferable therapeutic effect in improving IBS patients’ quality of life compared with sham acupuncture or western medicine.37,38 Of note, a latest published high-quality study including 531 patients found that acupuncture showed more effective results in reducing IBS symptoms and improving quality of life compared with pharmacologic therapies.20 In addition, these effects could last up to 3 months, which is also in line with our consensus statement that relief of IBS symptoms can be maintained for 1 to 6 months after one course of acupuncture treatment. As a disease with a sort of psychiatric disorder, refractory IBS symptoms can exacerbate patients’ coexisting psychological distress.4 The experts reached a complete consensus (100%) that acupuncture could ameliorate the psychological or psychiatric conditions of patients. In light of the absence of corresponding evidence, we suggest that the outcomes of mental state should be focused on in more future studies, so as to verify the efficacy of acupuncture in this respect.
Based on the classical Meridian theory, specific acupoints are specially defined points on the 14 main meridians, which may have their specific therapeutic effects. Specific acupoint selection is very critical that different specific acupoints should be applied in distinct diseases, and stimulation on relevant specific acupoints in a certain disease can result in more outstanding effects than other acupoints.42,43 Therefore, in particular, we laid stress on the application of specific acupoints and set 3 items to discuss these issues. The recommended specific acupoints in this consensus were Tianshu (ST25), Zusanli (ST36), and Zhongwan (CV12). Of note, this result is concordant with the founding of two SRs that these acupoints were 3 of the top 6 most commonly adopted acupoints.40,44 Although the 3 specific acupoints are always taken into consideration for the treatment of gastrointestinal diseases and supported by the Chinese Medicine theory, further clinical trials are still needed to corroborate their superiority over other acupoints in treating IBS. In addition, the overwhelming majority of experts acknowledged Mu acupoints and Xiahe acupoints as the common species of specific acupoints, and Xiahe-Mu combination as the common acupoint combination, which is in line with the traditional concept that Mu acupoints and Xiahe acupoints are the preferred choices in treating Zangfu diseases such as IBS.44 Because acupoint selection and combination plays a crucial role in enhancing the efficacy of acupuncture, we suggest that not only clinical practice but also new trial designing may refer to these expert consensus statements.
Due to its inherent characteristic as a kind of complex therapy, the optimal-defined conditions have impact on the maximal efficacy of acupuncture. Unfortunately, these fatal issues reported within the distinct papers vary extremely, which may cast uncertainty on the comparability with each other. It has been supported by published literatures that better clinical outcomes produced by acupuncture may be dose-dependent and influenced by appropriate acupuncture protocols.45 Nevertheless, this point is always overlooked by most of RCTs that focus much more on validity assessment but are of less note to the optimal algorithm.31 There were 5 items discussed to ensure the adequate stimulation of acupuncture for achieving the ideal therapeutic effects, including course of treatment, acupuncture manipulation, number of acupoints per session, duration time of needle retention, and treatment frequency. Although more than 80% of the experts considered that it was reasonable about these acupuncture parameters in the final statements, we suggest more studies to explore their reliability.
As a kind of chronic and recurrent disease, IBS patients may always need long-term treatment. Given this, there is no wonder that safety should be of great concern among patients and clinicians. Patients have to be faced with the increased risk of adverse drug reactions, especially after long-term utilization. The consensus agrees that adverse events are uncommon in the treatment of IBS with acupuncture, which is another superiority of acupuncture demonstrated by previous SRs.18,37 Therefore, physical therapy like acupuncture with few side-effects may be considerable for patients suffering from IBS for a long time.
Evidence-based medicine, especially RCTs and SRs, has an invaluable impact on medical research and clinical practice. Meanwhile, they may possess several intrinsic weaknesses simultaneously. Generally speaking, they require rigid inclusion and exclusion criteria to restrict the study population and may not be generalizable to common clinical treatment.26 Additionally, there are as well some clinical measures such as parameters of acupuncture and acupoint selection, which acupuncturists actually and urgently concern about in real practice but have not been answered by published articles. These existent complicated clinical issues are hardly solvable by RCTs and SRs in a short time, either. A study of expert consensus sets up a structured process to collect information from a series of semi-open questions of interest with controlled feedback, followed by multi-round expert votes to achieve agreement on specific issues.35 The Delphi methodology has a particular merit that it can even be carried out securely while current research evidence is insufficient or there exists uncertainty in a certain field.46, 47, 48 The Delphi survey can synthesize experts’ opinions in a high-quality and scientific way and provides a considerable modality in determining the solution to some controversial clinical issues.30 Notwithstanding, aiming to run this study more systematically, before the formal expert consensus vote, we carried out a two-round clinical issue investigation to comprehensively understand the real need for acupuncture in the treatment of IBS among acupuncturists from different regions of China. Simultaneously, we still sought evidence from SRs and presented it as well as the results from the clinical investigation to provide the expert panel with more information for making decisions.
As this expert consensus is for mainly Chinese acupuncture clinical setting, it is of necessity to illuminate why Chinese SRs were not included in this study. Before making the search strategy, the steering group discussed about this issue and consulted the methodological experts for their opinions. Eventually we decided to choose the English-published SRs from PubMed, EMBASE, and Cochrane Library databases given the following reasons: (1) Most of the retrieved SR articles published in English had already included the Chinese-published RCTs originating from Chinese databases such as CNKI, CBM, VIP, Wanfang, etc. Therefore, the expert panel members could still learn about the meta-analysis results of these studies. (2) Even though a number of Chinese-published SR articles about acupuncture could be searched, but most of them did not follow the PRISMA statement strictly.49 Due to the cherry-picking RCTs for meta-analysis, the results from these literatures are mostly positive, but their methodological qualities are rated “low” or “critically low” according to the AMSTAR tool.50 To avoid confounding the expert panel for making decisions, we did not choose the Chinese-published SR articles. (3) Both of the positive and negative results could be extracted from the English-published SRs, and their reporting and methodological qualities were relatively higher, which were helpful for the experts in making an objective selection for the items.
It is arguable about the method of expert selection in this study and that the evidences are insufficient (or absent) for many items under discussion. We totally consent that a multidisciplinary expert panel is preferable for ensuring the balance of perspectives to generate an objective consensus. Consensus reached by an expert board comprising participants who are professionals in IBS but do not necessarily practice acupuncture can achieve a higher credibility. However, this survey is about a complicated intervention whose primary intention is to acquire some pragmatic answers from reputable experts for specific clinical issues under the circumstance that the relevant evidences are really too scarce. Almost 70% of the items voted in the consensus study are closely associated with the concrete acupuncture procedures that are necessary to be standardized and optimized. These recommendations are also what acupuncture practitioners really want to learn about in daily practice. Without referable evidences, these items have to be discussed among experts with background and actual clinical experience on acupuncture. It is difficult for experts who have never practiced acupuncture in IBS’s treatment to make a judgement on these actual issues, especially in the absence or lack of available evidences from the published literatures. Therefore, only authoritative acupuncture experts were recruited in our study. Nonetheless, to counteract the probable bias as possible, we set a more rigorous criterion (over 80% agreement) for consensus achievement in our study rather than 70% or 75% in usual.51,52 Moreover, we double the sample size of experts needed, while the minimum allowed sample size in Delphi survey is 12~15.29,53,54 The final expert panel is also a mix of practitioners and academics. On the other hand, evidences are indeed very essential in establishing guidelines for clinical practitioners. For a long time, alternative and complementary techniques are criticized and prone to base on ideology, beliefs, and personal experience, rather than on proper and well-built evidence. Although numerous acupuncture clinical trials have emerged in recent decades, most of these studies only focused on assessing the efficacy of acupuncture, and the fact that acupuncture is one kind of intricate intervention whose therapeutic effects can be influenced by a series of factors is always overlooked. The concrete acupuncture protocols vary greatly in distinct papers, which may confound the exact interpretation of their results. Due to scarce or absent evidences in the field, the comprehensive experts’ opinions collected with the assistance of Delphi method may provide another source of reference for acupuncturists to base their treatment on at present. Although many items in this consensus are the specialist’s recommendations rather than a guideline, and their validities still need further verification, these items can point out the reference directions for future studies at least, and make researchers know what can be taken into consideration in better designing the new study protocol and which optimal acupuncture parameters should be further explored.
There are several potential limitations within the study. The major limitation is that only Chinese TCM as opposed to international acupuncture experts participated in the survey, although they came from different areas of China. In spite of the careful expert panel selection and the rigorous Delphi method followed, it is likely that the consensus does not cover the entire acupuncture community’s opinion. While TCM style acupuncture is probably the most commonly used style of acupuncture in many countries, it uses diagnoses and treatment techniques that are different from other traditional acupuncture styles that are practiced for example in Japan, Korea, Australia, UK, US, Europe and so on. Hence, the transferability of these TCM-based recommendations to other countries may be hindered by the circumscribed medical theory background. Another limitation is that most of the final statements were based on the individual opinions and clinical experience of experts. The expert opinion is regarded as the lowest valuable source of evidence level system.55 Thus, the agreement of a certain item is at a particular point in time and may be changed with emerging new evidence and experience. In addition, even though there may exist some diversity between doctors’ and patients’ opinions toward the treatment,56 our clinical issue investigation was conducted only among clinicians, which means the consensus was based on doctors’ general perspective but not patient specific.
Given the limitations mentioned above, the current survey is more a preliminary dialectical consensus than a proper evidence-based guideline in some degree, which needs to be interpreted with caution by the readers. Notwithstanding, we still look forward to emerging proper RCTs and robust SRs, which can verify the experts’ recommendations and further underlay an ideal expert consensus or clinical guideline. When more evidences can be provided, an updated version of multidisciplinary, international, and thoroughly evidence-based expert consensus survey will be feasible and indispensable.
Taken together, it is of necessity to provide patients suffering from IBS safer and more cost-effective therapeutic choices, and acupuncture is one of the promising alternative non-drug interventions in IBS treatment. Therefore, we delineated a therapeutic paradigm to improve standardization into clinical practice and trials given that current research evidence is fairly insufficient. The proposed recommendations are not claimed the best or the most correct ones, but do give TCM acupuncturists some pragmatic guidelines to polish and refine their treatment.
Acknowledgements
The authors would like to thank the experts for their active participation and valuable advice during the process.
Author contributions
Conceptualization: Xin-Tong Su and Cun-Zhi Liu. Methodology: Li-Qiong Wang. Software: Na Zhang. Validation: Ling-Yu Qi. Formal analysis: Li-Qiong Wang. Investigation: Xin-Tong Su. Resources: Cun-Zhi Liu. Data curation: Yu Wang. Writing – Original Draft: Xin-Tong Su. Writing – Review & Editing: Jing-Wen Yang. Visualization: Jin-Ling Li. Supervision: Guang-Xia Shi. Project administration: Cun-Zhi Liu. Funding acquisition: Cun-Zhi Liu.
Conflict of interest
The authors declare that they have no conflicts of interest.
Funding
This research was supported by the National Key Research and Development Program of China (Grant no. 2019YFC1712103).
Ethical statement
No ethical approval was required as this study did not involve human subjects or laboratory animals.
Data availability
The data that support the findings of this study are available within the article and its supplementary materials.
Footnotes
Supplementary materials associated with this article can be found, in the online version, at doi:10.1016/j.imr.2021.100728.
Contributor Information
Xin-Tong Su, Email: suxintong@126.com.
Cun-Zhi Liu, Email: lcz623780@126.com.
Appendix. Supplementary materials
References
- 1.Lovell R.M., Ford A.C. Global prevalence of and risk factors for irritable bowel syndrome: a meta-analysis. Clin Gastroenterol H. 2012;10:712–721. doi: 10.1016/j.cgh.2012.02.029. [DOI] [PubMed] [Google Scholar]
- 2.Sperber A.D., Bangdiwala S.I., Drossman D.A., Ghoshal U.C., Simren M., Tack J. Worldwide prevalence and burden of functional gastrointestinal disorders, results of rome foundation global study. Gastroenterology. 2021;160:99–114. doi: 10.1053/j.gastro.2020.04.014. [DOI] [PubMed] [Google Scholar]
- 3.Lovell R.M., Ford A.C. Effect of gender on prevalence of irritable bowel syndrome in the community: systematic review and meta-analysis. Am J Gastroenterol. 2012;107:991–1000. doi: 10.1038/ajg.2012.131. [DOI] [PubMed] [Google Scholar]
- 4.Ford A.C., Lacy B.E., Talley N.J. Irritable bowel syndrome. New Engl J Med. 2017;376:2566–2578. doi: 10.1056/NEJMra1607547. [DOI] [PubMed] [Google Scholar]
- 5.Everhart J.E., Ruhl C.E. Burden of digestive diseases in the united states part i: overall and upper gastrointestinal diseases. Gastroenterology. 2009;136:376–386. doi: 10.1053/j.gastro.2008.12.015. [DOI] [PubMed] [Google Scholar]
- 6.Everhart J.E., Ruhl C.E. Burden of digestive diseases in the united states part ii: lower gastrointestinal diseases. Gastroenterology. 2009;136:741–754. doi: 10.1053/j.gastro.2009.01.015. [DOI] [PubMed] [Google Scholar]
- 7.Quigley E.M.M., Fried M., Gwee K., Khalif I., Hungin A.P.S., Lindberg G. World gastroenterology organisation global guidelines irritable bowel syndrome. J Clin Gastroenterol. 2016;50:704–713. doi: 10.1097/MCG.0000000000000653. [DOI] [PubMed] [Google Scholar]
- 8.National Institute for Health and Care Excellence (UK). Irritable bowel syndrome in adults: diagnosis and management. UK NICE Website. https://www.nice.org.uk/guidance/cg61/chapter/1-Recommendations#pharmacological-therapy. Published 2017. Accessed 26 May 2020. [PubMed]
- 9.Ford A.C., Talley N.J., Spiegel B.M.R., Foxx-Orenstein A.E., Schiller L., Quigley E.M.M. Effect of fibre, antispasmodics, and peppermint oil in the treatment of irritable bowel syndrome: systematic review and meta-analysis. BMJ. 2008;337:a2313. doi: 10.1136/bmj.a2313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Chang L., Chey W.D., Harris L., Olden K., Surawicz C., Schoenfeld P. Incidence of ischemic colitis and serious complications of constipation among patients using alosetron: systematic review of clinical trials and post-marketing surveillance data. Am J Gastroenterol. 2006;101:1069–1079. doi: 10.1111/j.1572-0241.2006.00459.x. [DOI] [PubMed] [Google Scholar]
- 11.Ford A.C., Talley N.J., Schoenfeld P.S., Quigley E.M.M., Moayyedi P. Efficacy of antidepressants and psychological therapies in irritable bowel syndrome: systematic review and meta-analysis. Gut. 2009;58:367–378. doi: 10.1136/gut.2008.163162. [DOI] [PubMed] [Google Scholar]
- 12.Chey W.D., Maneerattaporn M., Saad R. Pharmacologic and complementary and alternative medicine therapies for irritable bowel syndrome. Gut Liver. 2011;5:253–266. doi: 10.5009/gnl.2011.5.3.253. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Su X.T., Wang L., Ma S.M., Cao Y., Yang N.N., Lin L.L. Mechanisms of acupuncture in the regulation of oxidative stress in treating ischemic stroke. Oxid Med Cell Longev. 2020;2020 doi: 10.1155/2020/7875396. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Schneider A. Acupuncture treatment in irritable bowel syndrome. Gut. 2006;55:649–654. doi: 10.1136/gut.2005.074518. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Li H. Acupuncture and regulation of gastrointestinal function. World J Gastroenterol. 2015;21:8304–8313. doi: 10.3748/wjg.v21.i27.8304. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ouyang H., Chen J.D.Z. Therapeutic roles of acupuncture in functional gastrointestinal disorders. Aliment Pharm Ther. 2004;20:831–841. doi: 10.1111/j.1365-2036.2004.02196.x. [DOI] [PubMed] [Google Scholar]
- 17.Cain K.C., Headstrom P., Jarrett M.E., Motzer S.A., Park H., Burr R.L. Abdominal pain impacts quality of life in women with irritable bowel syndrome. Am J Gastroenterol. 2006;101:124–132. doi: 10.1111/j.1572-0241.2006.00404.x. [DOI] [PubMed] [Google Scholar]
- 18.Wu I.X.Y., Wong C.H.L., Ho R.S.T., Cheung W.K.W., Ford A.C., Wu J.C.Y. Acupuncture and related therapies for treating irritable bowel syndrome: overview of systematic reviews and network meta-analysis. Ther Adv Gastroenter. 2019;12 doi: 10.1177/1756284818820438. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Manheimer E., Cheng K., Wieland L.S., Min L.S., Shen X., Berman B.M. Acupuncture for treatment of irritable bowel syndrome. Cochrane Database Syst Rev. 2012;(5) doi: 10.1002/14651858.CD005111.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Pei L., Geng H., Guo J., Yang G., Wang L., Shen R. Effect of acupuncture in patients with irritable bowel syndrome: a randomized controlled trial. Mayo Clin Proc. 2020;95:1671–1683. doi: 10.1016/j.mayocp.2020.01.042. [DOI] [PubMed] [Google Scholar]
- 21.Lowe C., Aiken A., Day A.G., Depew W., Vanner S.J. Sham acupuncture is as efficacious as true acupuncture for the treatment of IBS: a randomized placebo controlled trial. Neurogastroenterol Motil. 2017;29:e13040. doi: 10.1111/nmo.13040. [DOI] [PubMed] [Google Scholar]
- 22.Lembo A.J., Conboy L., Kelley J.M., Schnyer R.S., McManus C.A., Quilty M.T. A treatment trial of acupuncture in IBS patients. Am J Gastroenterol. 2009;104:1489–1497. doi: 10.1038/ajg.2009.156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Shi G., Yang X., Liu C., Wang L. Factors contributing to therapeutic effects evaluated in acupuncture clinical trials. Trials. 2012;13:42. doi: 10.1186/1745-6215-13-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Lin L., Liu Y., Shi G., Zhu W., Zhang X., Yan C. Influential factors and mechanism of non-specific effect in acupuncture. Zhōngguó zhēnjiǔ. 2016;36:1319–1323. doi: 10.13703/j.0255-2930.2016.12.026. [DOI] [PubMed] [Google Scholar]
- 25.Hohmann E., Brand J.C., Rossi M.J., Lubowitz J.H. Expert opinion is necessary: Delphi panel methodology facilitates a scientific approach to consensus. Arthros: J Arthrosc Relat Surg. 2018;34:349–351. doi: 10.1016/j.arthro.2017.11.022. [DOI] [PubMed] [Google Scholar]
- 26.Atkins D., Best D., Briss P.A., Eccles M., Falck-Ytter Y., Flottorp S. Grading quality of evidence and strength of recommendations. BMJ. 2004;328:1490. doi: 10.1136/bmj.328.7454.1490. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Atkins D., Eccles M., Flottorp S., Guyatt G.H., Henry D., Hill S. Systems for grading the quality of evidence and the strength of recommendations i: critical appraisal of existing approaches the grade working group. BMC Health Serv Res. 2004;4:38. doi: 10.1186/1472-6963-4-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Hasson F., Keeney S., McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32:1008–1015. [PubMed] [Google Scholar]
- 29.Hohmann E., Cote M.P., Brand J.C. Research pearls: expert consensus based evidence using the Delphi method. Arthros: J Arthrosc Relat Surg. 2018;34:3278–3282. doi: 10.1016/j.arthro.2018.10.004. [DOI] [PubMed] [Google Scholar]
- 30.Hsu C., Sandford B. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12:1–8. [Google Scholar]
- 31.Su X.T., Wang L.Q., Li J.L., Zhang N., Wang L., Shi G.X. Acupuncture therapy for cognitive impairment: a Delphi expert consensus survey. Front Aging Neurosci. 2020;12 doi: 10.3389/fnagi.2020.596081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Trevelyan E.G., Turner W.A., Robinson N. Developing an acupuncture protocol for treating phantom limb pain: a Delphi consensus study. Acupunct Med. 2015;33:42–50. doi: 10.1136/acupmed-2014-010668. [DOI] [PubMed] [Google Scholar]
- 33.Sun N., Wang L., Shao J., Zhang N., Zhou P., Fang S. An expert consensus to standardize acupuncture treatment for knee osteoarthritis. Acupunct Med. 2020;38:327–334. doi: 10.1177/0964528419900789. [DOI] [PubMed] [Google Scholar]
- 34.Ravensbergen H.J.C.R., Mann D.L., Kamper S.J. Expert consensus statement to guide the evidence-based classification of paralympic athletes with vision impairment: a Delphi study. Brit J Sport Med. 2016;50:386–391. doi: 10.1136/bjsports-2015-095434. [DOI] [PubMed] [Google Scholar]
- 35.Hohmann E., Angelo R., Arciero R., Bach B.R., Cole B., Cote M. Degenerative meniscus lesions: an expert consensus statement using the modified Delphi technique. Arthros: J Arthrosc Relat Surg. 2020;36:501–512. doi: 10.1016/j.arthro.2019.08.014. [DOI] [PubMed] [Google Scholar]
- 36.Lim B., Manheimer E., Lao L., Ziea E., Wisniewski J., Liu J. Acupuncture for treatment of irritable bowel syndrome. Cochrane Database Syst Rev. 2006 doi: 10.1002/14651858.CD005111.pub2. [DOI] [PubMed] [Google Scholar]
- 37.Manheimer E., Wieland L.S., Cheng K., Li S.M., Shen X., Berman B.M. Acupuncture for irritable bowel syndrome: systematic review and meta-analysis. Am J Gastroenterol. 2012;107:835–847. doi: 10.1038/ajg.2012.66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Zheng H., Chen R., Zhao X., Li G., Liang Y., Zhang H. Comparison between the effects of acupuncture relative to other controls on irritable bowel syndrome: a meta-analysis. Pain Res Manag. 2019;2019 doi: 10.1155/2019/2871505. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Chao G.Q., Zhang S. Effectiveness of acupuncture to treat irritable bowel syndrome: a meta-analysis. World J Gastroenterol. 2014;20:1871–1877. doi: 10.3748/wjg.v20.i7.1871. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Zhu L., Ma Y., Ye S., Shu Z. Acupuncture for diarrhoea-predominant irritable bowel syndrome: a network meta-analysis. eCAM. 2018;2018 doi: 10.1155/2018/2890465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.MacPherson H., Tilbrook H., Bland J.M., Bloor K., Brabyn S., Cox H. Acupuncture for irritable bowel syndrome: primary care based pragmatic randomised controlled trial. BMC Gastroenterol. 2012;12:150. doi: 10.1186/1471-230X-12-150. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Choi E.M., Jiang F., Longhurst J.C. Point specificity in acupuncture. Chin Med. 2012;7:4. doi: 10.1186/1749-8546-7-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Tjen-A-Looi S.C., Li P., Longhurst J.C. Medullary substrate and differential cardiovascular responses during stimulation of specific acupoints. Am J Physiol-Regul, Integr Compar Physiol. 2004;287:R852–R862. doi: 10.1152/ajpregu.00262.2004. [DOI] [PubMed] [Google Scholar]
- 44.Yan J., Miao Z., Lu J., Ge F., Yu L., Shang W. Acupuncture plus chinese herbal medicine for irritable bowel syndrome with diarrhea: a systematic review and meta-analysis. eCAM. 2019;2019 doi: 10.1155/2019/7680963. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.White A., Foster N.E., Cummings M., Barlas P. Acupuncture treatment for chronic knee pain: a systematic review. Rheumatology. 2007;46:384–390. doi: 10.1093/rheumatology/kel413. [DOI] [PubMed] [Google Scholar]
- 46.Sandrey M.A., Bulger S.M. The Delphi method: an approach for facilitating evidence based practice in athletic training. Athl Train Educ J. 2008;3:135–142. [Google Scholar]
- 47.Keeney S., Hasson F., McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006;53:205–212. doi: 10.1111/j.1365-2648.2006.03716.x. [DOI] [PubMed] [Google Scholar]
- 48.Powell C. The Delphi technique: myths and realities. J Adv Nurs. 2003;41:376–382. doi: 10.1046/j.1365-2648.2003.02537.x. [DOI] [PubMed] [Google Scholar]
- 49.Liberati A., Altman D.G., Tetzlaff J., Mulrow C., Gøtzsche P.C., Ioannidis J.P.A. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PloS Med. 2009;6 doi: 10.1371/journal.pmed.1000100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Shea B.J., Reeves B.C., Wells G., Thuku M., Hamel C., Moran J. Amstar 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. doi: 10.1136/bmj.j4008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Sumsion T. The Delphi technique: an adaptive research tool. Brit J Occup Ther. 1998;61:153–156. [Google Scholar]
- 52.Diamond I.R., Grant R.C., Feldman B.M., Pencharz P.B., Ling S.C., Moore A.M. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67:401–409. doi: 10.1016/j.jclinepi.2013.12.002. [DOI] [PubMed] [Google Scholar]
- 53.Murphy M.K., Black N.A., Lamping D.L., McKee C.M., Sanderson C.F.B., Askham J. Consensus development methods, and their use in clinical guideline development. Health Technol Asses. 1998;2:1–88. [PubMed] [Google Scholar]
- 54.Okoli C., Pawlowski S.D. The Delphi method as a research tool: an example, design considerations and applications. Inform Manag-Amster. 2004;42:15–29. [Google Scholar]
- 55.Phillips B. Towards evidence based medicine for paediatricians. Arch Dis Child. 2004;89:489–490. doi: 10.1136/adc.2004.055343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Luedtke K., Basener A., Bedei S., Castien R., Chaibi A., Falla D. Outcome measures for assessing the effectiveness of non-pharmacological interventions in frequent episodic or chronic migraine: a Delphi study. BMJ Open. 2020;10:e29855. doi: 10.1136/bmjopen-2019-029855. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data that support the findings of this study are available within the article and its supplementary materials.