Skip to main content
Journal of General and Family Medicine logoLink to Journal of General and Family Medicine
letter
. 2021 Mar 29;22(5):307–308. doi: 10.1002/jgf2.437

Insufficient comprehensiveness of study collection and leaping to the conclusion

Yuki Honda 1,2,✉, Toshinori Nishizawa 3, Keita Morikawa 4, Tomohisa Oku 5, Hideki Tsunoda 6, Yuki Kataoka 7,8,9,10
PMCID: PMC8411405  PMID: 34485005

To the Editor,

We have read with interest the article by Ota et al.1 We applaud the authors’ efforts in conducting a systematic review on the efficacy of shakuyaku‐kanzo‐to (SKT) for the treatment of muscle cramps. However, we have three concerns.

First, the systematic review may have been insufficiently comprehensive.2 The authors did not search major literature and clinical trial registration databases, such as Embase, the International Clinical Trials Registry Platform, and ClinicalTrials.gov. Three randomized‐controlled trials were included in their review, but a more extensive search could have yielded a more comprehensive collection of studies. The use of clinical trial registration databases to collect unpublished studies could have reduced publication bias.

Second, the authors should clarify whether they included herbal medicines from countries other than Japan in their review, because herbal medicines such as SKT are frequently used in China and Korea. If herbal medicines from other countries were included, then databases containing Chinese‐ or Korean‐language articles should also have been included. For example, one systematic review on the effect of herbal medicine on post‐stroke spasticity by Cai et al utilized Chinese databases and clinical trial registration websites.3 In China, SKT is known as “shao‐yao‐gan‐cao‐tang”; the randomized‐controlled trials reviewed by Cai et al frequently utilized shao‐yao‐gan‐cao‐tang and its two ingredients, “bai shao” and “gan cao,” as interventions. If the authors reviewed studies on Chinese herbal medicines, they should have included “shao‐yao‐gan‐cao‐tang,” “bai shao,” and “gan cao” in their search.

Finally, the authors’ conclusion in the abstract, “SKT might show efficacy in treating muscle cramps in patients with cirrhosis or lumbar spinal stenosis,” was a leap that may lead readers to overestimate the review's results. Of the three trials analyzed, two were at high risk of bias and one was of concern; moreover, a meta‐analysis was not performed because of high heterogeneity. These issues raise doubts about the certainty of the evidence, making it difficult to conclude anything about direction of the effect.

CONFLICT OF INTEREST

There is no conflict of interest associated with this publication for all authors.

ACKNOWLEDGEMENT

There has been no significant financial support for this work.

REFERENCES

  • 1.Ota K, Fukui K, Nakamura E, et alEffect of Shakuyaku‐kanzo‐to in patients with muscle cramps: A systematic literature review. J Gen Fam Med. 2020;21(3):56–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Onishi A, Furukawa TA. Publication bias is underreported in systematic reviews published in high‐impact‐factor journals: Metaepidemiologic study. J Clin Epidemiol. 2014;67(12):1320–1326. [DOI] [PubMed] [Google Scholar]
  • 3.Cai Y, Zhang CS, Liu S, et alAdd‐on effects of Chinese herbal medicine for post‐stroke spasticity: A systematic review and meta‐analysis. Front Pharmacol. 2019;10. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of General and Family Medicine are provided here courtesy of Wiley

RESOURCES