Joseph Pizzorno, ND, Editor in Chief
In this issue Assistant Editor Alan Gaby, MD—a longtime leading author and educator in nutritional medicine—has written a provocative Commentary on alarming evidence of fraud in natural health products research. Since this is very worrisome, I asked 4 leaders in the field of integrative/natural/functional/environmental medicine to comment on his paper. I have compiled their responses alphabetically below.
To assist clinicians who directly read the research, we have created a sidebar with guidance on how to recognize problematic articles.
Jeffery Bland, PhD, Associate Editor, co-Founder functional medicine, longtime leader and educator in nutritional medicine.
My compliments to Alan for this very well written and important article. His advocacy and efforts in this area are laudable. I, too, have questioned a number of the clinical studies that are now being published with what I consider “improbable” validity.
Dr. Gaby has done a Herculean job of digging deeply into this issue and has brought his scholarship and clinical experience to help us all understand the growing lack of trust we have for “peer reviewed” clinical studies. This is a topic that is important to us all. The principle of integrity and truth in the representation of published clinical and basic scientific research is the cornerstone from which the whole of decision making about approaches to patient management is built.
I believe that this problem is more than just Asemi’s work. This is only the tip of an iceberg. I am reminded of the 2018 story of Dr. Bharat Agarwal and his curcumin research as a first example that was widely publicized in the area of scientific fraud in our space. There is presently much in the clinical literature that is being published that has questionable data in support of the conclusions in both the conventional and alternative medicine literature.
Francesco DiPierro, PhD, Highly published major researcher of natural health products
I completely agree with the Author and with the first comment you have received.
Unfortunately, it is like that.
The problem also concerns Italy, I guess. Maybe not as much as described but “some” Italian researchers do the same…
I add some more… as you know I try to carry on some trials to tests ideas, substances, probiotics… I always start organizing all with placebo… well, 90% of enrolled guys do not sign unless I do assure them they will not be in the placebo group. They do not want to risk being in the “fake tab group”…
90% of times I have then to change my protocol…
5 years back I was testing Boswellia Phytosome at Besta Hospital (European Excellence for brain cancer). My idea was to check the anti-edema properties of the product (edema can be caused both by tumour and by therapy in case of gliomas). Well, after 7 weeks I have changed the protocol and I have submitted to the EC a new study cancelling the idea of placebo group. All but one of the enrolled guys asked me to be in the treated group. On one hand I perfectly understand their desire to be in the treatment group. On the other hand my question is: when researchers publish studies where placebo groups were not use, how can their research be respected?
Everybody can check. I published the study writing that “respecting the protocol” was impossible.
So, my question is: how can some researchers get to publish in some cases a huge number of studies with large numbers of subjects in double-blind placebo controlled condition?
Michael Murray, ND, Leading author and educator in natural medicine
Publishing the article is important. Alan is spot on here and getting in front of the story. And it might be important to state this problem is also occurring in conventional medicine as well. See https://jme.bmj.com/content/47/12/e19
Jacob Schor, ND, Assistant Editor and prolific reviewer of the natural medicine research
Aside from the general devaluation of truth in many settings this exposition of seemingly fictional research by our friend Dr. Gaby raises several additional questions.
What first comes to mind is whether individuals who follow or are interested in these topics related to natural substances are more gullible or more naive than conventional medical practitioners in general? Are these topics of focus where it’s easier to get away with sloppiness or outright lying? That is, are we and our colleagues more naive or easier to con? For many of us belief in the efficacy of natural substances as treatment for morbidity is a matter of faith as well as experience, thus, easy to accept outcomes that reinforce our initial belief.
The second question would be whether similar patterns of article quality are showing up from these countries on other topics unrelated to natural therapies? Or are we the preferred target audience?
I recall many years ago being taught some digit rule that helped one spot falsified data. Apparently, Google suggests that this is called Benford’s Law. https://en.wikipedia.org/wiki/Benford%27s_law.‘
Benford’s law - Wikipedia
Benfords law, also known as the Newcomb–Benford law, the law of anomalous numbers, or the first-digit law, is an observation that in many real-life sets of numerical data, the leading digit is likely to be small. In sets that obey the law, the number 1 appears as the leading significant digit about 30 % of the time, while 9 appears as the leading significant digit less than 5 % of the time. en.wikipedia.org
There are apps to test for Benfords Law. Here is one example. https://jasp-stats.org/2021/11/30/benfords-law-jasp/.
But I suspect any credible forger will have a system to generate seemingly random data that nevertheless conforms to the rule.
As I look at this I’m not sure that it was Benford who came up with this principle but a rather earlier character. Simon Newcomb, fits the story I remember better.
I recall that a number of years back several news stories focused on the belief that the card game poker is derived from a Persian card game that dates back centuries, suggesting a long cultural tradition in Iran of valuing and rewarding those individuals most adept at bluffing others. Perhaps this a modern expression of what has been a steady epigenetic shift.
The importance of telling the truth is one of those fundamentals that is required for a society to function. We’ve seen various cultural efforts to cultivate or embed this trait into social structures going back to the ten commandments, George Washington’s cherry tree myth and even to the Boy Scout oath. It’s a habit that doesn’t come easily for some. I suppose there is some evolutionary advantage for the individual who lies well while there is a disadvantage to a society with too many lying members.
This information is something that might be difficult to hear. It reminds me of the cellphones making corn kernels pop YouTube videos from several years back. Once released there was no way to convince people that they were faked.
I’m glad IMCJ is taking a stand on this. It would be nice to have an accompanying article or side bar that provides an ‘idiot list’ of traits of falsified research to help practitioners spot questionable papers.