Flawed scientific studies block progress and sow confusion

ABSTRACT

Abstract Research in crop science in recent years has advanced at an unprecedented rate, and the intermingling of old and new crop breeding technologies has made the term “genetically modified” – and its variant, Genetically Modified Organism, or “GMO” – virtually obsolete. A kind of pseudo-category, it is primarily used pejoratively to refer to the use of the newest, most precise, most predictable, molecular genetic techniques. Prodigious amounts of time, effort and care have been expended to ensure that crops developed for commercialization using molecular techniques are safe, and that new traits are beneficial. Â Yet, despite these advances, some skepticism persists about them, partly due to the publication of fraudulent, poorly designed, and biased studies by a few “rogue scientists” whose intention is to contaminate the scientific literature and sow mistrust about molecular genetic modification among regulators and the public. We discuss how such flawed studies make it to publication and how the scientific community can combat such disinformation.

What would you think if a few “research” groups repeatedly claimed to have evidence that protein, not DNA, was the carrier of hereditary information, and their findings were consistently debunked? And what about the claim of an association between measles-mumps-rubella (MMR) vaccine and autism, which has been repeatedly and convincingly repudiated.¹ Eventually, you’d probably conclude that either the concept was flawed, or that the researchers were dishonest, or both.

Well, that is exactly what has been happening for at least two decades, as one article after another that supposedly reveals some harm or unexpected outcome of “genetic modification” to produce “genetically modified organisms,” or “GMOs,” has been exposed as having flaws in methodology or interpretation. In many cases, these reports are “advocacy research,” which appear to cross the line from carelessness or honest errors to professional misconduct.

First, what is a GMO? That’s a key question, and an accurate but perhaps unhelpful answer might be that everything is, or nothing is. By that, we mean that genetic modification to improve microorganisms, plants and animals has been with us for millennia. The term GMO is often used pejoratively, to refer to organisms made with precise molecular techniques which have been around since the 1970s. However, discussions of which techniques should be included seem to us irrelevant, if not ridiculous, not unlike discussing whether taping a plastic cone to the forehead of a horse makes it a unicorn.

Genetic modification is accomplished by a seamless continuum of techniques,² including selection/hybridization, mutagenesis, and wide crosses (in which plants from different species or genera are made to hybridize); and for almost half a century, molecular techniques have been applied to microorganisms, plants, animals, and even humans. The newer techniques have created stunning scientific, economic, and humanitarian successes, from crop plants that kill insects and are drought-resistant to gene therapy that cures genetic diseases.³

Some would even cite “natural genetic modification” as resulting from the movement of genes in the wild. Genetic engineering using molecular techniques has been around for almost half a century, initially as a technology that could modify bacteria and advance our ability to make life-saving drugs such as insulin and hepatitis B vaccine. While many of these technological advances have been readily welcomed by an appreciative public, the same cannot be said for genetically engineered plants and animals.

The genetic engineering of plants has adhered to a trajectory that is similar to that of microorganisms. Today’s crops are the accomplishment of a continuum of techniques, and their products can be found in any grocery store, from broccoli, kale and cauliflower (selective breeding) to ruby red grapefruit and barley (mutagenesis) to several stone fruit varieties (wide-cross hybridization). Many of these adorn dinner tables the world over, and we consume them with hardly a second thought.

As our technologies became ever more sophisticated, we are able to produce an even greater diversity of fruits and vegetables. With this solid and robust history of plant breeding behind us, we continue to develop new breeding technologies that are more refined, precise, and predictable, including, most recently, genome editing of food crops. We are now on the cusp of growing a plethora of new crops that will make farmers’ lives easier, are better tasting, and more nutritious.

To get to this point, scientists have accumulated a body of evidence that demonstrates that the new breeding technologies do not confer incremental risk, and that they are highly precise and predictable. It is only when the products (organisms) are shown to be safe and functionally successful that they are made commercially available.

Today’s mixture of old and new crop breeding methods has blurred the term “genetically modified.” With the exception of wild berries, wild mushrooms, seafood and wild game, all of our food has been genetically modified in some way. Therefore, how odd it is that we find ourselves in a quagmire of over-regulation and activists’ antagonism toward crops modified with molecular techniques.

Even more perplexing is that the science itself behind these new plant breeding methods is being questioned, and why our uninterrupted pathway of technological breakthroughs and improvements in crop breeding that have become a hallmark of civilization as we know it is now suddenly being challenged. An answer to this can be found in the unethical behavior of a few rogue players, scientists who have concocted research findings that supposedly contradict the voluminous evidence of the safety of genetically engineered plants. Not only do these rogue scientists confuse nonexperts, but they also waste valuable resources as the scientific community is forced to address and disprove their spurious findings.

The infamous paper by G.E. Seralini and colleaguesis the prototype of these kinds of flawed anti-genetic engineering studies.⁴ The paper contained claims that genetically engineered corn was linked to the growth of tumors in rats, with the authors parading hideous photos of the “results” of their study – at a press conference, before the publication of the paper. Severely flawed, it was eventually retracted, only to be republished in a different journal and without peer review. The methodological shortcomings of the study were obvious. The strain of rat used is predisposed to tumors and the small sample size made it impossible to draw any statistical conclusions. Although multiple replications of the Seralini studies by reputable investigators, requiring time, energy, and funds, demonstrated that the original findings were false, they continue to be cited by anti-genetic engineering activists and propagandists.

Then, there is Federico Infascelli’s research group at the University of Naples, which questioned the safety of genetically engineered soybeans fed to rabbits.⁵ Among other things, the journal that published the work eventually voiced concerns about analytical gels in some of the figures that had been manipulated in an effort to demonstrate that GE soybeans were harmful. Elena Cattaneo, an Italian senator and scientific researcher who prompted investigations into this and other publications authored by the same researchers, did in fact succeed in obtaining a retraction. [For her heroism in having her investigative team generate a 1,500-word compilation of available scientific evidence regarding the safety of GE crops, Sen. Cattaneo was demonized in the Senate as a “lobbyist for Monsanto.”) Another Infascelli article, in Nutrition and Food Science, was also retracted for fabrication of results; this time, his study made claims that modified genes could wind up in the blood and organs of the young offspring of mother goats who had been fed GE soybeans.

A particularly egregious example of the failure of editorial and peer review occurred in a,⁶ articlein the Proceedings of the National Academy of Sciences (“PNAS”]. The authors claimed to show that pollen from genetically engineered corn was injurious to insects called caddisflies in a laboratory aquatic ecosystem, but the conclusions were not justified by the data. Inexcusably, the authors failed to measure the actual levels of pollen from the genetically engineered corn, and thus the two pillars of toxicology – exposure and dose – were both unknown. It is perfectly possible, and, in fact, highly likely, that the caddisfly larvae in question were not exposed to any genetically engineered component at all.

Equally important, the researchers had reported elsewhere that they failed to find these same effects in studies in the field – which they neglected to reveal in the PNAS article. This is a critical omission because laboratory studies are designed to mimic what happens in the “real world.” In other words, even if the laboratory studies had been performed correctly and carefully, positive results arguably would have been irrelevant because they don’t mimic what happens in the field.

A particularly atrocious paper that embodies two kinds of scientific misconduct appeared in Critical Reviews in Food Science and Nutrition in 2009.⁷ The article, “The Health Risks of Genetically Modified Foods” by Dona and Arvanitoyannis of the Universities of Athens and Thessaly, respectively, cited non-peer-reviewed “evidence” and myths found on anti-technology websites, as well as some long-discredited papers such as those described above; at the same time, it systematically omitted numerous key references that, contrary to the authors’ conclusions, establish the safety of genetically engineered crops.

Even if the authors were simply unaware of the references that contradict their views [which is highly unlikely), this kind of bias is considered unethical in science. Worse than such ignorance, the review article was extensively plagiarized, with large blocks of text lifted from other papers without quotation marks or attribution.

The list goes on and on. A,⁸ publicationin Reproductive Toxicology by Aris and Leblanc claims to have detected traces of the Bt toxin (a protein from the bacterium Bacillus thuringiensis] used in GE crops in maternal blood (including in umbilical cords). The article has numerous problems, however. The authors conveniently ignored that Bt proteins could actually have come from use in organic farming. (Intact, non-engineered B. thuringiensis bacteria are widely applied to crops, in both organic and conventional farming.) The assay examined in the study was, in any case, inappropriate for measurements in humans. In fact, a pregnant woman would have to consume kilograms of corn a day to achieve the measurements claimed to be detected in blood. Moreover, humans do not have the receptor for Bt, and the toxin is active only in an alkaline environment (such as the gut of an insect), not in humans’ stomach acid.

Another example is the study that linked animal feed containing GE grain to stomach inflammation in pigs.⁵ The data were cherry picked so that inflammation also found in the non-GE control group was not addressed. Compositional variations between the test GE and control non-GE feed were also not considered, as were the unusually high rates of pneumonia found in the pigs, a red flag that something with the study was awry. These significant problems were flagrantly disregarded by the authors, in order for them to conclude inappropriately that their analysis impugned the safety of the GE feedstock.

Although the scientific community can see through such chicanery and rejects the work of these fraudulent studies (although occasionally, sloppy or corrupt peer-reviewers do not), the general public are not as well versed on scientific methodology, and the damage can be severe and lasting. Advocacy research that supports certain ideological ends or financial interests [in this case, the organic agriculture and “natural products” industries, which reject molecular genetic engineering and see it as a threat) continues to reverberate in the social media echo-chamber.

By far the most important, systematic review of articles in the scientific literature that claim to have found potentially harmful effects of GE technology or organisms was publishedin⁹,by Miguel Sanchez and Wayne Parrott. They found that about five percent of studies related to the safety of GE plants show adverse effects that are a “cause for concern and tend to be featured in media reports.” Most significant, they found that “a close examination of these reports invariably shows methodological flaws that invalidate any conclusions of adverse effects,” and that the 35 studies “tend to come from just a few laboratories and are published in less important journals.”

Finally, Sanchez and Parrott concluded that “twenty years after commercial cultivation of [GE] crops began, a bona fide report of an adverse health effect due to a commercialized modification in a crop has yet to be reported.” That brings us back to the analogy to the claims that protein is the stuff of heredity, or that MMR vaccine causes autism.

Deliberate fraud, poor design, ignorance of contravening data, and biased interpretation of results are the playbook of opponents of molecular genetic engineering. As we and Sanchez and Parrott have discussed, occasionally such work somehow makes its way to publication, either through gaps in the editorial/peer review process or by taking advantage of the rise of pay-to-play predatory journals, which will publish virtually any paper, no matter how flawed, for a hefty fee and often with no peer review at all.

Once published in a predatory journal, it is a short step for an intentionally misleading study to be picked up by a journalist with an agenda and to be broadcast on social media; and then the damage has been done. Even if the unscrupulous paper is eventually caught, condemned, and retracted, activists will continue to cite the work for their own purposes, and even arrange to have it republished elsewhere in another disreputable journal.

Via shoddy or agenda-driven journalism, the publication of flawed “scientific” studies sets the stage for activists to amplify the message they want the public to see. We should note that this strategy is not limited to genetic engineering but is also a staple of propaganda related to products and activities disfavored by various activists, such as chemical pesticides, vaccines, e-cigarettes, and nuclear power. The goal of activists is eventually to make use of the weaknesses in the scientific publication process to garner sufficient public and political support to spur public policy changes that further their own agendas, as is currently the case for attempts to ban the important agrichemicals glyphosate and neonicotinoids.

Studies to confirm or discount the validity of such research are sometimes performed, but they divert time, energy, and funding away from other, important scientific endeavors.

How can these problems be addressed? There are many ways that the scientific community can fight back and prevent the promulgation of flawed research studies:

• The “business” of science, from the securing of funding to publication, has become too porous and must be tightened to safeguard against this form of abuse.

• Improve the reliability of results (including insisting on the use of appropriately large sample sizes and careful selection of controls] to avoid misrepresentation of data.

• Establish large international research consortia, so that accountability between research groups will strengthen study design and the interpretation of results and make it more difficult for malfeasance to occur.

• Establish a set of norms surrounding the statistical analysis of research data, how independent replication studies are to be conducted, and perhaps even data-sharing among independent research groups.

• Encourage more rigorous peer review (which should have blocked the most egregious examples of flawed research reported in legitimate journals, such as Dr. Andrew Wakefield’s infamous paper about MMR vaccine causing autism, and some of the others discussed above)

• Researchers should identify and discuss in their articles any form of uncertainty that is inherent in their results, including limitations in statistical significance, the number of variables, and even the overall complexity of the system under analysis. (This could even be implemented by including a new section within each journal article, in addition to abstract, methods, results, and discussion.)

Predatory journals are a significant part of the problem, and journalists should be made more aware that their reports are unreliable sources of information. Science communicators can provide a “heinous hundred” list of them, which would help not only journalists but also the public and university tenure committees to identify when they are being misled. Research institutions should penalize investigators for publishing in predatory journals.

Some red flags are easy to spot. They include obvious conflicts of interest between researchers and stakeholders who may have skin in the game, which can create an inherent bias. Other warning signs include results that conflict with the current body of research literature or scientific topics that are particularly controversial, such as GE crops; and those should be examined thoroughly by journalists with particular care to confirm their validity. Although trust in scientists in general remains high, fewer than a quarter of the public understand the scientific method, which makes it difficult for the public and media alike to find the flaws in a study of questionable merit. Journalists should therefore make an effort to engage the topic of fraudulent scientific behavior with their audiences and identify issues of concern such as agendas or conflicts of interest, in order not to misrepresent the significance of a study.

Because science is (or is supposed to be) self-correcting – a thesis is put forth, tested, and ultimately revised on the basis of new data – misinformation conveyed to the scientific community distorts the entire process. It disrupts the planning of follow-up experiments, retards progress, and confounds societal decision-making about a vast spectrum of critical issues.

As the effective arbiters of what represents scientific “truth,” the editors of scientific journals have a responsibility to be rigorous and diligent about detecting and punishing the misconduct of authors and the incompetence of their peer-reviewers.

Journalists also have an important role – the responsibility to cite only legitimate research findings from respectable sources.

The undeserved attention given to a handful of poorly designed research studies or deliberately misleading analyses of genetically engineered crops and other scientific subjects is a menace that must be addressed, if the scientific community is to receive public support of their efforts to address some of the planet’s greatest challenges. We allow the distortion and pollution of the scientific literature at our peril.

Disclosure statement

No potential conflict of interest was reported by the authors.