Academic urban legends

Abstract

Many of the messages presented in respectable scientific publications are, in fact, based on various forms of rumors. Some of these rumors appear so frequently, and in such complex, colorful, and entertaining ways that we can think of them as academic urban legends. The explanation for this phenomenon is usually that authors have lazily, sloppily, or fraudulently employed sources, and peer reviewers and editors have not discovered these weaknesses in the manuscripts during evaluation. To illustrate this phenomenon, I draw upon a remarkable case in which a decimal point error appears to have misled millions into believing that spinach is a good nutritional source of iron. Through this example, I demonstrate how an academic urban legend can be conceived and born, and can continue to grow and reproduce within academia and beyond.

Bauerlein et al. (2010) claim that we are currently experiencing an ‘avalanche of low-quality research’, and academia has become an environment where ‘[a]spiring researchers are turned into publish-or-perish entrepreneurs, often becoming more or less cynical about the higher ideals of the pursuit of knowledge’. Whether the current state of affairs is better or worse than before, it seems reasonable to assume that corner-cutting is an unfortunate side effect of publication pressure and competition for academic positions and scarce resources, especially in milieus where counting publications is more important than reading and evaluating them. In this article, I explore a particular set of corner-cutting techniques that reveal much about strategies of reading, writing, and citation, as well as the development of academic urban legends.

The digital revolution within academia

Twenty-five years ago, it could take weeks to obtain a specific source document needed in order to verify or explore a reference that for some reason had caught one’s attention. As a consequence of the digital revolution, it is possible today to obtain a wide spectrum of sources within minutes or seconds. Formidable databases, advanced scanners, optical character recognition (OCR) technology, and new features of reference management software such as Endnote have made it possible, with some experience, to read an academic text together with the sources it refers to.

For those of us who are old enough to know what a card catalog is, it is outright fascinating to sit in front of a computer with two monitors, reading an academic text on one of them and having the sources it refers to (or perhaps should have referred to) on the other. Having immediate access to most, if not all, of the sources behind an academic text opens up a number of exciting opportunities, but also exposes some unpleasant surprises. In the past few years, it has become dramatically easier to identify cases of plagiarism and scientific misconduct, and also to discover other types of academic shortcuts, and to see how shockingly frequently they are employed.

In this article, I will limit my focus to one specific type of situation that seems to cause problems for a large number of researchers and students and that provides a breeding ground for a wide variety of academic shortcuts. The situation of interest is one that all writing academics will encounter numerous times during their career: when we read a text and find a statement or specific point that we would like to use ourselves, and we discover that it is already accompanied by a reference.

Spinach as a good source of iron

I will illustrate this situation with an example I encountered not long ago, in which a scientific article I was reading presented me with some new and outright fascinating knowledge. The following quote, including the reference, is taken from an article published by K. Sune Larsson in the Journal of Internal Medicine:

The myth from the 1930s that spinach is a rich source of iron was due to misleading information in the original publication: a malpositioned decimal point gave a 10-fold overestimate of iron content [Hamblin, 1981]. (Larsson, 1995: 448–449)¹

The quote caught my attention for two reasons. First, it falsified an idea that I had carried with me since I was a child, that spinach is an excellent source of iron. The most striking thing, however, was that a single decimal point, misplaced 80 years ago, had affected not just myself and my now deceased parents, but also a large number of others in what we place on our table.

After reading Larsson’s article, I took a poll of colleagues at my institute, asking them why they think spinach is healthy. The conclusion was quite clear. The belief that spinach is a good source of iron, although falsified 30 years ago by Hamblin in a British Medical Journal article, is still widespread among my colleagues, all of whom have, at minimum, a master’s degree in health sciences. In fact, the history of spinach consumption in the Western world indicates that we are dealing with a decimal point error of enormous consequences. For generations, parents have been wasting their time and energy, nagging their more or less anemic children to eat a vegetable that the young typically abhor, ruining family social events in the process.

Truth be told, there is iron in spinach, but not significantly more than in other green vegetables, and few people can consume spinach in large quantities. A larger problem with the idea of spinach as a good source of iron, however, is that it also contains substances that strongly inhibit the intestinal absorption of iron (see e.g. Garrison, 2009: 400). Simply put, spinach should not at all be the first food choice of those suffering from iron deficiency.

Larsson’s article made me aware of the remarkable fact that a large number of people in the Western world have been misled for a staggeringly long time. Since so many people still believe that spinach is a good source of iron, I have good reason to convey this newfound knowledge to others. The story of this decimal point error is, in addition, a brilliant illustration of how a small stroke may fell a great oak, and a reminder of the importance of accuracy and quality control in the production and distribution of scientific knowledge.

How, then, should I properly pass on the important messages I learned from a single sentence in Larsson’s article? The following seems like a fairly appropriate paraphrasing of the original text:

The idea that spinach is a good source of iron is a myth that was born in the 1930s, due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value.

How should I refer to my source? If I want to include this sentence in an academic publication, what should I place after my sentence? There are several options in this particular case, and I will present the most common among them and discuss what consequences the various alternatives may have.

The first alternative is to leave the sentence as it is, without any reference at all. This is something I can do if I am distributing common knowledge, which obviously is not the case here. Should I choose to omit a reference, I could, in the worst case, be accused of plagiarism, and the most naive among my readers would perhaps think that I was the one who discovered the decimal point error with the dramatic consequences. A more likely outcome would be that my readers would become puzzled, or perhaps irritated, by the fact that I did not provide any form of documentation for how such a remarkable thing could occur. In principle, it should be impossible or very difficult to get undocumented statements of this kind published in scientific publications, but as we will see toward the end of this article, it happens from time to time.

The simple truth is too simple

In academia, the following is fortunately a far more common way of passing on such a message:

The idea that spinach is a good source of iron is a myth that was born in the 1930s, due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value (Larsson, 1995: 448–449).

Here, I simply and honestly refer directly to the source where I found the information, and I am even courteous enough to provide exact page numbers for readers who would like to verify it, or who may be interested in exploring whether there is more to learn from Larsson. The problem in this case is that I omit a piece of information: the fact that Larsson’s statement is based on an entirely different source, namely Hamblin (1981). In other words, I am referring to an article that I very well know is a secondary source, and thus hide from my readers the fact that Larsson actually just passed on information published by Hamblin 14 years earlier. A good reason for avoiding the use of secondary sources in academia is that messages that pass through several links have the unfortunate tendency to become modified or altered along the way, as in the whisper game. My readers will in this case think that Larsson is the primary source, and my statement will therefore look more solid and trustworthy than it actually is.

Providing this type of reference has other negative consequences. This time it is not me, but Larsson who gets undeserved credit for the discovery of the decimal point error. Another consequence is that readers who try to verify my statement will get an unpleasant surprise when they look up the source (Larsson, 1995: 448–449) I have provided. They will then discover that they have become, quite unwillingly, participants in a kind of treasure hunt. Having reached the first post at page 449 in Larsson’s article, their only options would be either to give up the quest for verification, or to proceed to the next destination, Hamblin’s (1981) article in British Medical Journal.

In a case like this, when I am aware of the fact that my citation has weaknesses, it can be tempting to try to make the statement more convincing by adding more references I might have easily available. In our digital age, it is not difficult to find other sources that contain the story about the decimal point error and its dramatic consequences. If the supply is as rich as in this case, it is a good idea to select alternatives published fairly recently in respectable journals, such as the article by Frangoulis et al. (2010: 43), or maybe the book by Carroll and Vreeman (2009: 114). If I want, I can add numerous sources like this, getting a long and impressive list of references, full of prestigious journal names and publishers. This would, of course, be an academic confidence trick, but it would not be exposed until readers took the time to look up the sources I listed. Only then will they be able to see that they are all secondary sources, and that they all refer back to the same single sentence in Hamblin’s (1981) article.

Honesty is not always the best policy

A third and even more honest alternative would be to refer to my source in this way:

The idea that spinach is a good source of iron is a myth that was born in the 1930s, due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value (Hamblin, 1981, cited in Larsson, 1995: 448–449).²

This is a perfectly legitimate way of referring to sources in cases where it is difficult or impossible to obtain a primary source. The 1981 volume of British Medical Journal is, however, easily available for anyone with Internet access. Should this type of reference be used in this particular case, it could reflect a case of academic laziness, but coupled with utmost honesty. Another and perhaps more likely explanation is that we are dealing with an academic who has not understood the importance of the principle of striving to use primary sources in order to minimize the whisper game effect.³

This type of citation does not necessarily have to be explained by laziness or lack of knowledge, but rather by an almost touching degree of confidence and trust. In this case, I put my trust in Larsson that he has read and interpreted Hamblin correctly and that he has good enough reasons for putting his trust in Hamblin. Whatever explanation, I deserve credit for having made it perfectly clear that I have not consulted the primary source, and that my statement is the last and therefore the least trustworthy link in a chain of sources.

The risk of buying a pig in a poke

A fourth alternative, which unfortunately is far more common than we should wish, is to solve the problem the following way, without consulting Hamblin (1981):

The idea that spinach is a good source of iron is a myth that was born in the 1930s, due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value (Hamblin, 1981).

In this case, I am referring directly to a source that I have not consulted myself, and in doing so I am committing an academic lie. The same degree of trust as in the previous alternative is present, but the difference is that the stakes are now much higher. What I hope to achieve with this type of reference is that nobody will discover my laziness. I simply pretend that I have taken the effort to consult Hamblin (1981), without having done so. In short, I have plagiarized the Hamblin reference from Larsson.

An attractive aspect of this academic shortcut is that it is usually impossible to discover and to prove the sin committed. Academics such as Larsson presumably check their sources thoroughly, and double-check that their own text corresponds with the sources it refers to. If Larsson has understood Hamblin correctly, and Hamblin is worthy of his trust, then there would be no negative consequences from this highly dubious type of reference, neither for my readers, nor for the truth and reliability of what I am writing.

Referring to sources that one has not consulted can be, however, a risky business. Academics, as other human beings, do from time to time misinterpret or make errors that are not discovered by peer reviewers or editors, even in respectable journals such as the Journal of Internal Medicine. When several authors independently of each other manage to misrepresent a single source in exactly the same erroneous way, the explanation is either a statistically unlikely coincidence, or a case where authors have plagiarized references. Systematically patterned distributions of errors and misinterpretations are in fact common enough to make it possible to study the prevalence of citation plagiarism and the unfortunate consequences of the practice. Such studies are indeed sad readings for those who are concerned about safeguarding academic principles of honesty and quality control (see, for example, Harzing, 2002; Morrisey, 2004: 152–154; Wetterer, 2006; Wright and Armstrong, 2008).

The narrow path has its temptations

The four alternatives treated so far all represent various types of academic shortcuts, and they all share attributes with various ways in which rumors are spread. The common denominator of all of them is that I do not consult the assumed primary source in this case. I simply end up, in a more or less honest way, passing on what I have read that Larsson has read in Hamblin (1981).

The final alternative is to follow the short and narrow path back to Hamblin’s article to see what he wrote on the issue. It is, of course, wise to check the accuracy of what we base ourselves upon when we write and publish, and there is also the possibility that we might learn something even more valuable about the issue.

In this particular case, there is in fact a lot more to learn from what we have so far assumed is the primary source, not just about the decimal point error, but also about academic shortcuts, and, not the least, about the conception, birth, and growth of academic urban legends. As we shall see, it turns out that Larsson has in fact made several errors when reproducing Hamblin’s message, and on top of it all, Hamblin is not at all to be trusted in this particular case.

We find the following in Hamblin (1981):

The discovery that spinach was as valuable a source of iron as red meat was made in the 1890s … German chemists reinvestigating the iron content of spinach had shown in the 1930s that the original workers had put the decimal point in the wrong place and made a tenfold overestimate of its value. … For a source of iron Popeye would have been better off chewing the cans. (p. 1671)

The myth about the iron content of spinach was not born in the 1930s, as told by Larsson. The decimal point error was made 40 years earlier, but was disclosed in the 1930s, and Hamblin makes it perfectly clear that he himself was not the one who made the discovery. The third, and greatest surprise is not immediately clear from the quote above: Hamblin does not provide a reference to support his claim that a decimal point error actually was made; nor does he give any names, dates, or other information that could help us verify how the error was made and by whom, or who should be credited for its discovery and correction.

I have now taken the effort to consult the assumed primary source, only to discover that the case was considerably more complex than I had anticipated. In such a case, it may be tempting to suppress what I just discovered, that the fascinating decimal point error is thus far an undocumented piece of information, what we usually call a rumor. I could correct my first statement, for example, in this way:

The myth that spinach is a good source of iron has its origin in a decimal point error in the 1890s. German scientists discovered the error about 40 years later (Hamblin, 1981: 1671).

Here, I have done my academic duty by consulting Hamblin directly, and I have corrected my statement in a manner that corresponds much better with what Hamblin actually wrote. If I publish this text, I am still guilty of spreading a rumor, and also of disguising its rather obscure origin through a reference to a prestigious scientific journal. I very well know that the decimal point error story suffers from a lack of documentation, but I cannot resist the temptation to use it, perhaps because it fits so nicely into the argument of an article I am writing.

In fact, despite the vagueness and complete lack of documentation that characterize Hamblin’s account of the decimal point error, the story has been picked up by numerous authors who have redistributed it through journal articles and books, turning it into a full-blown and still blooming academic urban legend.

Realizing that a remarkable account like this lacks documentation, the safest and wisest thing would be to forget the whole issue. A tempting compromise could be to use terms and formulations that communicate reservations or doubt, as David A. Kronick did when he wrote the following, accompanied by a reference to Hamblin (1981):

Typographical errors, for example, do occur. It has been suggested that [emphasis added] spinach got its reputation as a dietary supplement because of a misplaced decimal point in which the iron value was given as ten times higher than it was. (Kronick, 1985: 75)

Kronick deserves praise for including his reservation: ‘It has been suggested that …’, but such phrases tend to change or disappear when knowledge passes from one text to another. Hypotheses, assumptions, and suggestions can in such a way be transformed into knowledge and scientific facts. We shall soon see a spectacular example of how this can happen, and what unfortunate consequences can follow.

An ironic point concerning the two articles we have quoted (Hamblin, 1981; Larsson, 1995) is that they are both written by academics placing themselves at the frontline of the fight against bad science and academic carelessness. In his article titled ‘The dissemination of false data through inadequate citation’, Larsson appears almost furious with the ways in which some scientists uncritically redistribute research findings and conclusions of dubious character. Larsson obviously has an important message, but he manages to do exactly what he is criticizing others for in quite a spectacular way. Larsson’s (1995) next sentence adds even more momentum to the irony:

The myth from the 1930s that spinach is a rich source of iron was due to misleading information in the original publication: a malpositioned decimal point gave a 10-fold overestimate of iron content [Hamblin, 1981]. Once a paper with misleading information has been published, it is almost impossible to stop citation. (pp. 448–449, emphasis added)

How true. There are no indications that the decimal point error ever was committed, and the whole story could have ended there, as a small slip of tongue by Hamblin in the 1981 Christmas issue of British Medical Journal. When the slip has turned into a widely distributed academic urban legend, it is thanks to the effort, or rather lack of effort, of a very large number of uncritical academics, among them Larsson himself.

The conception and birth of an academic urban legend

Hamblin’s article, with the short but telling title ‘Fake!’, is not just being used and cited by scientists concerned with academic quality and honesty. His story about the decimal point error extends far beyond academia, to newspapers, magazines, TV-programs, and web pages, including some that specialize in debunking myths and urban legends.

Very few have questioned Hamblin’s claim that a decimal point error is the reason why many of us still erroneously believe that spinach is a good source of iron, and many surely continue to pester their children as a consequence of this belief. There is, however, one exception.

Mike Sutton is a British criminologist who did not give up after having read Hamblin’s article and, like me, struggled his way through the rather confusing list of references without finding a single trace of any decimal point error. Sutton (2010a: 7) did what we all should do more often in such cases. He contacted Hamblin directly and asked him from where he had learned of the decimal point error. Hamblin replied that he could not remember, but that he was sure he had not made it up.

Sutton (2010b) argues convincingly that there were entirely other reasons, such as contamination during the analysis or the confusion between fresh and dried spinach, that caused the exaggerated figures in the 19th century. He also criticizes Hamblin for perpetuating another related misconception: that Popeye was created in order to promote spinach for its iron content. According to Sutton (2010a: 13–14), Elzie Crisler Segar had an entirely different nutrient, vitamin A, in mind when he invented Popeye and contributed to a massive increase in spinach consumption in the United States during the 1930s.

Shortly after Sutton published his article in 2010, a reader by the name Bonnie Taylor-Blake made Sutton aware that a certain Arnold E. Bender could be the source that Hamblin was unable to recall. The criminologist immediately brought the investigation a step further (Sutton, 2010b). In Bender’s inauguration lecture at Queen Elizabeth College, University of London in 1972, he made the following statement about the iron content of spinach: ‘the fame of spinach may well have grown from a misplaced decimal point’ (Bender, 1972: 11).

Bender repeated exactly the same sentence in a book titled The Facts of Food that was published 3 years later (Bender, 1975a: 15) and in a journal article (Bender, 1975b: 142) that was printed the same year. Two years later, he wrote a short comment letter (Bender, 1977) in The Spectator, where the wording is slightly more assertive: ‘The fame of spinach appears to have been based on a misplaced decimal point’. In 1982, the year after Hamblin, based on something he had heard or read somewhere, stated that a decimal point error was actually made, the following phrase appeared in a textbook for medical students (Bender and Bender, 1982: 55): ‘the belief can be traced back to a mistake in the transcription of analytical results in 1870, when a decimal point was misplaced’.

Bender’s suggestion about a possible decimal point error has now turned into an assertion linked to a specific year, two decades earlier than Hamblin’s purported date. But again, the statement still lacks a reference or any other type of documentation. It is therefore difficult to know if Bender’s (1982) increase in certainty could be a result of his own more thorough investigations on the issue or whether he had been influenced by Hamblin’s (1981) article. A third possibility is that Bender’s gradual change of wording, consistently without any precise documentation, could be a consequence of a common human weakness which allegedly has a particularly high prevalence among anglers telling stories about their past catches.

Bender and Hamblin have not just provided us with examples of how even top-level academics can sometimes be careless in their attitude to references and documentation. These publications between 1972 and 1982 have also given us an excellent opportunity to study the birth of an academic urban legend and to explore the micro dynamics behind the often dramatic outcomes of the whisper game. The decisive moment was most likely when Bender’s ‘appears to have been’ became replaced by Hamblin’s assertive statement. In other words, Hamblin had certainly heard about the decimal point error from someone, but accidentally turned Bender’s suggestion into a piece of fact, which then was blessed with the stamp of reliability associated with British Medical Journal.⁴

What could be more appropriate in this article about academic carelessness than to present a positive example, highly worthy of imitation? Terence Hamblin, the man who made the undocumented and most likely non-existent decimal point error known to the world in 1981, submitted the following comment to Mike Sutton’s web page almost 30 years later:

Thanks for pointing out my mistake of 29 years ago. I never could remember where I had first seen the decimal point story – I thought it was in Reader’s Digest. I am very pleased to see that you have uncovered the whole story and very willing to admit that I was wrong. Incidentally my name is Terence not Terrance. (Sutton, 2010b, ‘comment to the article’)

Completion of the circle

It turns out that Reader’s Digest has in fact published the story of the decimal point error. The following quote is from a book titled Facts and Fallacies: Stories of the Strange and Unusual, published in 1988, 7 years after Hamblin made his unfortunate statement in British Medical Journal:

How Popeye got it wrong

Popeye the Sailor has done more for spinach than any salesman could ever have dreamed possible. When Popeye made his first appearance in the 1930’s, spinach consumption in the United States rose by 33 percent. Why? It was spinach that gave Popeye his mighty strength and bulging muscles, because it was so full of iron. But the belief that spinach promotes strength was based on a very simple mathematical error. Nutrition researchers in the 1890’s put a decimal point in the wrong place, thus giving spinach 10 times more iron than it actually contains. (Reader’s Digest Association, 1988: 264)

Popeye was not wrong at all, but Hamblin was, and with him a formidable flock of academics and others – among them Reader’s Digest Association – who uncritically followed in his footsteps. The decimal point error that Bender hinted about, and which Hamblin, based on rather obscure evidence concluded actually occurred, has now become common knowledge within sectors of academia to the extent that it can be used in journal articles and books without any reference at all, neither to Bender, Hamblin, Larsson, nor others (see, for example, Adesman, 2009: 39; Coughlin and Zitarelli, 1984: 116; De Beuckelaer, 2002: 194; Gustavii, 2003: 116).

Honor where honor is due

Even though the decimal point error appears to have become common knowledge in some environments, there is no doubt that Hamblin in many cases gets his deserved credit for his ‘discovery’. There are, however, good reasons to question to what extent Hamblin appreciated statements such as this one:

Hamblin debunked the belief that spinach is a rich source of iron by tracing the Popeye-spinach myth to a mistake by the original investigators in the 1930’s who put the decimal point in the wrong place and made a ten-fold overestimate of iron content. (Skrabanek and McCormick, 1989: 32)

The quote is from a widely cited book, somewhat ironically entitled: Follies and Fallacies in Medicine. We have seen, however, that Hamblin was not at all the one who debunked the exaggerated values for iron in spinach, that the exaggeration took place long before the 1930s, and that there is no evidence indicating that the decimal point error ever occurred. If we go back to the first introductory quote from Larsson’s article, we will get an excellent example of this puzzling but not infrequent phenomenon in scientific publications: authors who independently of each other manage to commit the same set of errors. Larsson (1995) does not refer to Skrabanek and McCormick (1989) but makes exactly the same combination of mistakes. In addition, his wordings are strikingly similar to Skrabanek and McCormick’s. The great question is, of course, whether Larsson has actually consulted the source he directly referred to (i.e. Hamblin, 1981), or whether he took the message and the reference from a book that does not appear at all in his list of references: Skrabanek and McCormick (1989).

Skrabanek and McCormick are most likely the originators for the rather impressive concentration of errors that I have so far have attributed to Larsson. Whatever types or combination of shortcuts Larsson has taken, he has provided us with an excellent example of the topic contained in the title of his own article: ‘The dissemination of false data through inadequate citation’.

When academics plagiarize from each other, whether it is an idea or a reference, a single (and highly erroneous, as in this case) interpretation will appear as two or more mutually independent statements, reinforcing the reliability or truth value of each other in a way that is entirely undeserved. This is likely an important part of the explanation behind the fact that some academics no longer find it necessary to refer to a source when telling the story about the decimal point error. In the past 30 years, a large number of apparently independent sources have mutually confirmed the ‘fact’ that a decimal point error was made, whether it was in 1870, in the 1890s, or in the 1930s. Nothing indicates that the decimal point error ever was made, but the account about it will most likely live a long and colorful life, just like its parent myth, the belief that spinach is a good source of iron.

Patterns of irony

There seems to be a remarkably consistent pattern of irony in the ways various authors have dealt with the decimal point error. By using the non-existent, or at least undocumented, decimal point error as an illustration of the importance of accuracy and critical investigation, Bender, Hamblin, Larsson, and Skrabanek and McCormick all end up doing the opposite of what they are preaching.

Follies and Fallacies in Medicine by Skrabanek and McCormick (1989) deserves special attention, not just because it is probably the most widely distributed and read printed version of the decimal point error story. These authors are concerned with

distortions which set obstacles in the path of rational thought and enquiry. The progress of science and the growth of knowledge depend upon clearing away rubbish and challenging accepted dogma and belief. (Skrabanek and McCormick, 1989: 1)

When the third edition of the book was published in 1998, the original section on spinach was kept intact, including the rather obvious misrepresentations of what Hamblin actually wrote on the issue and his role as a ‘debunker’ of the exaggerated reports on iron content in spinach. James McCormick brought the irony to yet another level in the preface to the third edition:

One of the necessary tasks which preceded this preparation was to reread the previous (second) edition. The most surprising thing was that there did not appear to be anything which required deletion or modification. The Follies and Fallacies which we described are even more widespread than before. All I have been able to do is to add a small number of more recent references which demonstrate the persistence of sloppy thinking and foolish conclusion. It would be gratifying if we continue to be read. (Skrabanek and McCormick, 1998: ix)

Since the first edition was published in 1989, the book has been reprinted several times and translated to seven languages. According to Philip Steer, an Emeritus Professor who reviewed Follies and Fallacies as a ‘medical classic’ in British Medical Journal in 2008, the book ‘is on the reading list of medical schools around the world, to encourage an appropriate scepticism about medical dogma’ (Steer, 2008: 673).

Bender, Hamblin, Larsson, and Skrabanek and McCormick are all involved in projects trying to establish a boundary between science and knowledge, on the one hand, and bad science, non-science, or plain nonsense, on the other. In their endeavor, however, they end up violating a large number of basic academic principles for handling of knowledge, sometimes in outright spectacular ways. This puzzling pattern of irony may perhaps be illuminated by an observation made by Sergio Sismondo (2005), drawing upon Gieryn’s concept of ‘boundary-work’, in his article on the ‘Science Wars’:

When people discuss the work of competing fields, misrepresentations on a scale that would never be acceptable within their own field are perfectly ordinary. … philosophers are willing to take negative portrayals of competing fields at face value, and are uninterested in interrogating them closely. (p. 245)

Popeye always wins

In February 2011, an article titled ‘Popeye had it right: Spinach really does make you stronger’ was published in the web version of The Independent (2011). Here, a new nutrient, inorganic nitrate, pops into the scene, complicating the discussion on whether iron or vitamin A is the source of Popeye’s strength. This is a topic that is far beyond my own fields of competence, but I nonetheless found something interesting among the readers’ comments at the end of the article:

The story that the iron content of spinach was a myth based on a misplaced decimal point is itself a myth. Spinach has a lot of iron, just like other green vegetables, but it is unavailable for absorption.

I should know, I was the one who was responsible for propagating the myth in a BMJ article.

The comment was signed ‘Terry Hamblin’. It will most likely do little to curb the spread of the academic urban legend that was born at Christmas time in 1981. Nonetheless, if the signature is authentic, there is something heroic about this public confession. This comment led me, for the first time, to click the ‘like’ button often found next to such readers’ comments.

The invisible heroes

There is something sad about all the urban legends that are brutally falsified or lose their edge because so many people find meaning in investigating them and publish their findings in books and on web pages for everyone to see. Urban legends are not just fascinating, entertaining, and colorful stories; they are also a part of our social communicative repertoire. When we lose them, we have to find other things to talk about, which are most likely not as funny, engaging, or bridge building as urban legends.

Nevertheless, I do not feel bad about having contributed to destroying the stories about spinach being a good source of iron, or the myth that a decimal point error made millions eat more spinach. Academic publications should have different standards and requirements regarding truth and accuracy from other mediums, and should not be a playground for rumors and urban legends. Accurate, complete, and relevant references to reliable sources are the best tools in order to avoid such a scenario.

This article may have drawn a rather dismaying picture of academic citation practices. There is, however, much to be admired in the ways various actors have approached the rumor that Hamblin accidentally turned into a piece of fact. Kronick sensed that the documentation was not as solid as we would expect, and included his small reservation. Sutton did a tremendous job, digging back to the 19th century and exposing for us all that Hamblin’s debunking of the myth about the exaggerated iron content was itself a myth. Another hero in this scenario is Hamblin himself. He admitted his mistake, not only on Sutton’s web page and in the commentary field of a spinach-related newspaper article. In December 2010, he published a detailed account titled ‘Spinach – I was right for the wrong reason’ on his own web site:

Now some fascinating research by Mike Sutton has found out the whole truth behind the decimal point and the iron in spinach myth and I am pleased to be able to say that I was right about spinach being useless as a source of iron, but utterly wrong about why the myth has taken hold. … The moral of this story is that a good story is not necessarily a true story. (Hamblin, 2010)

Hamblin committed a blunder more than 30 years ago, but he was doing his best until he died in February 2012 to stop the urban legend he was instrumental in creating. It was a hopeless fight, and the main reason is that many of us have a tendency to assume that everything we see in print is true, with or without a reference, as long as it is printed in a fairly respectable scientific journal.

Perhaps the greatest heroes are, however, invisible in this landscape: those who read about the decimal point error in one or more of these sources, but found out that they could not use it because the references and the documentation were not solid enough. Individuals with such attitudes are among the most important propellers of scientific development and accumulative knowledge, but many of them nonetheless end up as losers in systems where quantity is more important than quality, and where academic production is reduced to units being counted, rather than something worth taken into account.

An avalanche of low-quality research?

Are we experiencing an ‘avalanche of low-quality research’, or are the complaints about the current state of affairs primarily based on grumpy old-timers using the ‘good old days’ to kick the young and upcoming? Recently S. Rajasekaran (2012) expressed his worries in an article titled ‘Publish to flourish: Is it corrupting science?’ He laments that ‘the published paper is no longer sacred’ (p. 6), pointing to increasing incidences of fraud, plagiarism, and consequent retractions. Young researchers should be dissuaded ‘from joining the treadmill of the numbers game and encourage[d to pursue] focused publishing of high-quality research instead’ (p. 7). This is an argument in line with a large number of other commentators, but what is particularly welcome in Rajasekaran’s contribution is that he presents hard facts that are well suited to illustrate that there is good reason to ask the question of whether we are experiencing an ‘explosion of knowledge or just junk science’:

data show that only 45% of the articles published in the 4500 top scientific journals are cited within the first five years of publication, a figure that appears to be dropping steadily. … it appears that most published papers are inconsequential to science and simply pad the curricula vitae of researchers. (p. 5)

These are useful figures for those of us who may be accused of merely using anecdotal evidence when lamenting the current deterioration of academic standards. There is a problem, however, with this shockingly low figure and the enormous amount of time, energy, and money invested and apparently wasted in the remaining 55 percent articles that have not been cited 5 years after publication. The reference accompanying these figures leads us to Bauerlein et al. (2010) and their commentary we started out with in The Chronicle of Higher Education, ‘We must stop the avalanche of low-quality research’. This is obviously a secondary source, but even more worrisome, it is a paper that does not have any references at all. There is, however, enough information in the text to make it possible, with some creative searching, to make a fairly good guess at where the authors got the 45 percent figure from: a news article in Science (Hamilton, 1990), based on calculations from papers published between 1981 and 1985. In other words, Rajasekaran is using somewhat old figures to illustrate the miserable state of affairs in 2012.

The main problem with the 45 percent figure is not that we have to work hard in order to find out where it comes from, nor that it is old and rather irrelevant for the situation in 2012. We have in fact stumbled across yet another academic urban legend. The calculations behind the figure were based on a very heterogeneous composition of publications, including editorials, meeting abstracts, obituaries, notes, and letters, that is, types of works that are rarely cited, and for very good reasons (Pendlebury, 1991). A few years later, Eugene Garfield (1998), the founding father of the Institute for Scientific Information (ISI), wrote ‘I can only groan when I see errors perpetuated year after year’, pointing to the fact that Hamilton’s ‘misguided reports on uncitedness have unduly influenced many scholars and policy makers ever since’. Lariviere et al. (2009) state the claim ‘that most articles are never cited [is] a common lore that comes back periodically in the literature’ (p. 858), and their conclusions about uncited publications are in fact directly opposite to what Rajasekaran wrote 3 years later. Lariviere et al.’s data show that most scientific articles are in fact cited, and that rates of ‘citedness’ are not dropping at all, but are steadily increasing.

If we are concerned about the consequences of ‘low-quality research’, ‘junk science’, or the prevalence of academic urban legends, there are good reasons to shift the focus away from uncited and ignored publications, and worry instead about those that are cited, but which should not have been published. I will refrain from joining the debate on whether academic standards are deteriorating, or whether scientists are busier and take more shortcuts than before. What is clear, however, is that since 1981, the year Hamblin published his article about the decimal point error, the digitization of knowledge has dramatically improved the tools we have for enhancing the quality of research, including verification of references and documentation. The digital revolution has provided us with marvelous weapons for exposing and cutting down the prevalence rates of rumors and urban legends in academia. The problem is that the same devices can also be used for other purposes, such as contributing to what Haralambos Gavras (2002) has called the ‘Lazy author syndrome’: throwing a few keywords into a database to come up with an impressive list of references which at first glance cannot easily be exposed as secondary, irrelevant, unreliable, or sources not even read by the author.

The digital revolution has certainly made it easier to expose and debunk myths, but it has also created opportunities for new and remarkably efficient academic shortcuts, highly attractive and tempting not just in milieus characterized by increasing publication pressure and more concerned with quantity than quality, but also for groups and individuals strongly involved in rhetorics of demarcation of science, but less concerned with following the scientific principles they claim to defend. Some academic urban legends may perish in the new digital academic environment, but others will thrive and have ideal conditions for explosive growth.

Author biography

Ole Bjørn Rekdal is Associate Professor at Bergen University College (Norway). He is a social anthropologist and has co-coordinated several health-related interdisciplinary research and competence-building programs in East Africa. His teaching and research activities are primarily within medical anthropology, and he has a special interest in citation practices, handling of knowledge, and information-seeking behavior in academia.