Misinformation in and about science

Abstract

Humans learn about the world by collectively acquiring information, filtering it, and sharing what we know. Misinformation undermines this process. The repercussions are extensive. Without reliable and accurate sources of information, we cannot hope to halt climate change, make reasoned democratic decisions, or control a global pandemic. Most analyses of misinformation focus on popular and social media, but the scientific enterprise faces a parallel set of problems—from hype and hyperbole to publication bias and citation misdirection, predatory publishing, and filter bubbles. In this perspective, we highlight these parallels and discuss future research directions and interventions.

Misinformation has reached crisis proportions. It poses a risk to international peace (1), interferes with democratic decision making (2), endangers the well-being of the planet (3), and threatens public health (4, 5). Public support for policies to control the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is being undercut by misinformation, leading to the World Health Organization’s “infodemic” declaration (6). Ultimately, misinformation undermines collective sense making and collective action. We cannot solve problems of public health, social inequity, or climate change without also addressing the growing problem of misinformation.

Most of the research efforts and interventions examine broad, public consumption of misinformation—modeling the spreading dynamics of falsehoods (7, 8), examining social network effects (9, 10), and evaluating crowd-sourced mediation (11), with a special focus on crisis events (12) and political elections (13). In this article, we turn the spotlight on science. We look at the ways that misinformation can travel within science due to misaligned incentives, out-of-date publishing norms, and sociotechnical systems that concentrate attention and credit on a small subset of the literature.

Appealing as it may be to view science as occupying a privileged epistemic position, scientific communication has fallen victim to the ill effects of an attention economy. This is not to say that science is broken. Far from it. Science is the greatest of human inventions for understanding our world, and it functions remarkably well despite these challenges. Still, scientists compete for eyeballs just as journalists do. They face incentives to hype their work and to publish selectively those findings that are surprising and “clickable.” Like other information consumers and producers, researchers rely on search engines, recommendation systems, and social media to find relevant information. In turn, scientists can be susceptible to filter bubbles, predatory publishers, and undue deference to the authority of numbers, P values, and black box algorithms.

Hype and Hyperbole

The internet has changed the way we interact with the media. Some of us still read the morning paper at the breakfast table, but many more get the majority of their information on the internet. More people consume news online through social media (20%) or online news sites (33%) than in print form (16%) (14), especially among younger readers (15). While nearly half of Americans still get the bulk of their news from broadcast media (14), many of these sources are dangerously hyperpartisan.

Nearly 40% of Americans viewed content from untrustworthy websites during the 2016 US election, but these articles only represented about 6% of all news articles consumed (16), and sharing of “fake news” may be less prevalent than often reported in the media (17). That said, even seemingly trustworthy news sites push misleading headlines. Headlines from rival publishers appear side by side on our phones, competing for the ensuing clicks that drive advertising revenue. The unvarnished truth is not always enough to capture our attention. Thorough, detailed, accurate reporting has to compete with clickbait that manufactures emotional responses, curiosity, and anticipation (18). How can thoughtful analysis of minimum wage rates and unemployment trends possibly compete with yet another celebrity breakup, the one hygiene trick your dentist does not want you to know, or nine cats who look like characters from The Office? Publishers need flair, fluff, and sparkle to draw our attention—and they have responded enthusiastically.

Nuance falls by the wayside. Headlines often replace factual statements with promises of emotional experiences. Highly shared Facebook headlines pledge to “make you cry tears of joy,” “give you goosebumps,” or “melt your heart” (19). Rather than summarize the contents of the story, headlines deliberately obscure them to incite a click: “How to avoid the leading cause of death”; “Do economists think the Fed will cut rates?” Forward reference headlines (20) exploit our curiosity (21) by replacing key pieces of information with forward-referring pronouns. The reader has to click the story to discover their referents: “These pets are adorable but may carry a deadly disease”; “One-fifth of this occupation has a serious drinking problem”; “Many scientists overlook this crucial detail when reading PNAS.”

Parallel changes have taken place in the way that scientists write and read scholarly articles. Two decades ago, we received hard copies of journals and browsed through them regularly. Now, we mostly find articles using search engines or sometimes through social media. The result is a head-to-head competition between journal articles that parallels the competition among news stories on a smartphone. Among scientific papers, titles with positive, more interesting framing receive higher Altmetric scores (22).

Scientists likely feel increased pressure to hype their results because productivity metrics have taken on a greater role in scientific advancement (23). A publication is no longer merely a way of reporting results; it is a coveted prize that can make or break an early career (24). In some countries, a publication in a top venue draws bonuses—in China, up to $165,000 US dollars (25), although this practice was recently banned (26). Given that top journals often look for exciting results of broad impact, these policies encourage researchers to hype their work. Worse still, they may encourage fraud.

During a crisis, science can be forced into the media spotlight. Eager to accelerate the research cycle during the ongoing pandemic, scientists are making extensive use of preprint servers for polished papers and preliminary work alike (27). This can be a valuable mode of communication among researchers, but because it takes place in the open, journalists pick up on the work and do not always approach the findings with sufficient caution.

For example, there is no credible evidence that SARS-CoV-2 responsible for the COVID-19 pandemic has a bioengineered origin, but a series of preprints has pushed false narratives along these lines. One such paper, posted to bioRxiv (28), was quickly refuted by bioinformaticians and formally withdrawn—but in the interim, the paper received extensive media attention. If preprint servers try to vet the material, authors find other outlets. A two-page note—not even a research paper—claimed that SARS-CoV-2 is an escaped bioweapon and was posted to the academic social media platform ResearchGate (29). Though quickly deleted from the site, this document took off, particularly within conspiracy circles. A deeply flawed paper making similar arguments was posted to the file-sharing site https://zenodo.org/ (30). It received considerable attention after the author appeared on cable news promoting the claims and the US president tweeted a video clip of a cable news host praising the work (31).

Increasingly, we see research being released to the media prior to any publication even available for critique. Controversial work on detecting sexual orientation from photographs using artificial intelligence (32) was reported in The Economist and The Guardian before a preprint or white paper was available. Reports went viral about a research paper on the spread of COVID-19 by respiratory droplets from joggers—but no such paper existed, only an animated computer visualization (33). Controversial results from a Los Angeles County COVID-19 seroprevalence study were reported worldwide based on a press conference (34), but detailed information about methods and results was unavailable until weeks later.

In addition, researchers commonly misstate or overstate the implications of their work (35). In concert with researchers, university press offices play a particularly important role in communicating science—but too frequently do so in ways that prioritize web traffic over accuracy. Sometimes spin is carried over from the journal article itself (36); other times, it is added in the press release. A biomedical report might omit important caveats, draw inappropriate extrapolations from mouse models, and exaggerate prescriptive implications. One analysis found that nearly a third of 525 papers in top obesity or nutrition journals make inappropriate causal claims in their abstracts or titles (37); in another study, roughly the same fraction of health-related papers widely shared on social media used inappropriately strong causal language (38). Some fields may be more prone to hype than others. A new result on the geometry of Banach spaces may be more difficult to hype than a bioweapon claim, but we surmise that most fields are susceptible.

Much of this truth bending may be unnecessary. Most studies (39–41) fail to find an association between exaggeration and uptake by the news media. Admittedly, selection bias may play a role: perhaps the stories that are not exaggerated are those that do not need to be. In any case, high-quality press releases appear to drive higher-quality news stories on that research (42, 43).

Publication Bias

If you get your ideas about risk and safety from watching crime dramas—or even the local news—you probably think the world is a dangerous place (44). Intruders attack sleeping homeowners, children are kidnapped, and museums are burgled. “If it bleeds, it leads”—news outlets eager to attract views know that frightening stories of danger and tragedy capture our attention. We all want to learn what circumstances to avoid. The stories do not even have to be true; we all tend to talk about what scares us. In the 1970s and 1980s, urban legends about razor blades in apples led local police stations to set up X-ray machines for scanning Halloween treats and drove some communities to contemplate trick-or-treating bans—despite the fact that the scare was almost entirely fictitious (45, 46). An urban legend from our youth, about a parking lot slasher who hides under cars to slice his victims’ Achilles tendons (47), has reemerged in 2020, updated for the social media platform TikTok with a new twist about human trafficking (48).

Science features an analogous filtering process, though the bias trends toward good news rather than bad. One of the more disturbing realizations of the past decade is that many established scientific results in the social (49, 50) and biomedical (51–53) sciences cannot readily be replicated. This so-called “replication crisis” has been driven in part by the incentive structure of scientific publishing. Journals preferentially publish positive results with statistically significant outcomes. Scientists who obtain negative results or nonsignificance may choose to move on to another project rather than to invest in writing and publishing work thought to be of only modest interest. The result is publication bias, whereby the published literature provides a biased sample of the research actually conducted (54). With negative results buried in file drawers (55), conclusions drawn from the published record can be misleading, and “false facts” can become canonized in the literature (56).

How bad is the problem? We do not really know. It is relatively straightforward to measure the fraction of published results that are negative. One study found that only 15% of results published across the sciences are negative, with even lower levels in some fields such as ecology and psychology (57). However, to evaluate the effect of publication bias, we need to know what fraction of negative results is unpublished. To get at this more difficult estimate, Turner et al. (58) compared the Food and Drug Administration (FDA) registrations of antidepressant clinical trials with the published record in biomedical journals. In the published literature, 94% of the reported trials obtained positive results of drug efficacy. However, looking at the original registrations and the results as reported to the FDA, the team saw a different picture. Only 51% of the studies yielded definitively positive results according to the original outcome measures. Why the discrepancy? Almost all of the positive results were published, whereas fewer than half of the questionable or negative results were published. Moreover, many of the questionable or negative results were recast as positive via “outcome switching,” the questionable practice of reporting different outcome measures than those specified in the original trial registration. Reading the published literature, you would think antidepressants were ubiquitously effective. Seeing the full picture, the prognosis is more nuanced.

In response, researchers and publishers are beginning to experiment with registered reports (59, 60). Under this publishing model, reviewers evaluate proposed studies before they are conducted and offer in-principle acceptance: irrespective of the results, the study will be published if properly conducted. Advocates suggest that reviewing proposals instead of completed experiments will create a more reliable literature, both by reducing the incentive for scientists to mine data for surprising findings and by reducing publication bias against negative results. However, we do not see preregistration as a panacea. It may not be appropriate for all types of research; it discourages exploratory research, which can generate important, unexpected findings, and there is little evidence to date that it will appreciably reduce publication bias (61–63).

A form of publication bias arises in popular science reporting as well. News media eagerly report potential breakthroughs, often failing to clearly indicate their preliminary nature. COVID-19 reporting is no exception (33, 64). The withdrawn bioRxiv preprint mentioned previously was promoted so broadly that it garnered one of the highest Altmetric scores of all time (28). As another example, a Financial Times headline proclaimed “Coronavirus may have infected half of UK population—Oxford study,” even though it was reporting on a preliminary white paper that neither showed nor attempted to show anything of the sort (65, 66).

In pursuit of a splashy result, journalists sometimes extrapolate too far from scientific reports. One striking instance occurred when the Centers for Disease Control and Prevention (CDC) released guidelines indicating that because of acquired immunity, patients ordinarily did not have to be retested in the 3 mo after recovering from COVID-19. What the CDC was saying was that if you are within 3 mo of a previous infection ($p$), then you will ordinarily not be susceptible to reinfection ($q$). While the CDC asserted $p\to \neg q$, media reversed this, incorrectly interpreting the CDC as saying that $\neg p\to q$. That is, if you are not within 3 mo of a previous infection, you are necessarily susceptible to reinfection. Based on that misunderstanding, numerous news media erroneously reported that immunity to COVID-19 was now thought to be extremely short lived.

Another problem with reporting on preliminary studies is that journalists seldom report when the studies covered previously fail to pan out (67). Additionally, because journalism favors clicks, there is a heavy focus on findings that are highly surprising and perhaps, less likely to be correct. Reading about science in the mass media, one might reasonably conclude that man frequently bites dog but rarely, the converse.

Exaggeration in popular scientific writing misinforms the public, but it also misleads researchers. Even before the advent of online news and social media, scientific reporting in the popular press has been an important conduit for information even among professional researchers. A study based on papers published in 1978 and 1979 found that New England Journal of Medicine papers covered in the New York Times were cited at much higher rates than control papers, especially shortly after publication (68). Today, news articles, blogs, and social media are a valuable source of information about new research, particularly for younger scientists (69, 70). To the degree that those environments provide a distorted view and influence citations (71), scholars could be accordingly misled.

Citation Misdirection

In March of 2017, as the Trump administration’s aggressive antiimmigration stance stirred protests around the country, NBC News posted a tweet asserting that “International applications at American schools are down nearly 40%.” This struck us as an implausibly large effect size, given that many of these applications were submitted before Trump even took office. Indeed, if you trace back to the actual NBC News story (72), you will find that international applications went down at 40% of schools (by an unspecified amount), not by 40% total. That is a very different story. If you dig back further to the original scholarly report described in the news story (73), you will find that applications went down at 39% of schools but were up at 35% of schools. That is not news; it is Brownian motion.

Sadly, this is a common phenomenon. As information moves from primary literature to social media to popular press and back to social media, it is often distorted both intentionally and unintentionally like the messages in the children’s game of telephone.

One might think that the rigorous, disciplined nature of scholarly writing would prevent errors of this sort from arising in the scientific literature. Unfortunately, this is not so. Numerous studies have investigated the frequency of “quotation errors” (i.e., citations that are used to justify claims that are not factually supported by the cited documents). Depending on the field and the methodology, most analyses of the problem reveal that between 1 in 5 and 1 in 10 citations are used to support claims incongruous with the results of the cited paper (refs. 74–76 and references therein).

Intentions are difficult to measure; it is likely that many of these citations are due to honest mistake or laziness rather than deliberate obfuscation. When a paper misrepresents the papers it cites, this can be grounds for retraction (77). A bigger problem arises when one paper is frequently misrepresented by no fault of its own. In one notable case, a short letter published in the New England Journal of Medicine reported on opioid use and addiction among patients at the Boston University Medical Center (78). The authors concluded that “despite widespread use of narcotic drugs in hospitals, the development of addiction is rare in medical patients with no history of addiction.” This five-sentence letter has been cited over 600 times, most often as evidence for the incorrect assertion that opioids are not addictive. As of 2017, fewer than 20% of those citations acknowledged that the report was restricted to the hospital setting and does not apply to the in-home use where much of opioid addiction arises (79).

Retracted papers are frequently cited as legitimate even after retraction. In a recent study in radiation oncology, Daniel Hamilton (80) found that 92% of articles citing retracted articles subsequent to retraction cited them as if the retraction had never occurred. Presumably, this stems primarily from a lack of awareness, not deceitful intentions. The website retractionwatch.org/ lists the mostly highly cited retracted articles. A few observations are that retracted papers come from top-tier journals including New England Journal of Medicine, Science, and The Lancet; the top papers are cited thousands of times; and some papers are actually cited more after retraction than before retraction.

Citation bias is a related phenomenon, in which the claims associated with citations accurately report the results—but authors preferentially cite papers that support a claim over those that undermine it (81–84). Citation bias exacerbates the problems created by publication bias. If authors preferentially write up positive results and journals preferentially publish them, the citation record will be biased toward positive results even for incorrect hypotheses. If researchers also more likely cite positive results, the citation record will further distort our view of experimental findings.

Fake News and Predatory Publishing

The most successful fake story of 2016, “Pope Francis Shocks World, Endorses Donald Trump for President,” was published and spread by Macedonian teenagers who did not care a whit whether Trump or Clinton won the election (85). They were simply trying to generate advertising revenue—and they were wildly successful, bringing in hundreds of thousands of dollars.

This type of exploit became possible because of massive shifts in communication technology and associated economic structures for monetizing information. When the revenue model for news was based on subscriptions and circulation, there was little value to publishing a single catchy article; one needed an established paper, magazine, radio station, or television channel. Prior to the internet, authenticity was also hard to spoof. What malicious agent could print a million copies of a fake newspaper or take over television bandwidth with professional-quality broadcast content? Finally, how to attract readers or viewers? The onus was on the publisher to grow an audience through advertising and other costly measures. Social media and online ad revenue models allow anonymous or previously unknown actors to create and make money from content that can reach tens of millions of people.

A similar racket operates within the scientific ecosystem, in the guise of predatory publishers. Again, a shift in information technology made this possible. Digital typesetting and online distribution make authenticity easy to spoof: with a bit of know-how and a few days’ work, one can put together a website that looks like that of a scientific publisher. Changing economic models created new opportunities for malfeasance. The rise of electronic distribution established a market for online open access, in which the costs of publishing are borne by the authors instead of the readers. While the open access model has numerous advantages (86), it also results in a transfer of purchasing decisions from highly trained, highly motivated librarians deciding on journal subscriptions to untrained and heterogeneously motivated authors shopping for venues in which to publish single articles (87).

Predatory publishers are not invested in the gate-keeping, curation, and manuscript improvement roles of traditional journal publishers. They are focused on collecting open access publication fees, the funds that authors pay to make their work available to the world without subscription charges. How serious is the problem? According to one study (88), predatory publishers produced nearly half a million articles in 2014, bringing in around $74 million in publication fees. For comparison, the estimated market for reputable open access journals is around $250 million annually, and the number of articles in the Web of Science in 2014 was about 2.5 million. When including the entire literature, predatory publishing likely comprises about 5 to 10%.

So why do authors publish in these venues? Some authors may be duped by spam emails, but we suspect that in many cases, researchers are complicit. Scientists face strong pressures to publish frequently. With minimal or nonexistent peer review, predatory publishers offer an easy route to rapid publication (89). Thus, a predatory publisher may not need to fool prospective authors about its legitimacy. The publisher instead may be offering authors an opportunity to fool any bureaucracy or committee that assesses productivity by merely counting publications.

Yet more worrisome are the ways in which these publications mislead the public. Con artists publish fabricated or otherwise deceptive trials of snake oil therapies and use the publications in their sales pitches. The unapproved cancer treatment, Gc protein-derived macrophage activating factor (GcMAF), has been touted in several predatory journals (90). Denialists of various stripes—antivaxxers, creationists, HIV denialists, climate skeptics, chemtrail believers—use these venues for “peer review” legitimacy. This can be confusing to a public that has little training in detecting imposter science.

Scientists and the public need better ways of detecting untrustworthy publishers. We have developed methods for identifying suspicious journals that are exceedingly costly given their low influence (87), but more needs to be done to spot fictitious editorial boards and recently assigned web domains. Ultimately, the best solution will be to train the next generation of scientists, journalists, and the public to recognize legitimate scientific research (for a primer, see https://callingbullshit.org/tools/tools_legit.html).

Filter Bubbles and Echo Chambers

In the midtwentieth century, we relied on Edward Murrow and Walter Cronkite for nightly news. The rise of cable television and the 1987 repeal of the Federal Communication Commission’s fairness doctrine set into motion an increasing polarization of news (91). Today, algorithms learn to select content that our friends share, feeding us what we want to hear and not always what we need to know. As a result, we may be retreating into proverbial “filter bubbles” or “echo chambers,” despite increased access to diverse ideas, sources, and opinions. Some studies observe reinforcement of this sort (92); others provide conflicting evidence in both magnitude and direction (93–96).

Just as in society, gatekeepers are changing in science. Traditionally, journals have been the primary arbiters of content. Editors pick candidate papers; reviewers adjudicate. That has been the basic model for the last half century (97). However, over the past two decades, a new information milieu has emerged. Preprint archives, academic search engines, article recommendation systems, and social media do not require bound journals to deliver content. In this new communication environment, do journals still matter as gatekeepers, and do echo chambers exist in science?

In a recent study, we tracked citations of papers published on the arXiv before and after journal publication (98). After controlling for article quality, we find that arXiv articles published in higher-ranked journals received more citations than articles published in lower-tier journals. This indicates that journals retain gatekeeper roles for consumers. For producers, the story changes somewhat. We find that papers highly cited as preprints are less likely to be published in journals at all (98).

Changes in the curation and delivery of scholarly content extend beyond journals. Are search engines and recommender systems promoting epistemic diversity, or are they narrowing our view of the literature? One could easily imagine it going either way. Online access lowers the search cost of obtaining most articles; search engines and recommendation systems reduce the reliance on disciplinary journals. Thus, we might not be surprised that some studies have found that scientists read more broadly than previously (99). However, search engines such as Google Scholar return articles in an order influenced by previous citation counts and related criteria. This could easily accentuate a form of the Matthew Effect (100, 101) in which frequently cited papers attract an increasingly disproportionate share of citations as their fame grows. In our own investigations (102), we find minimal changes when correcting for marginals bias, which counters previous findings that show a narrowing of citation distributions (103, 104), but this result varies across disciplines.

Viewpoint diversity is important for science (105–107), so better understanding technology’s impact on this diversity is needed. In particular, we need to better understand the systemic effects of search engines on the literature. Google Scholar is one of the most important tools in science (108). Yet, the tool is a black box; the rules for ordering results are a mystery; the algorithms are continually changing, obviating any hope of reproducibility; the corpus is unknown, and estimates of its size differ dramatically (108); it is nonextensible and minimally customizable; and there has been little effort by Google Scholar to engage with researchers. Fortunately, there has been a flurry of development from other academic search engines including Semantic Scholar, Microsoft Academic Graph, Web of Science, and others.

Data and Science Distortion

Our world is quantified to an unprecedented degree. Our cell phones track our every move; arrays of ambient sensors monitor our cities; the internet of things tallies our domestic activity; and data exhaust from our online lives provides intricate detail about our interests, needs, and desires. Readily available data play an increasingly important role in decision making and public communication alike—but often, those data are misinterpreted by accident or cherry-picked to promote specific agendas.

Yet for all of the importance of data in contemporary decision making, we tend to associate misinformation with fake news or snake oil and less often think about how data—even accurate data—can misinform. Data appear objective, precise, and replicable but offer a near-endless array of presentations, framings, and comparisons that can be used to tell a wide range of stories. Matters get even worse with data visualization: choice of type, the scales and ranges of the axes, the bin sizes of histograms, the presence or absence of visual decoration, and other graphical conceits can influence a story in any direction a designer may desire (109–111). Without training, readers can be fooled readily. One recent study found that poor numerical literacy was associated with higher susceptibility to COVID-19 misinformation (112).

One of the most direct ways that numbers mislead is unfair comparison. For example, in the popular An Inconvenient Truth documentary about climate change, Al Gore showed increased monetary damages due to hurricanes (113). The data were correct, but costs were not corrected for inflation and rising home prices in coastal areas. Making these adjustments, the massive increase in hurricane damage largely disappears.

Even with the best of intentions, researchers can stumble when interpreting their data. Researchers try to navigate around statistical traps, including selection bias and confounds (114), data censoring (115), Simpson’s paradox (116), Will Rogers effect, (117), and observation selection effects (118). The ubiquitous but oft-misused P value even received a formal statement of caution from the American Statistician (119). With so many potential pitfalls, every statistical analysis deserves careful scrutiny. We need to better understand the scope across which numeric research findings can be generalized. While we often have intuitions about this, new work is finding ways to formalize it (120).

In the meantime, purveyors of propaganda go out of their way to create doubt even where it is unmerited. The field of agnotology studies how business interests, governments, and other agencies systemically create doubt around scientific findings and manipulate what we know and do not know about science (121). Whether designed to discredit the link between tobacco and cancer or to deny the reality of anthropogenic climate change, efforts at agnotogenesis—creating and spreading doubt—use a similar playbook (122). The aim is rarely to disprove the undesirable facts but rather, to induce sufficient doubt to “keep the controversy alive” and thereby, stave off regulatory action. The smoking gun is there for everyone to see; the goal is to provide people with alternative reasons to believe it might be smoking.

The “falsehood firehose” is another strategy that pushes huge volumes of self-contradictory disinformation (123), meant to deceive, confuse, disorient, and agitate (124, 125). The goal is not to promote one particular untruth but instead, to so thoroughly confound truth and falsehood that confidence in institutions—and even in the notion of truth itself—is undermined (123, 126). Recently, we have seen this approach adopted by science denialist factions as well (127). While perhaps accidental, the bungled COVID-19 risk communications out of the White House during February had similar effects. In late February 2020, for example, the president and director of the National Economic Council assured the US public that the epidemic had already been contained—at the same time as the director of the CDC was trying to brace the US public for extensive domestic spread and substantial disruption to everyday life. These and related blunders contributed to a growing sense of bewilderment and distrust toward the public health community.

Interventions

So what can we do about misinformation in and about science? Volumes have been written on regulatory, technological, and educational approaches to online misinformation (128–130), but this literature has largely focused on society broadly construed rather than on science in particular.

As a start, we should focus on incentives. The so-called New Economics of Science (131–133) models scientists as approximately rational actors motivated by nonepistemic considerations such as prestige and salary. Using this approach, we might be able to improve the efficiency of the scientific process by nudging science’s norms and institutions in the right directions. The aim is to create incentives that are compatible with the behaviors we want to encourage and that discourage the behaviors we want to eliminate (134).

Much of the present pathology of hype, hyperbole, and publication bias is associated with an overreliance on productivity metrics (23). Researchers, journals, and institutions are subjected to high-stakes quantification, from hiring to promotion and funding (135, 136). Goodhart’s law predicts the consequences. Restated concisely by Marilyn Strathern (137), the law observes that “when a measure becomes a target, it ceases to be a good measure.” Because universities and scientists are measured on these metrics, they face strong pressure to publish at high rates, and journal prestige takes on an inordinate significance (138). Scientific papers are “salami-sliced” into minimal publishable units, and claims are oversold. Though full-on P hacking may not be all that common (139), questionable research practices abound (140, 141), and the scientific enterprise rewards them (142–144) (but see ref. 145).

The peer review system is overtaxed by the volume of papers being written, and in many fields, there is no way for researchers to read the literature exhaustively (146). Changing the incentives around publication would help. Hiring committees, promotion committees, and funding agencies would do well to look closely at some fixed number of publications, thereby creating incentives for researchers to publish a smaller number of higher-quality papers (147).

We need to develop methods for identifying errors and statistical anomalies (148). We need to consider integrating preregistration (where appropriate) as standard practice to reduce the effects of publication bias, continue to develop tools for open science, and reward those scientists that adhere to these new standards. We need to encourage researchers to broaden their search platforms to reduce a possible “Google Scholar bubble.” We need better ways to evaluate reference lists to reduce citation errors. References are used not only by researchers but also, as primary input for search engine algorithms (149). They affect both the consumption and production of the literature. This may require an independent step in an already overtaxed peer review system, whereby additional reviewers examine only the citations. We need to do a better job helping the public identify legitimate science venues and strongly discourage scientists from publishing their research in predatory journals. Additionally, we need more science writers both within and outside science institutions. In 2009, there were only 79 full-time science reporters at newspapers in the United States (150). This paucity of science writers likely impacts public perception of, understanding of, and interest in science.

As society increasingly relies upon quantitative data, data reasoning skills become paramount. In 2017, we began developing a curriculum to address these issues of quantitative literacy (151). Our aims are twofold. First, we seek to teach students from nonscience, nonquantitative backgrounds how to hold their own in a data-driven society. We aim to dissolve the myth of numbers as impartial, hard, and unbiased; we show our students how to question numbers without technical training; and we do this, importantly, with a focus on how science works. Second, we aim to redress a major oversight in science, techology, engineering, and mathematics (STEM) education. In our experience, students develop impressive technical proficiency in coding, calculating, and conducting laboratory procedures. They less often receive adequate training in the elements of critical and humanistic thinking that underlie the productive use of these skills. Our class fills with students from more than 40 different majors, including many in the arts and humanities. We have shared our teaching materials with faculty from across the disciplinary landscape working at dozens of universities across the globe, and a number of universities now offer a similar course.

In our class, we present students with a simple schema for reasoning about data. Whether we are looking at statistical methodology, machine learning algorithms, or any other modes of data processing, there is a common structure to the analysis. First, data are collected. These go into a “black box” wherein the technical operations occur: logistic regression, random forest algorithm, or some other technology. The block box spits out summary statistics, data classifiers, or other forms of output. From that output, the investigator then derives various conclusions and interpretations. The black box may be inscrutable to most readers, but that is all right. Often, one does not need to open the box—to delve into the formal mechanics—to think critically about the analysis. When something goes wrong, the problem seldom resides within the black box (i.e., it is seldom a technical artifact of the analysis). Far more often, the data are flawed or unrepresentative, or the conclusions and interpretation are unjustified. Students do not need a great deal of technical training to spot these problems. Instead, we stress concepts such as selection bias, correlation vs. causation, relative vs. absolute risk, and plausibility checking via Fermi estimation.

For all these interventions, few will be effective if the public distrusts science. Pew Foundation surveys* of US residents have revealed declining trust in government, religious organizations, universities, business leaders, news media, and fellow citizens, with young people exhibiting particularly low levels of trust(152, 153). Fortunately, science remains among the few trusted institutions in the United States (154–157); however, that trust is declining in some regions and among some political orientations (158, 159).

Public engagement and understanding of science should be a priority for all scientists. This is not a matter of just teaching more astronomy or biology. Rather, it involves nurturing innate curiosity and teaching people to understand how science works, how to consider evidence when making conclusions, and how popular media distorts these conclusions. In our class, we spend nearly a quarter of our time talking about the nature of science and about the issues we have described here, from publication bias to predatory journals. We stress that while science has its problems, it incorporates mechanisms to correct mistakes. In our efforts, we have been inspired by the many other related courses developed elsewhere, notably Sense and Sensibility and Science at the University of California, Berkeley and Think Critically at the University of Texas at Austin and the University of Idaho.

We are optimistic that science and society alike will survive their immersion into new information technologies—but this will require education efforts in media literacy, data reasoning, and the philosophy of science. It will require policy makers and funders to support both research and public outreach, especially in rural regions of the world and in marginalized populations. Most importantly, this all needs to be done with a recognition that science relies on public trust for its funding and opportunities to interface with the world. Misinformation in and about science could easily undermine this trust. We cannot afford to let that happen.

Data Availability

There are no new data associated with this article.