Epistemic language in news headlines shapes readers’ perceptions of objectivity

Significance

Headlines are an influential source of information, especially because people often do not read beyond them. We investigated how subtle differences in epistemic language in headlines (e.g., “believe” vs. “know“) affect readers’ inferences about whether claims are perceived as matters of fact or mere opinion. We found, for example, saying “Scientists believe methane emissions soared to a record in 2021” led readers to view methane levels as more a matter of opinion compared to saying “Scientists know…” Our results provide insight into how epistemic verbs journalists use affect whether claims are perceived as matters of fact and suggest a mechanism contributing to the rise of alternative facts and “post-truth” politics.

Abstract

How we reason about objectivity—whether an assertion has a ground truth—has implications for belief formation on wide-ranging topics. For example, if someone perceives climate change to be a matter of subjective opinion similar to the best movie genre, they may consider empirical claims about climate change as mere opinion and irrelevant to their beliefs. Here, we investigate whether the language employed by journalists might influence the perceived objectivity of news claims. Specifically, we ask whether factive verb framing (e.g., "Scientists know climate change is happening") increases perceived objectivity compared to nonfactive framing (e.g., "Scientists believe [...]"). Across eight studies (N = 2,785), participants read news headlines about unique, noncontroversial topics (studies 1a–b, 2a–b) or a familiar, controversial topic (climate change; studies 3a–b, 4a–b) and rated the truth and objectivity of the headlines’ claims. Across all eight studies, when claims were presented as beliefs (e.g., “Tortoise breeders believe tortoises are becoming more popular pets”), people consistently judged those claims as more subjective than claims presented as knowledge (e.g., “Tortoise breeders know…”), as well as claims presented as unattributed generics (e.g., “Tortoises are becoming more popular pets”). Surprisingly, verb framing had relatively little, inconsistent influence over participants’ judgments of the truth of claims. These results demonstrate how, apart from shaping whether we believe a claim is true or false, epistemic language in media can influence whether we believe a claim has an objective answer at all.

Media consumption is central to how we form, maintain, and spread beliefs in the modern world. Moreover, how content is presented may be as important as the content itself. For example, consider “Scientists know methane emissions soared to a record in 2021” and “Scientists believe methane emissions soared to a record in 2021.” While “know” presents its complement (that methane emissions soared to a record in 2021) as a fact by presupposing that it is true, “believe” does not. Additionally, because believe does not presuppose the truth of its complement, it could be that it is not a matter of objective fact in the first place, communicating instead that methane levels could be a subjective matter of opinion.

Epistemic verbs such as “know,” “understand,” and “believe” that express propositional attitudes pervade widely consumed media (1–4). Yet, despite their prevalence, little is known about how their use in media influences people’s beliefs, both in terms of what people think is true and also whether they even think the broader issue has an objective truth value at all. Understanding both types of influence is increasingly important given the rising polarization and politicization of high-stakes social and scientific issues in US news media (e.g., climate change, vaccine safety, election integrity), where information from experts is often mediated through epistemic language (5–7). Furthermore, understanding how language shapes our perceptions of objectivity may reveal some of the mechanisms underlying the rise of “post-truth” politics and the dissemination of “alternative facts” (8, 9).

Prior work in communications has documented how journalists use epistemic language, hedges, and other expressions of (un)certainty (10, 11). These studies do not, however, explore the downstream influence of language on people’s beliefs. Moreover, they focus on breaking news, which employs expressions of uncertainty to communicate the limited and changing information available. While covering an ongoing fire, for example, journalists may report that “eyewitnesses believe the damage is significant.” In contrast, we examine how epistemic language shapes people’s inferences in nondeveloping contexts where journalists are better able to confirm the truth of claims.

Headlines are influential sources of information, constituting the majority of news media consumption (12), and are well-suited for controlled manipulation of epistemic language. In fact, many people rarely read beyond headlines (13), and even when they do, headlines can still affect the inferences readers draw from the article itself (14). Moreover, prior work has found partisan bias in how right- and left-leaning outlets use epistemic verbs to frame knowledge claims, such as a higher proportion of doubt-casting verbs in right-leaning articles on climate change (15). It remains unknown, however, to what extent such journalistic choices ultimately influence audiences’ beliefs.

Epistemic Language, Truth, and Objectivity

Research in linguistics with conversational content has documented when and how epistemic verbs presuppose truth (16–18) as well as how we make inferences about the world based on these semantic properties (19–21). In particular, factive verbs such as “know” and “understand” differ from nonfactive verbs such as “believe” and “think” in their entailment patterns: “Andy knows his package has arrived” entails that his package has indeed arrived; conversely, “Andy believes his package has arrived” does not necessarily entail that his package has actually arrived, even if it suggests that is has. In the case of news media, we would expect factive verbs to lead to stronger beliefs in headline claims, as their semantic properties predict. Nonetheless, a growing body of work has shown how contextual factors such as listeners’ prior beliefs and prosody can diminish the strength of factive entailments in conversation (22, 23). These and other pragmatic factors specific to a media context could modulate the influence of factive framing on truth judgments for headlines as well.

While prior work on epistemic language has focused on people’s perceptions of truth, our main goal in this work is to examine the influence of epistemic verbs on perceptions of objectivity. Although truth and objectivity are related, we argue that they are conceptually distinct. Regardless of whether a listener believes their package has arrived, for example, they likely consider the issue to be an objective matter of fact: either the package has arrived or not. In contrast, regardless of whether a listener believes a subjective claim, like chocolate being better than vanilla, they recognize that their preference is an opinion for which a ground truth does not exist.

Since using factive verbs implies that a ground truth exists, we predict their use might convey that the content they frame is more objective. Alternatively, using nonfactive verbs does not entail that a ground truth exists and therefore might convey that the content they frame is more subjective. For example, “most Americans think that climate change is man-made” could signal to a reader that a stronger factive verb (e.g., “understand”) is not justified, either because the true cause of climate change is unknown, or because the cause of climate change is a matter of opinion with no ground truth in the first place. Factive verbs do not carry the same uncertainty; “most Americans understand that climate change is man-made” unambiguously implies that there is a ground truth to the matter. Thus, framing claims with nonfactive verbs may lead readers to infer that the content is at least partially subjective, while factive verbs enshrine objectivity.

Judgments about objectivity also have implications for how people engage with evidence. While people’s beliefs about objective matters are in principle informed by empirical evidence, their beliefs about subjective matters do not hinge so much on the same considerations. Someone who views climate change as a matter of opinion, for example, may simply dismiss relevant evidence. Beliefs about subjective phenomena can even become decoupled from evidence entirely and instead aligned with group membership (24, 25) or identity more broadly (26, 27).

The Current Studies^*

To examine how epistemic language influences people’s beliefs about truth and objectivity, we showed them news headlines and manipulated which epistemic verbs framed headline content–factive framing (e.g., “Farmers know/understand drought conditions are worsening”), nonfactive framing (e.g., “Farmers believe/think drought conditions are worsening”), or no epistemic framing (unattributed generics; e.g., “Drought conditions are worsening”). Unattributed generics express direct statements without the qualification of any information source or their epistemic state, allowing us to examine how the presence of epistemic framing itself can impact participants’ judgments.

For each headline, participants 1) rated how much they thought the content of the headline was true (truth rating) and 2) the extent to which the content was a matter of fact or opinion (objectivity rating). To assess whether truth and objectivity ratings influenced each other, we manipulated across the studies whether participants rated both truth and objectivity in a within-subjects design (studies 1a–b) or only truth or objectivity in a between-subjects design (studies 2a–b, 3a–b, 4a–b). To investigate how epistemic language shapes beliefs about novel and familiar issues, we presented participants with headlines about relatively novel topics that they were unlikely to have strong prior beliefs about (studies 1a–b, 2a–b) as well as headlines about climate change (studies 3a–b, 4a–b), a familiar, polarizing topic.

Across topics, we predicted that participants would rate the content of headlines with factive verbs as more likely to be true and more objective compared to headlines with nonfactive verbs, although this effect could shift for misleading headlines. Additionally, because generics are often used to express facts and are particularly generalizable (28–30), we predicted that the content of headlines expressed as an unattributed generic would be rated as the most likely to be true and most objective.

Results

Studies 1a and 1b.

The first set of studies examined whether the presence and kind of epistemic verb in a headline simultaneously influences readers’ beliefs about the truth and objectivity of its content. Participants saw 15 headlines with factive, nonfactive, or unattributed generic framing (5 of each). For each headline, participants rated on a 0 to 100 scale how true the information in each headline was (e.g., “How likely is it that tortoises are becoming more popular pets?”) as well as how objective the headline content was (e.g., “Is there an objective answer to this question or is it more a matter of personal opinion?”). In study 1a, we used nonfactive “believe,” factive “know,” or no epistemic verb (unattributed generics). For the sake of generality, and because various epistemic verbs might pattern differently (31, 32), study 1b expanded the verbs to nonfactive “believe” and “think,” and factive “know” and “understand.”

To analyze an effect of verb type, we used two mixed-effects linear regression models, one for each rating type (truth, objectivity); rating was predicted by epistemic verb type (factive, nonfactive, none) as a fixed effect and random intercepts by participant and item.

Truth Ratings.

Across both studies, as predicted, participants rated content framed factively as more likely to be true than content framed nonfactively, (1a) t(2028) = 2.16, P = 0.031, (1b) t(3260) = 6.5, P < 0.001. Content framed with no epistemic verb (unattributed generics) was rated as most likely to be true overall, (1a) t(2028) = 3.93, P < 0.001, (1b) t(3260) = 6.3, P < 0.001.

Objectivity Ratings.

Across both studies, as predicted, participants also rated content framed factively as more likely to be objective matters of fact compared to content framed nonfactively, (1a) t(2028) = 4.9, P < 0.001, (1b) t(3260) = 8.47, P < 0.001. Content framed with no epistemic verb (unattributed generics) was rated as most objective overall, (1a) t(2028) = 10.33, P < 0.001, (1b) t(3260) = 11.03, P < 0.001.

In an exploratory analysis, we found that participants’ truth and objectivity ratings were correlated across both studies 1a (r = 0.49, P < 0.001) and 1b (r = 0.5, P < 0.001).

To summarize, when participants rated both truth and objectivity for the same headlines, those with factive verbs led participants to view their contents as more likely to be true and more objective compared to headlines with nonfactive verbs (Fig. 1).

Fig. 1.

Average rating by rating type and verb for studies 1a–b and 2a–b; error bars indicate 95% bootstrapped CI. When truth and objectivity were rated within-subjects (studies 1a–b), participants rated the contents of novel headlines containing nonfactive verbs (believe, think) as less objective and less likely to be true compared to headlines containing factive verbs (know, understand) or expressed as an unattributed generic. However, when rated between-subjects (studies 2a–b), this effect was only observed for objectivity.

Studies 2a and 2b.

In studies 1a and 1b, truth and objectivity were rated together and were correlated. It is therefore possible that one rating influenced the other. Thus, the next set of studies employed the previous procedure, but participants only rated truth or objectivity, between-subjects. We used the same analysis plan as studies 1a–b.

Truth Ratings.

Surprisingly, unlike the previous studies, participants in both studies 2a and 2b did not rate content framed factively as more likely to be true than content framed nonfactively, (2a) t(1987) = −1.28, P = 0.2, (2b) t(3402) = 0.93, P = 0.35. Unattributed generics were also not rated as significantly more likely to be true compared to content framed nonfactively, (2a) t(1987) = 0.09, P = 0.93, (2b) t(3402) = −1.3, P = 0.18.

Objectivity Ratings.

In contrast to their truth ratings, across both studies 2a and 2b, epistemic verbs continued to influence participants’ objectivity ratings. Participants rated content framed factively as more likely to be objective matters of fact compared to content framed nonfactively, (2a) t(2070) = 10.9, P < 0.001, (2b) t(3369) = 15.72, P < 0.001. Unattributed generics were rated as most objective overall, (2a) t(2070) = 22, P < 0.001, (2b) t(3369) = 20, P < 0.001.

In summary, when truth and objectivity were rated separately, epistemic verbs in headlines had little effect on whether readers viewed their content as more true but continued to influence how readers viewed the objectivity of their content.

Studies 3a–b.

The prior experiments used synthetic headlines about relatively neutral topics to explore the way epistemic verbs influence people’s views on issues for which they have weak prior beliefs. However, headlines are often about issues that people have already formed strong attitudes toward. Therefore, the subsequent set of studies examines the impact of epistemic verbs on headlines about climate change, a highly visible, politically contentious issue. For these studies, we assembled a dataset of actual climate change headlines from a variety of left- and right-leaning outlets using the MediaCloud API (33). More details are provided in Materials and Methods.

We then varied the verb framing in headlines following a procedure similar to studies 2a and 2b. Instead of “know” and “believe,” we used factive “understand” and nonfactive “think” in keeping with the most common epistemic verbs found in our dataset of climate change headlines. In addition, we used the verb “say” because it was the single most common verb used in climate change reporting and its influence was unclear. Study 3a examined headlines about specific climate change phenomena, e.g., methane emissions, while study 3b examined headlines explicitly mentioning climate change or global warming in general. We used the same analysis plan as the previous studies.

Truth Ratings.

Participants’ pattern of truth ratings was somewhat different for studies 3a and 3b. Participants did not rate content about specific climate change phenomena (3a) framed by “understand,” t(2678) = 1.22, or “say,” t(2678) = 0.15, P = 0.88, as more likely to be true than content framed with “believe.” Interestingly, unattributed generics were rated as the least likely to be true, significantly less than nonfactive verbs, t(2678) = −2.64, P = 0.008. In contrast, participants rated content about climate change more generally (3b) framed by “understand,” t(3071) = 3.09, P = 0.002, “say,” t(3071) = 3.03, P = 0.002, or no verb, t(3071) = 2.15, P = 0.031, as more likely to be true than content framed by “believe.”

Objectivity Ratings.

While participants’ truth ratings diverged in studies 3a–b, their objectivity ratings showed a broadly similar pattern. Participants rated content framed by “understand,” (3a) t(2734) = 6.77, P < 0.001, (3b) t(2689) = 7.8, P < 0.001, “say,” (3a) t(2734) = 7.05, P < 0.001, (3b) t(2689) = 5.32, P < 0.001, or no verb, (3a) t(2734) = 5.32, P < 0.001, (3b) t(2689) = 5.87, P < 0.001, as more likely to be objective matters of fact compared to content framed by “believe.”

In short, while the influence of epistemic verbs on participants' truth judgments was weak and varied between studies, nonfactive epistemic verbs reliably led readers to view headline content about climate change as more subjective (Fig. 2).

Fig. 2.

Average rating by rating type and verb for studies 3a–b and 4a–b; error bars indicate 95% bootstrapped CI. Studies 3a–b examined generally accurate headlines about climate change while studies 4a–b examined generally misleading or inaccurate headlines about climate change. Additionally, studies 3a and 4a presented headlines about specific climate change phenomena while studies 3b and 4b presented headlines about climate change in general. Across these studies, participants consistently rated the contents of climate change headlines containing “believe” as less objective compared to headlines containing “understand” or “say.” However, epistemic verbs had little, inconsistent influence over participants’ truth judgments.

Studies 4a–b.

While studies 1–3 presented participants with accurate or generally reasonable headlines, readers often encounter headlines that are misleading or false. How might epistemic language influence readers’ judgments in these cases? One possibility is that they affect accurate and misleading headlines similarly, with nonfactive verbs leading readers to view content as more subjective but not more or less true. Another possibility is suggested by recent work on belief- and fact-speaking, which finds that framing statements in terms of beliefs, as opposed to facts and knowledge, can be used to seemingly legitimate untrustworthy sources (34). If a reader already views something as unlikely to be true, factive framing could therefore lead to lower trust in the source, resulting in lower truth and perhaps even objectivity judgments. To examine whether epistemic language might affect readers’ intuitions differently when framing dubious content, studies 4a–b replicate studies 3a–b, but with misleading and false climate change headlines.

Truth Ratings.

Verb framing did not have a significant influence on participants’ truth judgments in either direction, (4a) ts(2701) < 1.1, Ps > 0.25, (4b) ts(2687) < 1.5, Ps > 0.25. However, participants’ overall truth ratings were notably lower compared to the content of accurate climate change headlines (studies 3a–b), (4a) t(5357) = 47.2, Ps < 0.001, (4b) ts(4960) = 60, Ps < 0.001, suggesting that participants were sensitive to the headline content itself.

Objectivity Ratings.

Participants’ objectivity ratings showed a broadly similar pattern to ratings for accurate climate change headlines (studies 3a–b). Participants rated content framed by “understand,” (4a) t(2704) = 7.9, P < 0.001, (4b) t(2660) = 4.7, P < 0.001, and “say,” (4a) t(2704) = 7.4, P < 0.001, (4b) t(2660) = 3.2, P = 0.001 as more likely to be objective matters of fact compared to content framed by “believe.” While content framed with no epistemic verb was rated as more objective compared to content framed by “believe” in study 4a, t(2705) = 2.5, P = 0.01, we observed the opposite for study 4b, t(2660) = −2.9, P = 0.004. Participants’ objectivity ratings in studies 4a–b were significantly depressed compared to studies 3a–b, (4a) t(4379) = 18.8, Ps > 0.001, (4b) ts(4343) = 27.7, Ps < 0.001, although less so compared to their truth ratings.

In sum, while participants generally rated inaccurate climate change headlines as less true and objective compared to the accurate headlines in studies 3a–b, once again epistemic language did not influence participants’ truth judgments. Contrary to the prediction that framing dubious content factively could undermine readers’ trust in the source and lead to lower objectivity ratings, “believe” continued to decrease perceived objectivity relative to “understand” and “say.”

Discussion

Our work investigates how epistemic language affects people’s inferences about the objectivity of claims in headlines. On the whole, when headlines use nonfactive verbs that do not presuppose truth (e.g., “believe,” “think”) compared to factive verbs that do (e.g., “know,” “understand”), people viewed their content as more likely to be mere opinions as opposed to objective matters of fact. This result was robust across a wide range of headline topics, including generally novel, uncontroversial issues, as well as both accurate and misleading headlines on climate change. Though we hypothesized that factive verbs could lead to lower objectivity ratings for misleading headlines, since factive verbs like “understand” may undermine the trustworthiness of the source when coupled with dubious claims (34), we continued to find that nonfactive verbs lowered perceived objectivity.

Impressively, judgments about objectivity were influenced by a single epistemic verb appearing in a single headline. This effect was most pronounced for headlines about unfamiliar topics (studies 1a–b and 2a–b), though still evident for headlines about climate change (studies 3a–b and 4a–b). This result suggests epistemic language can affect how people both form and update beliefs about a claim’s objectivity. Further, most people use social media to acquire at least some of their news, where they are exposed to a larger quantity of headlines at a time (4), often with partisan bias. For example, right-leaning sources (e.g., Fox News) use more nonfactive verbs and fewer factive verbs when discussing climate change (15). Thus, the influence of epistemic language could compound rapidly and lopsidedly. Whether intentional or not, the use of nonfactive verbs could represent a form of paltering (35, 36), in which a technically true statement is nevertheless misleading. Overall, epistemic language stands to impact whether we view topics as having a ground truth both after immediate and cumulative exposure, although more work is needed to understand how different patterns of exposure affect people’s beliefs.

Unattributed generics tended to lead participants to infer high levels of objectivity, especially for headlines about novel topics. One explanation is that, in contrast to epistemic framing, unattributed generics do not mention an information source, which may be perceived as susceptible to personal biases. Even though all headline information ultimately has some underlying information source, participants may still perceive headline content as more objective in the absence of explicit language that makes a source salient. Thus, unattributed generics are perhaps a strong way to state facts which are in principle externally verifiable. Our results coincide with existing research that has shown how generics are perceived as particularly generalizable (37, 38) and important (39), require little evidence for acceptance (28), and serve as a powerful learning mechanism for children and adults (40–42). The current findings contribute to this literature by showing how unattributed generics can convey that a statement is itself a matter of fact with an objective right answer. In the case of controversial topics, unattributed generics led to more inconsistent objectivity ratings, potentially due to the backfiring effect on source credibility (34).

Additionally, we examined the effect of the verb “say” because we found that it was the most frequently used verb in headlines about climate change and is often viewed by journalists as expressing an impartial stance. However, we found that “say” followed a similar pattern to the factive verb “understand” and consistently boosted the perceived objectivity of framed content.

Given prior work on inferences about factive verbs in conversational contexts (19, 20), we expected factive verbs to strengthen readers’ intuitions about truth in a media context as well. However, they only reliably did so when readers rated both truth and objectivity (studies 1a–b) but not when they only rated one or the other (studies 2–4). Although it may be tempting to attribute the weak effect on truth to general news skepticism, we found that participants routinely judged the content of neutral and true climate change headlines (studies 1–3) to be likely true, and rated the content of false headlines (studies 4a–b) to be likely untrue, showing that participants were still calibrating their truth judgments to the content of headlines.

One potential explanation is that, at least in a media context, epistemic verbs lead readers to make stronger pragmatic inferences about objectivity than truth. Both factive and nonfactive epistemic verbs imply that a speaker believes something is true, even if they might express different degrees of speaker confidence in that truth. For example, both knowing and believing that methane levels are increasing suggests a positive credence. Conversely, while factive verbs unambiguously imply that a speaker believes a truth exists, nonfactive verbs can be used to express both matters of fact and opinion (e.g., “I think apple pie is the best”). Thus, compared to factive verbs, there may be greater ambiguity in terms of what a nonfactive verb implies about an issue’s objectivity.

Another hypothesis is that reasoning about truth is a more automatic and intuitive task compared to reasoning about objectivity, and readers were able to rate the truth of framed content merely by assessing the claim itself. Conversely, reasoning about objectivity may be less intuitive and require more reflection, leading readers to consider the entirety of the headline, including the source’s epistemic state. Such an account could also explain why we observed an effect of verb choice on truth judgments when participants rated truth and objectivity together (studies 1a–b) but nowhere else.

Returning to objectivity, some factors we did not consider (e.g., audience political affiliation, trust in information source, headline tone) could mediate the influence of epistemic language. For example, framing political discourse in terms of subjective belief, rather than evidence-based knowledge, is becoming increasingly common among conservative politicians in the US (34), which may in turn affect how politically aligned individuals interpret and reason about epistemic language. Nonetheless, while people are more likely to trust headlines from outlets aligning with their own political affiliation, this influence of source is weak relative to the effect of the reported content itself (43, 44). Unfortunately, the political makeup of our sample—skewed slightly liberal with relatively few very conservative participants—renders the current studies underpowered to examine such an effect of political ideology or affiliation. It is also possible that in the studies about climate change, participants with greater trust toward scientific experts may have judged headlines containing such sources as more objective. Although audiences may believe information more when it comes from an expert source (45), no studies have examined how source expertise affects perceptions of objectivity.

Across all our studies, we found that framing a statement using a nonfactive verb led readers to judge that statement as more subjective compared to framing the statement factively. Importantly, we are not arguing that nonfactive verbs should never be used to discuss matters of fact or that communicating objectivity is always desirable. Moreover, epistemic language does not solely determine how readers perceive the objectivity of headline content. Many other factors, including felicity, novelty, and the nature of relevant evidence likely also influence readers’ perceptions of objectivity and may trade off with the use of epistemic language. Ultimately, headline writers should consider both the potential impact of epistemic language on readers’ beliefs as well as how headlines might shape the way readers perceive issues as matters of fact or opinion more broadly.

Materials and Methods

All studies were approved by the Stanford institutional review board (protocol 14174), and all participants gave informed consent.

Study 1.

Our hypotheses and analysis plan for studies 1a and 1b were preregistered at https://aspredicted.org/9mu4e.pdf and https://aspredicted.org/ry2a8.pdf.

Participants.

We recruited 152 participants (84 males, M.age = 27) for study 1a and 251 participants (120 males, M.age = 27) for study 1b from Prolific. They completed the study for $1.60 payment, taking around 5 min on average. Six were excluded from 1a and 17 from 1b for failing any of two attention checks.

Items.

We created 15 synthetic headlines of the form: [plural source] [epistemic verb] [complement clause], e.g., “Russian churchgoers believe/know that church attendance in Russia is declining”, “French crane operators believe/know that pigeon populations are growing in Paris.” When no epistemic verb was included for unattributed generics, the headline only contained the complement clause. The content of the headlines was designed to be reasonably, but not undeniably, believable. The subject of the headline was always a relevant, plural source rather than an individual to mirror real-world headlines containing epistemic verbs and avoid preferences for or against a particular individual. See https://osf.io/q3kwc/ for all stimuli.

Procedure.

The study was conducted as a Qualtrics survey. First, participants were told that they were going to read headlines and answer some simple questions about them and asked not to consult external sources. Next, they completed 15 trials plus an attention check trial. In study 1a, 5 trials each had headlines containing “believe,” “know,”, or no epistemic verb. In study 1b, 3 trials each had headlines containing “believe,” “think,” “know,” “understand,” or no epistemic verb. The headline–verb combinations were counterbalanced and their presentation order was randomized across participants.

For each trial, participants saw a headline at the top of the page. Two rating questions appeared as sliders ranging from 0 to 100 underneath. The first rating question (truth rating) asked participants to rate how likely the complement clause in the headline was true (e.g., “How likely do you think it is that church attendance in Russia is declining?”). A number was marked every 25 points, with three anchor labels, “very unlikely”, “moderately likely”, and “very likely”. The second rating question (objectivity rating) also used a 0 to 100 scale and asked participants to rate whether there is an objective answer to the above question or whether they thought it was more of a matter of opinion. The scale contained two anchors, “definitely a matter of opinion” and “definitely a matter of fact”. After completing all 15 trials, participants completed an additional attention check, a short demographic questionnaire, and were debriefed.

Study 2.

In studies 1a–b, participants completed both truth and objectivity ratings within-subjects. In studies 2a–b, participants either rated truth or objectivity between-subjects. Our predictions were the same as the previous set of experiments, preregistered at https://aspredicted.org/cy624.pdf and https://aspredicted.org/xg2dj.pdf.

Participants.

We recruited 303 participants (146 males, M.age = 29) for study 2a and 501 participants (237 males, M.age = 28) for study 2b from Prolific. They completed the study for $1.60 payment. Eleven were excluded from 2a, and 15 were excluded from 2b for failing the attention check.

Items.

We used the same headlines from studies 1a and 1b.

Procedure.

The overall procedure was similar to studies 1a and 1b, except participants were assigned to either the truth or objectivity rating condition between-subjects. Additionally, participants received two example items beforehand where the rating scale was explained. Participants then completed 15 rating trials. In study 2a, 5 trials each had headlines containing “believe,” “know,” or no epistemic verb. In study 2b, 3 trials each had headlines containing “believe,” “think,” “know,” “understand,” or no epistemic verb. The headline–verb combinations were counterbalanced and their order was randomized. In each trial, participants first read a headline by itself, then saw the headline with the rating question below.

In the truth rating condition, participants were asked whether they thought the complement clause was true (e.g., “Do you think church attendance in Russia is declining?”). The rating scale was a 0 to 100 slider with 5 anchors: “definitely not”, “probably not”, “neutral”, “probably”, “definitely”. In the objectivity rating condition, participants were asked whether they thought that there was a single, correct answer to whether or not the complement clause is true (e.g., “Do you think there is a single, correct answer (100) to whether or not church attendance in Russia is declining (a matter of objective fact), or do you think there are multiple possible answers based on personal opinion (0) (a matter of opinion)?”). The scale also had 5 anchors: “entirely a matter of opinion”, “mostly a matter of opinion”, neutral, “mostly a matter of fact”, and “entirely a matter of fact”.

We also conducted a full replication of study 2b with simplified language and scale for objectivity rating (e.g., “Is it a matter of objective fact whether or not church attendance in Russia is declining?”) and found that the pattern of results fully replicate; see SI Appendix for a full analysis.

Study 3.

In studies 1a–b and 2a–b, participants rated synthetic headlines about neutral topics. In study 3a–b, we turn to actual headlines about a controversial topic, climate change. Predictions and analyses were preregistered at https://aspredicted.org/ru4fq.pdf and https://aspredicted.org/ct544.pdf.

Participants.

We recruited 405 participants (207 males, M.age = 27) for study 3a and 401 participants (217 males, M.age = 29) for study 3b from Prolific. They completed the study for $1.60 payment. Seventeen were excluded from 3a and 12 from 3b for failing an attention check.

Items.

See https://osf.io/q3kwc/ for a list of all headlines used. We sourced headlines from a dataset of real-world climate change articles (N = 70K) published between January 2000 and April 2020 retrieved using the MediaCloud API (33). The keywords used for API requests were “climate change”, “global warming”, “fossil fuels”, “carbon dioxide”, “methane”, and “co₂”. We excluded articles with irrelevant URL tags (e.g., “/arts/”, “/obituaries/”) and duplicate articles using criteria adapted from ref. 46. From manual inspection, we found that many headlines from right-leaning outlets made misleading or false statements. We thus sampled headlines from only center- and left-leaning outlets with generally accurate climate change headlines.

We next applied a natural language processing pipeline to find headlines with the same syntactic structure as our artificial headlines, i.e. [plural source] [epistemic verb] [complement clause]. We further selected headlines that used “say” since it was the most common verb used by journalists and does not select for any particular semantic content of its complement clause. Next, we manually selected headlines from our filtered pool as stimuli. This selection process ensured that headlines were comprehensible without the broader article context, e.g., excluding vague headlines like “Scientists say that our response to climate change is missing something big.” We also selected headlines that reflected the target level of specificity: about a specific facet of climate change/global warming (study 3a) or about the phenomenon as a whole (study 3b).^† Finally, we performed a series of manual edits to ensure the consistency of headline syntax across items and to remove problematic references (e.g., overtly biased words like “alarmist”). See SI Appendix for additional details.

Procedure.

The procedure was identical to 2a and 2b, except that participants read 15 headlines about climate change phenomena. Additionally, headlines contained either “believe,” “understand,” “say,” or no epistemic verb. The headline-verb combinations were counterbalanced and their order was randomized.

Study 4.

Studies 4a–b replicated studies 3a–b, but with headlines that expressed skeptical, misleading, or false information about climate change. Predictions and analyses were preregistered at https://aspredicted.org/bb5w6.pdf and https://aspredicted.org/jf6jy.pdf.

Participants.

We recruited 401 participants (215 males, M.age = 36) for study 4a and 401 participants (162 males, M.age = 37) for study 4b from Prolific. They completed the study for $1.60 payment. Thirteen were excluded from 4a and 17 from 4b for failing an attention check.

Items.

See https://osf.io/q3kwc/ for a list of all headlines used. The selection process was similar to studies 3a–b, except we sampled headlines from right-leaning outlets that expressed general skepticism or falsehoods about the scientific consensus on man-made climate. Like studies 3a–b, we also selected headlines that reflected the target level of specificity: about a specific facet of climate change/global warming (4a) or about the phenomenon as a whole (4b).

Procedure.

The procedure was identical to studies 3a–b.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

Anonymized participant response data (as csv files) have been deposited in OSF (https://osf.io/q3kwc/) (47).