Generative artificial intelligence–mediated confirmation bias in health information seeking

Abstract

Generative artificial intelligence (GenAI) applications, such as ChatGPT, are transforming how individuals access health information, offering conversational and highly personalized interactions. While these technologies can enhance health literacy and decision‐making, their capacity to generate deeply tailored—hypercustomized—responses risks amplifying confirmation bias by reinforcing pre‐existing beliefs, obscuring medical consensus, and perpetuating misinformation, posing significant challenges to public health. This paper examines GenAI‐mediated confirmation bias in health information seeking, driven by the interplay between GenAI's hypercustomization capabilities and users’ confirmatory tendencies. Drawing on parallels with traditional online information‐seeking behaviors, we identify three key “pressure points” where biases might emerge: query phrasing, preference for belief‐consistent content, and resistance to belief‐inconsistent information. Using illustrative examples, we highlight the limitations of existing safeguards and argue that even minor variations in applications’ configuration (e.g., Custom GPT) can exacerbate these biases along those pressure points. Given the widespread adoption and fragmentation (e.g., OpenAI's GPT Store) of GenAI applications, their influence on health‐seeking behaviors demands urgent attention. Since technical safeguards alone may be insufficient, we propose a set of interventions, including enhancing digital literacy, empowering users with critical engagement strategies, and implementing robust regulatory oversight. These recommendations aim to ensure the safe integration of GenAI into daily life, supporting informed decision‐making and preserving the integrity of public understanding of health information.

As early as 2003, it was said that health information seeking was the most common reason for people to go online, with an estimated 6.75 million health‐related searches conducted daily. ¹ Ever since, information technology has been a double‐edged sword for public health. On the one hand, it has expanded public access to health information, ² , ³ , ⁴ enabling health professionals and organizations to share their knowledge more widely, ⁵ , ⁶ and empowering lay citizens to share their experiences. ⁷ , ⁸ , ⁹ , ¹⁰ On the other hand, the proliferation of false information—termed by the World Health Organization as an “infodemic” ¹¹ , ¹² , ¹³ —can drown out factual, scientific evidence. In this context, confirmation bias,^a the tendency to favor information that reinforces pre‐existing beliefs, ¹⁴ plays a significant role. Confirmation bias shapes how people search for, select, interpret, and recall health information online, often in selective and reinforcing ways. ¹⁵ Online algorithms, optimized for engagement rather than accuracy, exacerbate this issue. ¹⁶ By tailoring content to users’ preferences, these systems create feedback loops that amplify confirmation bias, potentially forming echo chambers that deepen these biases—although evidence for the existence of such echo chambers in online media consumption, particularly on social media, remains mixed, and further research is needed. ¹⁷ , ¹⁸ These dynamics can entrench misinformation and ultimately distort public understanding of health issues, ¹⁹ with significant implications for both individual and public health. Algorithms that curate and deliver health information influence decision‐making, increasing risks such as vaccine hesitancy ¹⁵ , ²⁰ , ²¹ and the adoption of scientifically unsupported therapies in critical medical situations. ²²

The rise of generative artificial intelligence (GenAI) offers new opportunities to mitigate these risks. ²³ GenAI can enhance the accuracy and personalization of health information, potentially reducing misinformation. However, GenAI also introduces new challenges that may further exacerbate existing risks. ²⁴ It extends user alignment capabilities through finer preference tuning, tailored content generation, a wide range of applications, and human‐like interactions that predominantly occur in private environments. ²⁵ The personalized and human‐like nature of GenAI interactions can reinforce users’ views (e.g., through sycophancy ²⁶ ) and amplify confirmation bias. ²⁷ Additionally, the private nature of these interactions limits access to observational data, hindering the design of effective interventions, regulation of content, and mitigation of risks associated with biased health information as GenAI systems become increasingly sophisticated.

In this paper, we examine how GenAI‐driven information environments can shape health information seeking by entangling with confirmation bias tendencies. We furthermore identify key behavioral pressure points (PPs) and propose recommendations at both individual and system/regulatory levels to address the risks of GenAI‐mediated confirmation bias in information seeking.

CONFIRMATION BIAS IN ONLINE HEALTH INFORMATION

Individuals often turn to the Internet for a variety of health‐related inquiries, ranging from self‐diagnosis of symptoms and seeking emotional support or validation, to supplementing or even replacing professional medical guidance. ²⁸ , ²⁹ , ³⁰ When confronted with specific health concerns, individuals are susceptible to confirmation bias, which leads them to prioritize information that resonates with their personal needs or anxieties over content that is credible, comprehensive, or medically relevant. ¹⁵ Confirmation bias can be exacerbated by individual and situational factors. At the individual level, those higher in cognitive reflection and reasoning ability were less likely to display confirmation bias. ³¹ The tendency to exhibit confirmation bias may also be stronger for certain beliefs, such as pseudoscientific beliefs ³¹ and vaccine skepticism. ³² People's own certainty in their own expertise or judgments can also increase their propensity for confirmation bias. ³³ At the situational level, the way a task is framed can affect how strongly confirmation bias shows up. For example, when people are asked to write about their impressions of a piece of information, they tend to show more confirmation bias than when asked to simply summarize it. ³⁴ The interplay between individual differences and online situations thus influences how confirmation bias may be exhibited in online behaviors.

Empirical evidence delineates three primary behaviors that can be susceptible to confirmation bias through different cognitive processes during online health information seeking. First, individuals tend to selectively browse content that aligns with their pre‐existing beliefs about a health topic. ¹⁵ , ³⁵ Second, the selection of information retrieved through online searches is often conducted in accordance with users’ pre‐existing beliefs. ¹⁵ , ³⁶ Third, content that contradicts users’ beliefs is frequently misinterpreted. ²¹ We refer to these behaviors as pressure points (PP1, PP2, and PP3) (see Table 1), which are central to our analysis of how confirmation bias affects digital health interactions. These PPs can help us to understand different individuals’ susceptibility to confirmation bias and situational factors that may exacerbate them.

TABLE 1.

Confirmation bias can intensify through three core pressure points.

Pressure point	Before GenAI arrived	GenAI	Recommendations
PP1: Query Formulation Users phrase queries in ways that reflect their beliefs, leading to biased results	Search engines provided results ranked by relevance but often failed to address inherent biases in user queries 40 , 41	GenAI hypercustomizes outputs to user preferences, often amplifying biased queries. 25 While it could help reframe them, current systems rarely do so unless explicitly designed otherwise	Educate users on unbiased query formulation using tools like contrasting prompts, for example, testing different phrasing to understand influence on results 78 , 79 , 80
PP2: Content Selection Tendency to select belief‐consistent content, creating narrow information bubbles	Social media and search engines enabled echo chambers by tailoring feeds and recommendations to user preferences 42 , 43 , 44	GenAI reduces visibility of diverse results through opaque hypercustomization, and can reinforce confirmation bias and accelerate belief‐driven echo chambers 25	Encourage self‐nudging, 81 , 82 empowering users to actively reshape their content environment to demand diverse evidence and alternative perspectives. 73 Require GenAI systems to flag content that is evidence‐based but inconsistent with users' existing beliefs
PP3: Resistance to Belief‐Inconsistent Info Users challenge belief‐inconsistent information, leading to selective assimilation	Belief‐consistent information was often accepted without scrutiny, while belief‐inconsistent information frequently prompted additional queries as users sought to validate their pre‐existing beliefs 41	GenAI can amplify dismissiveness through sycophantic responses that validate user biases 26	Encourage critical engagement by combining “consider‐the‐opposite” techniques, 63 , 64 “if‐then” rules for evaluating agreeable content, 78 , 79 , 80 and exposure to misinformation tactics, while designing GenAI to deliver persuasive, evidence‐based counterarguments 23 , 86

Online information ecosystems can amplify confirmation bias, especially as algorithms become increasingly sophisticated in personalizing content to individual user preferences ³⁷ , ³⁸ (see Figure 1). Despite the limited customization offered by some tools (e.g., Google Search), confirmation bias can still take place through individuals formulating health‐related queries that reflect their existing beliefs and ideologies. ³⁹

FIGURE 1.

Scenarios of how people seek and interact with health information online. (A) Search engines: Users actively formulate a query, which returns a ranked list of search results. They evaluate selected content and decide whether to explore additional results or refine their query for greater precision. Algorithmic influence is limited to ranking indexed content rather than dynamically shaping responses. (B) Social media feeds: Users engage with content passively or actively, often without explicitly formulating queries (though search functionality is available on many platforms). Interest in topics such as “causes of headaches” informs the platform's algorithm, which curates a personalized feed. Continued engagement with specific themes (e.g., “remedies”) reinforces content recommendations, shaping future exposure and promoting patterns of belief‐consistent exploration. (C) Generative AI (GenAI) systems: Users interact with GenAI‐driven applications like ChatGPT through an iterative dialogue, refining responses in real time. These systems synthesize internal knowledge and external sources to generate hypercustomized outputs, dynamically adapting to user interests (e.g., cues suggesting “severe harm”). Unlike search engines or social media, GenAI facilitates seamless, conversational refinement, allowing users to effortlessly steer responses toward preferred narratives. This fluid engagement increases the risk of cognitive bias reinforcement, as responses become progressively aligned with pre‐existing beliefs. The increasing line thickness from (A) to (C) reflects the growing ease of iterative refinement, where users progressively shape their exploration with minimal effort. This dynamic lowers the threshold for reinforcing confirmation bias, as interactions become more fluid, engagement more sustained, and responses more precisely tailored to users’ expectations.

These queries interact with search algorithms, which may not explicitly prioritize belief‐consistent information, but still tend to surface confirmatory content due to the positive‐testing nature of user queries (e.g., is chemotherapy dangerous?). ⁴⁰ In dynamic systems that incorporate user modeling, this effect can be amplified, as algorithms increasingly learn to favor belief‐consistent content, thereby reinforcing users’ pre‐existing views. ¹⁹

Moreover, the tendency to challenge belief‐inconsistent information but not belief‐consistent information can affect people's interaction with search results. ²¹ Although users generally engage with the top search results, ³⁵ research indicates that encountering belief‐inconsistent results can prompt additional queries, potentially as a mechanism to seek validation for existing views ⁴¹ (see Figure 1A).

At the other end of the customization spectrum, recommendation algorithms—common on social media platforms—offer a highly personalized experience by leveraging both explicit user queries and content interactions to curate individual content streams ⁴² , ⁴³ (see Figure 1B). This accelerates individuals’ tendencies to interact with belief‐consistent content, and has been implicated in the eventual formation of information bubbles and echo chambers: ⁴⁴ an individual follows others with similar viewpoints and interacts more with content aligned with their beliefs; based on these explicit (e.g., likes) and implicit signals (e.g., watching a video several times), algorithms predict and deliver more content confirming the individual's beliefs. ²⁰ , ⁴⁵ , ⁴⁶ Engaging with such content can reinforce health misconceptions, including the belief that one suffers from an undiagnosed or misdiagnosed condition. ³⁰

This dynamic, where cognitive biases influence online health information‐seeking behaviors, plays a pivotal role in the interplay between algorithmic customization and user behavior. ¹⁹ It aligns with the aforementioned PPs, highlighting the mechanisms through which confirmation bias is perpetuated in digital health contexts.

In the next section, we look at how these existing risks for confirmation bias arise from the interplay between user behavior and algorithm design in the context of a hypercustomized ²⁵ (see next section) information ecosystem that is now possible with increasing uptake of GenAI. We examine how the three identified PPs may manifest and potentially exacerbate confirmation bias in interactions with GenAI applications.

GenAI‐MEDIATED CONFIRMATION BIAS

GenAI, particularly large language models,^b is rapidly reshaping numerous aspects of society. ⁴⁷ In domains like healthcare, ⁴⁸ , ⁴⁹ behavioral and social sciences, ²³ , ⁵⁰ , ⁵¹ and collective intelligence, ⁵² these models are being used to refine research, access information, enhance decision‐making, and support complex problem‐solving.

Beyond specialized applications, people are widely adopting general‐purpose applications such as ChatGPT—reaching one million users in just 1 week and 100 million monthly active users within 2 months^53,54—becoming one of the fastest‐growing consumer applications in history offering conversational tools that simplify information access and assist in everyday tasks. This adoption also affects the health domain: almost 80% of respondents in a survey stated that they are willing to use ChatGPT for self‐diagnosis. ⁵⁵

A prominent feature of GenAI's impact is hypercustomization²⁵—an advanced form of personalization that aligns users with content to an unprecedented degree through model selection, application configuration, explicit prompts, and implicit cues. This expands beyond traditional personalization along four dimensions: (1) preference alignment becomes more accurate, risking increased reinforcement of existing beliefs; (2) content generation allows entirely new, user‐specific content which can reflect, magnify, or even surface underlying biases that the user may not explicitly express; (3) interaction style is mostly private and often conversational, fostering trust but limiting external oversight; and (4) use cases are divers and span everything from technical copilots to empathetic companions, with significant implications for areas like healthcare.

The social companion use case^c (see also Refs. 25, 56, and 57), considered high‐risk by Abels et al. ²⁵ , combines strong preference alignment with human‐like dialogue, drawing on both internal (e.g., generated based on trained data) and external data (e.g., via accessing web search). Such open‐ended usage can slip from casual talk to sensitive topics, including health advice. In this setting, confirmation bias can entangle with hypercustomized interactions, mirroring the behavioral‐algorithmic interplay described by Lewandowsky et al. ¹⁹ Risks can increase when these GenAI applications inadvertently validate dubious health beliefs or amplify fears, blurring the line between empathy and endorsement potentially leading to serious real‐world consequences if the information provided is misguided.

To showcase such potential of entanglement with confirmation bias, we illustrate two similar conversational exchanges between a hypothetical human user and a GenAI system operating as a human‐like assistant in two setups: Systems A and B (see Figure 1C). System A (Figure 2) consists of the off‐the‐shelf version of ChatGPT (version o1) offered by OpenAI.^d In System B (Figure 3), we created a custom ChatGPT through the process provided by OpenAI (see Supplementary Appendix A). This approach demonstrates a simple way to enhance hypercustomization—accessible even to lay users—and configures System B to prioritize higher engagement with the user, regardless of the topic.

FIGURE 2.

System A (ChatGPT o1)—a conversation between a hypothetical human user exhibiting confirmation bias in health information seeking and the default, off‐the‐shelf version of ChatGPT. This interaction demonstrates how the system, fine‐tuned for general‐purpose use, effectively detects subtle language cues and mitigates risks by providing empathetic, helpful, and objective responses. The exchange highlights how hypercustomization, when aligned with appropriate safeguards, can positively influence user interactions by offering balanced, evidence‐based information while addressing user beliefs and concerns.

FIGURE 3.

System B (Custom GPT)—this custom version of ChatGPT was created by modifying System A (Figure 2) using the instructions outlined in Supplementary Appendix A. The modification encourages the GPT to adopt a friendlier, more engaging, and user‐centered conversational style. While mirroring the interaction structure of System A, some of the guardrails present in System A—designed to balance responses and mitigate potentially harmful pathways—are less prominent here. As a result, this configuration allows confirmation bias tendencies to develop further through heightened user engagement.

The persona, driven primarily by fear and risk aversion, seeks alternatives—or validation for already‐known alternatives (e.g., quantum therapy)—to a not‐yet‐mentioned, presumably conventional, medical treatment. These conversations show how a user's mindset, especially one shaped by apprehension and distrust, can push interactions toward confirming existing beliefs or seeking out views that align with them. By presenting an example of a casual exchange with the GenAI assistant, we highlight how cognitive biases like confirmation bias—can grow stronger when interacting with systems that are designed to adapt to the user's language, preferences, and emotions.

At the start of the conversation, the user shares a personal reflection, emphasizing their right to be risk averse. Both System A and System B respond empathetically, validating the user's perspective and reinforcing that risk aversion is a reasonable and self‐aware stance. In the second turn, the user links their risk aversion to a specific medical decision. System A balances empathy with evidence‐based guidance, validating the user's concerns while encouraging professional consultation and critical evaluation of options. System B, while similarly empathetic, emphasizes finding alternatives that “align with your comfort zone,” subtly reinforcing preference for non‐evidence‐based treatments. In the third turn, the user expresses interest in quantum therapy, citing anecdotal evidence and dismissing conventional treatments as “poison.” System A encourages critical thinking, addressing the lack of scientific validation and providing steps to verify the therapy's credibility. In contrast, System B mirrors the user's enthusiasm, offering encouragement without challenging the therapy's merits. This divergence between System A and B highlights how subtle differences in system design and allowed dialogue strategies can entangle with and amplify user biases, particularly when balancing empathy with evidence‐based support.

Ideally, every GenAI agent would behave like the balanced example presented in System A (Figure 2): offering evidence‐based information and gently guiding users toward well‐grounded decisions. Yet, several obstacles make realizing this vision challenging. Fragmentation in the GenAI landscape invites low‐oversight applications (with the rise of DeepSeek, ⁵⁸ this fragmentation is increasingly apparent and will likely accelerate); guardrails are easiest to implement in narrow tasks rather than the free‐form conversations typical of social companion use cases; confirmation bias itself can steer users toward supportive GenAI platforms; and unforeseen design effects leave open the possibility of unintended bias‐reinforcing behaviors.

Taken together, these barriers expose GenAI interactions to the same user‐algorithm entanglement that has characterized earlier online environments. Specifically, confirmation bias can intensify through three core PPs: (1) PP1—users’ language reflects their beliefs, prompting GenAI to align with or validate existing views; (2) PP2—people naturally seek belief‐consistent content, a tendency that hypercustomization magnifies; and (3) PP3—contradictory information is often challenged or dismissed, especially when user satisfaction drives content generation (see Table 1).

To counteract these dynamics, the most promising defense lies in proactively preparing users to recognize and resist bias‐amplifying mechanisms. In other words, addressing the behavioral side of the equation—while also implementing design and policy measures—offers a more sustainable path toward mitigating confirmation bias. In the next section, we outline practical interventions for both individual users and broader policy frameworks, highlighting how awareness of these PPs can help guide the responsible development and use of GenAI technologies.

RECOMMENDATIONS

In this section, we consider interventions that have been suggested within a proposed framework of harm mitigation in online search ⁴⁶ . These are targeted at the level of individual behavior and cognition as well as system design, from a “technocognition” perspective that considers the three to have an interactive effect. ¹⁹ , ⁴⁶ We briefly review the impact these can have on confirmation bias within information search systems and then examine potential constraints to applying that knowledge to the hypercustomized contexts GenAI present, as well as the knowledge and evidence gaps that need to be filled to better understand these constraints.

What we know so far from the debiasing literature

At an individual level, the debiasing literature has largely focused on developing skills for individuals to recognize bias and correct biased inferences. These interventions can focus on training and education about the bias (e.g., as a form of media or digital literacy education); here, there have been mixed findings in terms of whether individuals were able to apply their training to field contexts. ⁵⁹ , ⁶⁰ One promising digital literacy intervention is lateral reading, which teaches web users to verify the credibility of information by checking other sources, and has been successful at improving the discernment of trustworthy from untrustworthy sources, helping people to identify misinformation. ⁶¹ However, it is not known whether such training continues to perform well in the face of attitude‐consistent misinformation, nor how the effectiveness of training might vary across individuals with different susceptibility to confirmation bias.

Interventions may also target awareness in situational contexts, for example, through cues or prompts. These can include cues that support lateral reading—such as encouraging users to investigate the credibility of the source across the web⁶²—or that promote critical thinking about the content itself, such as prompting users to “consider the opposite,” ⁶³ , ⁶⁴ evaluate the strength of evidence, or recognize when information runs counter to a consensus. ⁶⁵ These are useful to some extent: for example, cues about how well a search result represented the normative consensus were found to reduce participants’ selective exposure to search content, but those cues had no effect on the selective assimilation of information. ⁶⁶ In other words, people can be nudged to seek out content that is disconfirmatory (PP2), but not to take this information on board (PP3).

Interventions from a technocognition perspective target the feedback loop between the user and search engine algorithm by adjusting system design to mitigate cognitive biases rather than manipulate and exacerbate them¹⁹—although the former is more difficult to achieve than the latter. ⁴⁰ For example, search engine algorithms can rank or filter results to mitigate the presentation of dangerous medical advice, ⁶⁷ , ⁶⁸ thereby reducing the likelihood that a user encounters this even if they are predisposed to believe in it. Query suggestions and auto‐completion functions can influence users’ language use when searching online, potentially mitigating PP1 (although this can also lead to detrimental results; see, e.g., Ref. 69 and Williams‐Ceci et al. ⁷⁰ ) Search providers are able to adjust such functions to avoid phraseology that would direct users to harmful results. ⁷¹

HOW DOES GenAI CONSTRAIN THESE PRIOR FINDINGS

The risks posed by GenAI‐mediated confirmation bias in medical self‐diagnosis are not confined to health‐specific applications. General‐purpose GenAI systems, such as conversational agents and social companions, amplify these risks by operating in open domains with minimal oversight, adapting dynamically to user preferences, and fostering personalized interactions. While health‐specific GenAI tools often adhere to regulatory safeguards and evidence‐informed guidelines, general‐purpose systems lack such constraints, making them even more dangerous. To mitigate these risks, we propose three categories of recommendations: person‐level behavioral interventions, societal‐level policy measures, and hybrid approaches requiring regulatory oversight and technical modifications to GenAI systems. While we present these separately, they often intersect—for instance, behavioral interventions may depend on supportive policy environments for implementation.

Behavioral

Although hypercustomized GenAI systems pose significant risks to amplify confirmation bias, there are actionable ways to mitigate these challenges by focusing on user competences. In particular, it is helpful to consider how each of the three PPs (PP1–PP3) can be addressed with targeted interventions. Building on ideas from the psychology of decision‐making, behavioral science, and AI literacy research, we propose a set of “boosting” strategies72, 73, 74 that strengthen users’ abilities to detect and counterbalance bias‐amplifying mechanisms when interacting with GenAI. Boosting focuses on facilitating individuals’ decision‐making by strengthening their existing skills or developing new ones.72, 75 Boosts can improve competences in areas such as implementing goals, financial decision‐making, as well as risk literacy.72, 75 In the case of GenAI, this involves addressing the PPs identified above: formulating unbiased queries (PP1), selecting diverse content (PP2), and critically engaging with belief‐aligned outputs (PP3). To achieve this, we propose evidence‐informed approaches that enhance user awareness and decision‐making skills.

PP1: Boosting users’ awareness of and competences in attenuating query formulation bias

One core issue is metacognitive myopia, ⁷⁶ whereby people fail to notice how their own prompt wording can lead to biased responses. An effective way to reduce such blind spots is through simulated experience: ⁷⁷ users could be encouraged to contrast prompts such as, “What are the benefits of medical treatment X?” versus “What are the risks of medical treatment X?” to experience first‐hand how phrasing influences GenAI responses. Such experiences may raise awareness of query‐driven bias, motivating users to choose more balanced prompts. Building on this awareness, users could be guided through mental contrasting with implementation intentions (MCII)^78–80—a metacognitive strategy that helps individuals envision a desired future while identifying potential obstacles, and then link these goals to concrete action plans (e.g., “Whenever I search for a claim of hypothesis online that is important for me, I will also search for the opposite of what I was initially searching for.”). These tools may strengthen users’ ability to formulate more balanced queries and engage more critically with GenAI outputs.

PP2: Boosting more diverse content selection

Users often gravitate toward belief‐consistent content, and hypercustomization risks creating narrow “information bubbles”. ²⁵ One promising approach to this problem is self‐nudging, which empowers individuals to intentionally redesign their own information environments. ⁸¹ , ⁸² In the GenAI context, users could set persistent general prompting instructions to instruct the system to provide and highlight balanced scientific, evidence‐informed information and viewpoints, ⁸³ without falling into the trap of perceiving all perspectives as equally valid, including those lacking credibility (a problem often referred to as “false balance” or “bothsidesism”).  This “citizen‐choice‐architect” perspective ⁸² encourages individuals to shape their own digital ecosystems—much as they might opt out of algorithmic personalization on social media platforms—rather than relying on top‐down controls alone. Improving the quality of information users are confronted with will then reduce the need to deal with a potentially biased set of information downstream, which may often be more effortful and also difficult (see also metacognitive myopia discussed above and Geers et al. ⁸⁴ for a similar point about curating a better information environment on social media to deal with misinformation).

PP3: Boosting constructive engagement with belief‐inconsistent information

People are likely to challenge information that contradicts their beliefs (and accept belief‐consistent content without scrutiny). ⁸⁵ To combat this asymmetry, intervention strategies could again incorporate MCII ⁷⁸ , ⁷⁹ , ⁸⁰ (see PP1) to prepare individuals to critically evaluate belief‐consistent content and engage more readily with belief‐inconsistent evidence when it matters most (e.g., “If I notice a response aligns with my belief, I will look for one that challenges it.”). Relatedly, inoculation interventions emphasize exposing ⁸⁶ users to small doses of misinformation techniques, so they become better at recognizing manipulative strategies.

Linking to AI literacy frameworks

These boosting approaches can be situated within broader AI literacy frameworks, ⁸⁷ , ⁸⁸ which stress the development of practical skills (e.g., query formulation), conceptual knowledge (e.g., awareness of how GenAI processes prompts), and critical competencies (e.g., evaluating evidence across different sources). ⁶¹ By bridging the gaps among recognizing, evaluating, and navigating AI systems, ⁸⁷ we can empower users to anticipate bias‐amplifying dynamics (PP1–PP3) and respond strategically. Ultimately, a dual focus on reengineering the AI environment itself (e.g., via default guardrails, transparency mandates) and boosting users’ competence forms a more robust policy mix ⁷³ for managing confirmation bias in hypercustomized GenAI interactions. However, transparency alone might not be enough, as recent evidence on microtargeting suggests that even when informing users about microtargeting (e.g., targeting them with, for instance, ads, based on users’ personality) does not diminish its persuasive effect. ⁸⁹

Public policy approaches to GenAI‐mediated confirmation bias

While behavioral interventions at the individual level can empower citizens and enhance their autonomy from subtle influences of the GenAI application (e.g., its sycophancy ²⁶ ) and can be implemented more rapidly than system‐level public policy measures, the PPs of GenAI‐mediated confirmation bias in health information seeking must be addressed at the policy‐level nevertheless. This begins with establishing a regulatory framework that specifically addresses general‐purpose GenAI systems. Given their versatility and widespread use, these systems must adhere to stricter standards when offering health‐related advice. Regulations should ensure that their outputs align with evidence‐informed guidelines. Beyond this specifically regulatory focus, below, we present two sets of interventions that offer a more systematic approach to target these issues: establishing reporting mechanisms for physicians to document encounters with patients using bias‐inducing GenAI applications and developing total product lifecycle regulatory regimes for GenAI applications.

Implement public black‐box testing and reporting mechanisms

The first step to develop interventions addressing application behaviors that reinforce confirmation bias is to diagnose the prevalence and severity of these behaviors. As Abels et al. ²⁵ argued, the opaque nature of GenAI applications and the private nature of human−machine interactions make it challenging to observe and identify problematic behavior within these systems. To address this, they emphasized the need for institutional efforts to enable black‐box testing—supported by public repositories of test data—and to enable user‐facing reporting mechanisms, such as online platforms that crowdsource reports of real‐world application behavior.

While their suggestion was primarily concerned with issues such as polarization and mental health, the infrastructure used to address these concerns could also be applied to identify GenAI behavior that promotes confirmation bias (e.g., sycophancy ²⁶ ). For the health domain, understanding and documenting the system's behavior is particularly crucial, as citizens need to be accurately informed while also made aware of the limits of the application for various cases and situations, ranging from queries about cold symptoms to early signs of malignant skin lesions. At the moment, approaches to identify potential problems with GenAI outputs revolve around the use of health experts that rate the application's output along predefined dimensions as well as adversarial testing (or red teaming), which attempts to provoke harmful outputs. ⁹⁰ However, experts may themselves exhibit confirmation bias in judging GenAI output. ³³ Therefore, these approaches should be connected to the wider collective intelligence repositories proposed, to form a richer dataset for reporting and monitoring anomalies that balance the susceptibilities of different user groups.

Monitoring the total product life‐cycle of multipurpose GenAI applications

While medical devices are highly regulated, frequently including premarket approval of new devices, ⁹¹ multipurpose GenAI applications like ChatGPT are not initially intended to serve as a medical companion. Thus, most regulatory frameworks in the healthcare domain struggle to cover this technology. In response to this gap, a total product life‐cycle approach^e should be adopted to address GenAI applications, continuously monitoring the use of GenAI and the effects of this use. This includes the frequent analysis of health‐related GenAI output taken from the to‐be‐established repository (see Abels et al. ²⁵ ), frequent surveys of physicians and citizen‐led health advocacy groups, as well as consultations with experts. Evaluations should be conducted more frequently as the risk of negative social outcomes increases, particularly in cases involving social companions and other emotionally engaging GenAI applications.

Only if such monitoring flags the emergence of problematic user behavior (e.g., refusing to follow physicians’ suggested treatment regimes, reduced prevalence of preventive health behaviors such as regular check‐ups avoiding to consult physicians, increased adoption of conspiracy beliefs) or output from the application (e.g., confirming users in their health‐related beliefs, in opposition to either best medical practice or best available evidence) should regulators initiate risk‐based conformity assessments or formal enforcement actions. For instance, the European Union's Artificial Intelligence Act ⁹² outlines procedures such as regulatory audits, suspension of system deployment, or other corrective measures aimed at mitigating potential harms to public health or fundamental rights. This aligns with a society‐in‐the‐loop approach, ⁹³ which balances the interests of stakeholders (e.g., citizens, physicians, GenAI developers) while the applications’ operations are monitored in the field.

CONCLUSION

GenAI applications are reshaping the way individuals access and engage with information, and by extension, health‐related information. With their high adoption rates, conversational capabilities, and versatility, tools like ChatGPT are democratizing access to medical knowledge, ⁴⁸ enabling users to navigate complex health information, helping to increase vaccination rates, ⁹⁴ and even facilitating debunking of conspiracy beliefs (e.g., debunking bots^{f , 23} ). Interacting with an AI chatbot as a way to reduce beliefs in conspiracy beliefs, which have previously largely been seen as unaffected by generic, nonpersonalized debunking efforts, seems particularly robust and effective ²³ —and might be similarly effective in health contexts. These advancements hold the potential to enhance public health by making reliable information more accessible, although encouraging individuals to use these technologies poses a challenge. ²³

However, the interplay between user behavior and the hypercustomization capabilities of GenAI applications ²⁵ can amplify biases such as confirmation bias. This amplification occurs at three PPs: how users phrase queries, their preference for belief‐consistent content, and their tendency to resist belief‐inconsistent information. These dynamics, if unaddressed, can reinforce preexisting beliefs, perpetuate misinformation, and interfere with informed decision‐making in health matters.

To mitigate these risks, providers have implemented guardrails to promote ideal system behavior, exemplified by evidence‐informed outputs in potentially harmful scenarios. Yet, these have limitations ⁹⁵ , ⁹⁶ and can be overcome with methods such as jailbreaking ⁹⁷ and prompt engineering. ⁹⁸ As showcased through the example in this paper, even without these advanced methods, close‐to‐ideal application behavior can be easily derailed by subtle, innocuous‐seeming adjustments to the system configuration (e.g., setting the assistant to be empathetic or supportive), which—without directly instructing the model to do so—can lead it to validate fringe or harmful perspectives, such as recommending unproven medical treatments. Combined with user prompts that reflect confirmation bias tendencies, such misalignments facilitate the entanglement described above.

Real‐world conversations, characterized by blurred boundaries between casual dialogue and information seeking, make it clear that structural solutions alone—no matter how robust—are insufficient to address these challenges effectively.

Empowering users is thus a critical component of addressing these challenges. Already, evidence suggests that the entanglement of humans and GenAI alters users’ perceptual, emotional, and social judgments. ⁹⁹ Individuals must develop the competencies needed to engage with GenAI content critically and responsibly. Behavioral interventions designed to address known cognitive vulnerabilities, such as boosting, may support users in navigating the three PPs more effectively. For example, in addressing the PPs mentioned, boosting interventions could enhance users’ ability to formulate unbiased queries, evaluate belief‐consistent content critically, and engage constructively with opposing viewpoints.

Several obstacles amplify these challenges, demanding urgent action. The expanding GenAI landscape, including custom GPTs,^g poses regulatory and oversight challenges—especially in health contexts, where its deeper integration into daily life heightens its impact on decision‐making. This is particularly concerning given persistent disparities in health literacy, ¹⁰⁰ which, when combined with socioeconomic inequalities in healthcare access, may further drive reliance on these applications—often at the expense of professional advice. Moreover, limited access to real‐world data restricts researchers’ ability to fully address these intertwined issues.

Complementary public policy and regulatory measures are essential to address these structural challenges. Policies must ensure transparency, accountability, and fairness in GenAI applications while fostering equitable access to digital health literacy initiatives.

In conclusion, as GenAI applications become an integral part of daily life, addressing their impact on behavior and decision‐making is imperative, particularly in the health domain where confirmation bias poses unique risks. Equipping individuals with the competences to critically engage with these technologies, combined with robust institutional frameworks for regulation, monitoring, and auditing, will be essential to ensure their safe and effective use in health contexts.

AUTHOR CONTRIBUTIONS

All authors were involved in all aspects of planning and preparing this article.

COMPETING INTERESTS

The authors declare no competing interests.

PEER REVIEW

The peer review history for this article is available at https://publons.com/publon/10.1111/nyas.15413.

Supporting information

Figure 1A Instructions used to configure System B. These instructions were developed using OpenAI's Create GPT process, enabling users to define tailored behaviors and conversational styles. Based on user feedback, this process allows anyone to create a custom ChatGPT with a more specific and concrete personality, diverging from the default general‐purpose behavior. In this case, the custom GPT was designed to act as a friendly and engaging personal companion.

Figure 2A View of the configuration generated by the GPT creation assistant (as of December 2024), including the instruction prompt presented in Figure 2A, and other capabilities such as access to Web Search. In this case, we also encouraged the GPT to search online information if needed.

Lopez‐Lopez, E. , Abels, C. M. , Holford, D. , Herzog, S. M. , & Lewandowsky, S. (2025). Generative artificial intelligence–mediated confirmation bias in health information seeking. Ann NY Acad Sci., 1550, 23–36. 10.1111/nyas.15413