Identifying Credible Sources of Health Information in Social Media: Phase 2—Considerations for Non-Accredited Nonprofit Organizations, For-Profit Entities, and Individual Sources

May 23, 2023

Introduction

A recent review suggests that 5.03 billion individuals use the Internet worldwide, with approximately 4.70 billion, or 59% of the global population, using some form of social media (Kemp, 2022). While it is challenging to determine overlap across platforms, it has been reported that worldwide exposure to information on either Facebook or YouTube¹ is in excess of 2 billion people each (Meta, 2022; YouTube, 2022). The Pew Research Center reported that approximately 72% of adults in the United States use at least one social media outlet (Pew Research Center, 2021). It is estimated that some 90% of Americans use social media sources for health information, including searches related to serious conditions, general information searches, and searches for minor health problems (Bishop, 2019). Research demonstrates that online information can influence health beliefs, health behaviors, and decisions about seeking health care (Chen et al., 2018; Tan and Goonawardene, 2017).

In the context of so much of the public using the Internet and social media to inform health decisions, stakeholders—including medical and public health professionals—have raised serious concerns about the quality and reliability of health information on social media due to the lack of standards or regulations around what is posted and how content is monitored (Wang et al., 2019; Broniatowski et al., 2018). These concerns were heightened during the COVID-19 pandemic, and recent reports have found an increased volume of health mis- and disinformation online during the pandemic (Borges do Nascimento et al., 2022; Khullar, 2022; OSG, 2021). While mis- and disinformation about health care topics are neither new nor unique to social media, the viral nature of some posts, the presence of platform algorithms that elevate popular content, and limited resources to vet every item posted to social media amplifies the volume of misinformation that readers are exposed to on all social media channels.

In response to these concerns and the proliferation of mis- and disinformation online, Google/YouTube² has supported efforts, conducted in two phases by independent groups, to develop principles and attributes to guide social media and other digital platforms to rigorously identify credible sources of health information. Through such identification, consumers could be directed to credible sources first when they search for health information online. However, the authors of this paper recognize that identification of credible sources may not be sufficient to ensure that consumers are accessing high-quality information, and social media companies may need to employ parallel strategies such as content assessment, management of misinformation, addressing health literacy and culturally competent communication, and developing avenues for sources to self-regulate in order to truly address this complex issue.

The first phase of this work (Phase 1) was completed in 2021 by an expert advisory group convened by the National Academy of Medicine (NAM), which yielded foundational principles and attributes for determining credibility of health information sources³. The scope of Phase 1 was limited to U.S.-based entities and concentrated on identifying credibility among nonprofit and government entities with established vetting or accrediting procedures. In Phase 1, described below, the expert advisory group proposed three foundational principles to support assessment of source credibility and developed attributes for assessing a source's alignment with the principles (Kington et al., 2021).

Phase 1: Foundational Principles, Attributes, and Additional Findings

In Phase 1, an expert advisory group proposed three foundational principles to support the assessment of credibility of online sources of health information: 1) science-based; 2) objective; and 3) transparent and accountable. The members of the expert advisory group also provided a selection of material attributes that can be used by social media companies and others, including consumers, to assess a source's alignment with the three principles (Kington et al., 2021). See Table 1 for an overview of these principles and attributes.

TABLE 1. Phase 1 Foundational Principles and Attributes for Identification of Credible Sources of Health Information in Social Media.

Principle	Attributes (Phase 1)
Science-based: Sources should provide information that is consistent with the best scientific evidence available at the time and should meet standards for the creation, review, and presentation of scientific content.	Acknowledges the limitations and evolution of knowledge Clearly labels information with the date it was last updated and strives to reassess and update content Demonstrates subject-specific expertise Links to and is linked to by other credible sources [a] Provides citations for information shared and evidence to justify claims Synthesizes information from multiple sources, rather than a single source Uses a consensus process to develop the information shared [b] Uses peer review or another form of content review to vet information before sharing [c]
Objective: Sources should take steps to reduce the influence of financial and other forms of conflict of interest or bias that might compromise or be perceived to compromise the quality of the information they provide.	Keeps health information separate from financial, political, or ideological messages Maintains independence from funders [d] Separates lobbying activities from health information (or does not engage in lobbying) Does not include advertisements with relevant health information (or does not host advertisements at all) [e]
Transparent and accountable: Sources should disclose the limitations of the information they provide, as well as conflicts of interest, content errors, or procedural missteps.	Discloses financial and nonfinancial conflicts Discloses relevant policy positions and lobbying activities Follows FACA regulations or similar transparency policies [f] Posts public corrections or retractions Prioritizes accessibility and equitable access to information Provides a mechanism for public feedback Shares data, methods, or draft recommendations

SOURCE: Kington, R., S. Arnesen, W-Y. S. Chou, S. Curry, D. Lazer, and A. Villarruel. 2021. Identifying Credible Sources of Health Information in Social Media: Principles and Attributes. NAM Perspectives. Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/202107a.

NOTES:

^[a]For example, an organization could seek public comments on an interim set of health guidelines before finalizing and sharing the information more broadly.

^[b]A consensus process involves assembling a group of experts with diverse perspectives who assess a body of evidence and deliberate in order to arrive at an opinion or guidance that reflects the consensus of the group.

^[c]A peer review process involves sharing the draft of a publication or other product with reviewers who have expertise or experience in the given topic and can provide feedback as to the product's accuracy, balance, and appropriateness.

^[d]For example, an academic journal could maintain editorial independence (i.e., sole authority over published content) from the organization that funds it.

^[e]For example, an organization might host an advertisement for a cancer drug but keep this advertisement separate from the information it shares about cancer.

^[f]FACA stands for the Federal Advisory Committee Act, which established requirements for committees that advise the federal government. These requirements include public access to meetings and meeting notes, as well as summaries of expenditures (https://www.gsa.gov/policy-regulations/policy/federal-advisory-committee-management/advice-and-guidance/the-federal-advisory-committee-act-faca-brochure).

As noted by Kington and colleagues in a flowchart for credibility of sources of health information (see Figure 1) in Phase 1, organizational sources could be afforded a preliminary assumption of credibility if they were subject to pre-existing, standardized vetting mechanisms, including government accountability, accreditation, and academic journal indexing. Entities considered by the Phase 1 process included nonprofit and government sources like government organizations, academic journals, accredited health care organizations, educational institutions, and public health departments in the U.S.

FIGURE 1. Phase 1 Assessment Flowchart for Credibility of Sources of Health Information in Social Media.

SOURCE: Kington, R., S. Arnesen, W-Y. S. Chou, S. Curry, D. Lazer, and A. Villarruel. 2021. Identifying Credible Sources of Health Information in Social Media: Principles and Attributes. NAM Perspectives. Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/202107a.

NOTES: [a] This chart is developed for credibility assessment of nonprofit and government organizations only. For-profit companies and individuals that serve as sources of health information should also undergo separate credibility assessment processes.

[b] Pre-existing, standardized vetting mechanisms that align with the authors' principles and attributes include accreditation, academic journal indexing, and government accountability rules. Even sources subject to one of these mechanisms should strive to meet the authors' stated credibility principles and attributes.

[c] See Table 1 for a list of principles and credibility attributes.

[d] Ideally, a quality assurance system that includes content assessment should supplement assessment of source credibility.

In addition to identifying factors to assist in determining credibility, the Phase 1 expert advisory group also identified three key areas to be considered in the ongoing mission to elevate credible content.

The Phase 1 expert advisory group limited their deliberations to principles and attributes that could be applied to U.S.-based organizations. The World Health Organization (WHO), after the release of the Phase 1 paper, convened an expert panel to assess this initial guidance from a global perspective, and recommended that further work be done to extend the principles to make them more generalizable to international audiences. Globalization then became one of the charges to the authors of this paper (Phase 2).

The Phase 1 expert advisory group also identified health equity, diversity, and inclusion as critical components to be included in any system used to elevate credible sources.

Finally, while outside of the scope of work for both Phase 1 and 2, the Phase 1 expert advisory group highlighted the importance of content review, implementation, and research on the impact of credibility designations as important future directions for this work.

Phase 2: Advisory Committee Charge and Scope

Phase 2 was carried out by a multidisciplinary, independent advisory committee convened by the Council of Medical Specialty Societies (CMSS), in collaboration with NAM and WHO. The committee was charged with adapting the principles and attributes established in Phase 1 to allow for the evaluation of the credibility of other health information sources, including other nonprofit entities, for-profit entities, and individuals, with an eye towards global applicability (see Box 1).

Box 1. Phase 2 Project Objective.

Construct a globally relevant, expanded set of principles, attributes, and definitions applicable to a wider group of potential sources of credible information

Building upon the seminal work completed in Phase 1, the Phase 2 advisory committee was charged with considering three additional potential sources for health information found on various social media platforms:

• 1.Nonprofit organizations without pre-existing standardized vetting mechanisms, including foundations, patient disease organizations, community health organizations, and think tanks;
• 2.For-profit entities, including drug or device manufacturers; and
• 3.Individuals, including scientists and clinicians, other professionals, and patients.

This wide-ranging group of sources presented a variety of potential issues around feasibility of credibility assessment and varying levels of transparency into how health information was collected and content was generated.

Methods

For consistency and transparency, the methods and processes employed during Phase 2 were substantively similar to those employed during Phase 1 (Kington et al., 2021). Additionally, the Phase 2 advisory committee adopted the definitions of ‘credible' and ‘high-quality information' established during Phase 1 (see Box 2) and began their deliberations with the foundational principles and attributes proposed in Phase 1 and reaffirmed by the WHO (WHO, 2022; Kington et al., 2021).

Box 2. Key Terms.

The following are definitions and discussions of the key terms established in Phase 1 and used in the present paper (refer to Kington et al., 2021, for full discussion).

Credible

For the purposes of this paper, the authors present their own definition of credible in the context of sources of online health information: “offering information that is consistent with the best scientific evidence available at the time and employing processes to reduce conflict of interest and promote transparency and accountability.”

High-Quality Information

High-quality information is that which is “science-based,” or consistent with the best scientific evidence available at the time. Science and knowledge are always evolving, so the marker of time is an important component of this definition. The evolution of knowledge is also the reason that more absolute terms, such as accurate, are less appropriate. Although this paper does not consider information quality directly, increasing access to high-quality information is the goal of the approach under discussion.

Composition and Selection of the Advisory Committee

The advisory committee is composed of independent volunteers who were nominated by CMSS, WHO, and NAM based on their subject matter expertise. Individuals were not eligible to participate on the committee if they were currently employed by social media companies. The committee included authors of the Phase 1 paper and new members from multiple disciplines including information governance, health information development, public health and health equity, social media and misinformation, and science communication.

Managing Conflict of Interest

Similar to Phase 1, in order to minimize conflicts of interest, CMSS, WHO, and NAM took steps to ensure the independence and objectivity of the advisory committee and this paper, in that authors were required to disclose financial and non-financial conflicts of interest (Kington et al., 2021). This paper represents the opinions of the authors and does not reflect a consensus position of CMSS; NAM; the National Academies of Sciences, Engineering, and Medicine; WHO; or the authors' organizations. The advisory committee did not receive payment for their contributions to this paper.

Deliberative Sessions

The authors of this paper participated in one recorded orientation session, which was asynchronously viewed by the group, and three virtual, interactive, closed deliberative sessions between July and October 2022. Representatives from Google/YouTube attended the first live virtual session to explain the company's current policies, initial experiences with implementing Phase 1, and future goals regarding elevating high-quality health information from different sources, as well as answer questions from the authors. Representatives from Google/YouTube did not attend any part of the subsequent deliberative sessions, and were not involved in committee discussions, in drafting the principles and attributes, or in drafting or reviewing this paper.

Approach and Timeline

The Phase 2 advisory committee examined and deliberated on the three principles established during Phase 1 and their defining attributes in individual breakout groups that each focused on one of three potential sources of credible health information: for-profit organizations, non-accredited nonprofit organizations, and individuals. Each breakout group outlined attributes of each principle that were relevant to the assigned Phase 2 source. Each breakout group aimed to identify the key attributes for that source, focusing on those that were important to credibility, were identifiable, and were practical to implement.

Next, the entire committee virtually discussed and prioritized the suggested attributes from each breakout group, stating whether they agreed or disagreed with each revision and rating the attribute as Very Important, Important, or Not Very Important for each source. Items ranked as Not Very Important for a source were deleted, and items ranked as Important or Very Important would potentially be incorporated. At the next meeting, the breakout groups reconvened separately to discuss and critically assess the attributes suggested and prioritized by the entire committee for each source of information.

Public Comment Period

A draft of the proposed new attributes, a related questionnaire, and a preliminary draft of this paper were posted for public comment by CMSS from 12:00 PM CST on September 9, 2022, to 11:59 PM CST on September 19, 2022. Comments were specifically solicited from interested parties, including researchers, medical specialty society leadership, clinicians, creators of online health information, for-profit and non-profit organizations, health care providers, and members of the public. The committee reviewed all comments received. The comments were analyzed, sorted into themes, and summarized by program staff and the authors of this paper (see Box 3 for key themes from the public comment period and the committee's response).

Box 3. Key Themes Among Feedback Received During the Public Comment Period and Committee Response.

• The feedback generally supported requiring all sources to meet all principles and to meet a preponderance of prioritized attributes.

  • Committee: The committee agreed with commenters that a preponderance of attributes was appropriate. However, the committee determined that compliance with all attributes would be unrealistic, especially for the global community. The committee suggests that discussions about this approach continue beyond the publication of this paper.

• There was strong support for the importance of science-based and objective attributes, especially regarding use of citations that have undergone peer review.

  • Committee: For individual sources, consider an additional attribute regarding authorship of peer reviewed articles, preferably in high impact journals. Concerns were raised that this attribute could exclude some otherwise credible sources, especially in non-Western countries. The committee ultimately did not include this attribute.

  • Committee: Consider an approach that divides attributes into those that can be verified (“computable”) versus those that will likely require self-report via attestation.

• Commenters indicated strong support for disclosure of any sponsored and paid partnerships. Some commenters noted that this area needed more consistency across all sources. Commenters also noted the variability across social media platforms, including the inability to provide standardized bios or link to sources on some platforms (e.g., Twitter, TikTok).

  • Committee: Consider disclosure of “financial relationships” regardless of whether those financial relationships present a conflict of interest or not.

• There were many comments related to advertising revenue received by creators from social media platforms. Commenters referred to existing guidelines and guardrails in place to protect consumers (e.g., FTC regulations). Commenters recommended referencing existing guidelines and regulations.

  • Committee: While the committee believes that advertising revenue is not necessarily something that requires disclosure (unlike paid partnerships), focus on regulations alone (e.g., FTC regs) would only be applicable in the U.S. (not globally).

• Some commenters raised concerns around highlighting “does not engage in lobbying” for non-profit organizations since some may lobby on issues unrelated to the health information.

  • Committee: The current attribute allows for lobbying if it is “separate from health information,” but the wording of this attribute was updated and clarified.

• There was strong support for a testing or trial period on the accuracy of source attestation to the principles and attributes.

  • Committee: Strong interest in some form of testing prior to wide scale implementation, including assessment of use cases or formal piloting.

  • Committee: Need to ensure public buy-in to use of credible sources once they are established.

• Commenters also noted concerns about sources that are deemed credible that post misinformation. Potential solutions included on and off ramps for credibility assessment or regular attestation by social media platforms.

  • Committee: Strong interest in some form of regular assessment with on and off ramps for credibility of sources.

  • Committee: Acknowledge that this approach will provide a somewhat limited assessment of source credibility, although better than current state and unlikely to cause harm.

• Most commenters considered the principles and attributes appropriate from a global perspective and noted several non-U.S.-centric platforms for consideration (e.g., WeChat, WhatsApp, LINK Social).

  • Committee: Given variable “real estate” across platforms, there was strong interest in a standardized bio with licensure, expertise, conflicts, and other factors that could be used across social media platforms.

  • Committee: Assessment of non-U.S. platforms will likely require more authoritative studies.

• The feedback generally supported the new principle of inclusiveness and equitability. Some comments noted the need for further enhancement of the new principle and attributes, providing clarity around “diversity of voices” across a channel, and more information on how this principle might be operationalized.

  • Committee: Important aspirational call for improvement that may provide a future path to increase diverse voices.

  • Committee: Appreciated focus on targeted audiences.

  • Committee: Provides a cross-cutting theme for other criteria.

The final virtual meeting was then convened so the entire committee could discuss and incorporate feedback from the public comment process.

Phase 2 Principles

The Phase 2 advisory committee accepted the foundational principles developed during Phase 1 and discussed the relevance and applicability of the original attributes to the sources of information prioritized for examination during Phase 2. The Phase 2 advisory committee unilaterally agreed that all information, independent of source, should be held to the principles of being science-based, objective, and transparent and accountable. Although there is overlap between the principles of science-based and objective, the Phase 2 advisory committee distinguished between them by noting that objective focuses more on avoiding potential conflict of interest. Discussion of incorporating values of diversity, equity, and inclusion (DEI) arose throughout all principles and across all information sources. As a result, a new principle targeting inclusiveness was drafted and included in the Phase 2 final principles.

To address some of the challenges of implementing the proposed principles and attributes, the Phase 2 advisory committee suggested modifications to the Phase 1 attributes to make them more applicable to the information sources prioritized during Phase 2 (see Table 2) and discussed how to ensure, within all the principles and attributes, that credibility is actually achievable (see Table 2 for all proposed modifications to the attributes).

TABLE 2. Phase 2: Proposed Modifications to Attributes of Foundational Principles for Identification of Credible Sources of Health Information in Social Media.

SOURCE: Created by authors.

NOTE: Text that is bold represents the additions, changes, and deletions generated by the advisory committee in Phase 2. Attributes that are the same across sources are presented in a merged row.

Principle: Inclusive and Equitable

The Phase 2 advisory committee identified DEI as a cross-cutting theme of sufficient importance to elevate it as a new principle. The Phase 2 advisory committee then drafted attributes that could help in the evaluation of adherence to DEI standards for all potential sources of credible information in a global context, including those considered in Phase 1. The Phase 2 advisory committee believes this principle is necessary to ensure that the suggested attributes included with the other three principles do not inadvertently suppress credible information from diverse sources and voices (see Box 4 for more details about the new principle and Table 2 for a complete list of all principles and attributes). The Phase 2 advisory committee recognizes that the new principle includes attributes that apply to both content and source and therefore exceed the scope of work for this paper. Still, the Phase 2 advisory committee agreed that this new principle and accompanying attributes are an important aspirational call for improvement in the credibility of online health information and must be included.

Box 4. Fourth Principle: Inclusive and Equitable.

Principle

Inclusive and Equitable: Sources should prioritize inclusion of diverse, equitable, and trusted voices for health information that reflect the demographics of the audience

Attributes

• Use accessible and culturally appropriate language for intended population

• Avoid stigmatizing language about specific groups of people

• Prioritize equitable access to health information

• Contextualize and make research relevant for the intended population

Credibility Factors

While the overarching principles from Phase 1 were retained into the Phase 2 work, it became apparent through discussion that the individual attributes for each principle may differ across the sources prioritized in Phase 2. Each of the potential sources of health information considered in Phase 2 brings different challenges to both establishing credibility and avoiding inadvertently diminishing the reach of diverse voices.

For example, when considering information developed by for-profit organizations, such as a pharmaceutical manufacturer, the underlying motive is often to drive customers to the business. However, for-profit businesses may provide highly useful and credible information for patients, including education and resources. Regarding the reach of diverse voices, there is a concern about the risk of setting the bar of credibility at a level where it is never attainable by a given source. If credibility is not attainable, then diverse voices may be inadvertently or indirectly silenced. Conversely, while nonprofit organizations may be perceived as having a higher level of transparency and credibility than for profit organizations, nonprofits are often motivated to increase brand awareness and fundraising, which can raise similar concerns to those regarding for-profit motivations and potential conflicts of interest. Therefore, credibility needs to be uniquely assessed for nonprofits and for-profits, and an iterative process is needed to off-ramp or on-ramp entities based on how well they meet the criteria necessary to be deemed credible. Edits to these attributes across the sources of information prioritized in Phase 2 can be found in Table 2.

The advisory committee proposed that the attributes within each principle could be prioritized to allow social media platforms to focus on the most essential attributes necessary to identify credible sources of health information. Pragmatically, the Phase 2 advisory committee determined that the sources of information could be required to meet only a preponderance of the attributes, rather than demonstrating adherence to each and every attribute. Understanding that it may seem like a loophole to allow sources to meet only a preponderance of the attributes, the Phase 2 advisory committee worked to balance an ideal set of attributes with the inability of most sources, even those with significant resources, to realistically meet all the attributes necessary to be deemed credible. The advisory committee believes that meeting a preponderance of the attributes will allow for an effective deployment of this framework into actual practice.

Source-Specific Considerations

The Phase 2 advisory committee outlines below some of the important points raised during discussion of each source of information prioritized in Phase 2 and some suggestions to address concerns in assessing each individual source's level of credibility.

Non-Accredited Nonprofit Organizations

There was some discussion around the special nature of non-accredited nonprofit organizations. For example, accreditation can be a fluid state, whereby an organization may enter into and fall out of accredited status. If, for example, an accredited nonprofit loses its accreditation, it should be further vetted against the principles and attributes developed in Phase 2 and listed in Table 2. Alternatively, a non-profit that achieves new accreditation could then be solely considered against the principles and attributes outlined in Phase 1. Due to the fluid state of accreditation, it will be necessary to re-vet an organization periodically to determine its status and where it falls within the scope of defined attributes. Because there is no pre-existing vetting mechanism for the sources prioritized in Phase 2, social media companies should develop a standardized process and reassessment intervals for evaluating source alignment with the principles and attributes outlined in Table 2.

Organizations with nonprofit status tend to be viewed with a halo of credibility, impartiality, or in a positive light due to the nature of their mission. However, it should be recognized that nonprofit organizations, which are typically mission-driven, can still be sources of mis- or disinformation. While one of the important attributes underpinning the principle of science-based is providing citations and a synthesis of information from multiple sources, the Phase 2 advisory committee was concerned with the potential for citing pseudoscience or selectively choosing references that support a particular viewpoint without identifying and discussing conflicting evidence. The Phase 2 advisory committee also suggested that social media companies examine the currency of citations, where older information may be cited and newer citations demonstrating the progression of knowledge on a particular subjected may be ignored or not updated in a timely manner.

In terms of transparency, many nonprofits create information to facilitate fundraising efforts, which raises the question of whether content used for fundraising is less legitimate than other types of content. The Phase 2 advisory committee believes that delegitimizing such content could be needlessly punitive. Another consideration regarding transparency is that many nonprofits are beholden to very few or even a single funder. Therefore, even if a nonprofit attempts to maintain independence, it may not be possible to completely remove the bias of what the funder would want or not want represented. Finally, some nonprofits obtain most of their funding from revenue streams such as subscriptions (to journals or products), membership dues, or from annual society meetings, which can include income from vendors renting booths or advertising in journals. A potential approach a nonprofit in this situation could take to abide by the principle of transparency would be to clearly describe their process for segregating their funding sources from the health information presented.

For-Profit Organizations

Several themes arose during discussion of for-profit organizations, with the Phase 2 advisory committee expressing many similar points to the discussion of non-accredited nonprofit organizations. In many cases the overlap between these two sources of content was so broad that there is potential for the attributes for the different sources to be condensed and applied across both source types. The Phase 2 advisory committee discussed the tension between completeness of information for each potentially credible source and the pragmatic need for ensuring that implementation of the criteria is practical on a global scale across a variety of social media platforms.

The Phase 2 advisory committee recognized that all groups presenting health information, inclusive of nonprofits and individuals, would potentially realize financial or non-financial gains should they be deemed a credible source and elevated as such. Additionally, the Phase 2 advisory committee recognized that deeming one source as credible may potentially confer a commercial advantage over another source that was not deemed credible. However, determining the credibility of for-profit organizations is even more challenging in that they are, by definition, seeking financial gain through the dissemination of health information. This motive, on its own, establishes an explicit conflict of interest as the dissemination of health information cannot be uncoupled from pursuing financial gain. With that understood, for-profit organizations can mitigate concerns about this and other conflicts of interest by demonstrating adherence to the principles of being science-based, objective, transparent, and inclusive and equitable; not promoting their own product in health information posts; and meeting the attributes that support the principles outlined in Table 2. The Phase 2 advisory committee also acknowledged that for-profit entities (and individuals and non-profits supported by these for-profit entities) may be subject to U.S. Federal Trade Commission (FTC) and U.S. Food and Drug Administration regulations regarding allowable promotion of products.

Individuals

Identifying credibility attributes of individuals was identified by the Phase 2 advisory committee as their most challenging task.

Consideration of this group covers both individuals on their own and individuals within the context of an organization. One of the complex issues considered by the Phase 2 advisory committee is the relationship of an individual to their organization. For example, if an individual is the chief medical officer of a health system, how is their individual credibility related to the credibility of the organization? The relationship of the individual's credibility to the credibility of the organization would, necessarily, be informed by the status and attributes for credibility of the organization as a whole. Individual sources may not represent the consensus view of their organization and organizations may have a limited oversight role of the social media presence of individuals within the organization. There was agreement within the Phase 2 advisory committee that it may not be possible to hold individual sources to the same attributes as nonprofit or for-profit organizations with large staffs—particularly regarding attributes such as the use of a consensus process to develop the information that the individual is sharing (which would be untenable for a true individual). However, the Phase 2 advisory committee believes that, in order to be compliant with the principles listed in Table 2, individuals should consult experts and evidence-based sources on a topic prior to sharing information on social media.

Another attribute that posed a potential untenable burden on individuals was the criterion of updating content as science and knowledge evolves. The Phase 2 advisory committee felt it was unrealistic to expect individual creators to continue to update their posts constantly or frequently, but it was agreed that having an initial date accompany the posting was an important criterion to keep. The Phase 2 advisory committee also thought that it was reasonable for individuals to link out to other high-quality sources, but that it did not seem practical to expect that an individual would need to have other organizations or individuals linking to them as a marker of credibility.

The Phase 2 advisory committee believes that documentation of funding, advertising, and paid partnerships is particularly important for individuals to disclose. Additionally, it is important to delineate between content for which an individual is paid versus content that is strictly advertising, sponsored content, or lobbying. Some, but not all, social media platforms have requirements for clearly labeling advertising, so in the absence of this requirement, creators should disclose this themselves. Moreover, in some countries, local guidelines and regulations require these disclosures on social media posts, including the FTC (CFR, 2009). From a practical standpoint, it might be sufficient for individuals to provide attestations about independence of funding, but this is an issue to be considered in implementation.

Another challenge is establishing the credibility of an individual and their lived experience, and distinguishing “health stories” from “health information.” More specifically, information about various facets of the health care system and health care delivery from patients with lived experience is considered important for peer support and essential for clinical guideline development and patient-centered research. Online peer support in self-management of health concerns is a valued and valuable source of information for individuals living with illness, especially when clinical information is lacking. However, anecdotal health stories may or may not be based on scientifically reliable information, and although stories matter, they do not necessarily meet foundational principles of credibility regarding science-based information.

The Phase 2 advisory committee discussed various mechanisms to identify the scientific or medical credentials of individuals, as these could be considered important attributes for credibility. However, trying to find a single credential that is used across nations and the world is a significant barrier, as a single credential does not exist. While U.S. health care professionals may carry board certification and other credentials, these are not universally available in the global community. Relatedly, credible lay individual sources may not have such credentials, and ultimately, possession of such credentials does not necessarily ensure credibility (Rubin, 2022). The Phase 2 advisory committee suggested that individuals could provide disclosures of regionally appropriate licensure, education, training, and scientific expertise to social media platforms while recognizing that at present, not all social media platforms provide the means for an individual to disclose such information.

However, it is important to underscore that the Phase 2 advisory committee believes that individuals should be held accountable to the same principles as the other sources described in Phases 1 and 2, although the specific attributes may need to be modified to make credibility more achievable for an individual.

Implementation

Establishing a set of principles and attributes by which sources can be deemed credible is a vital first step, but the proof of its effectiveness will be in the implementation of these criteria. The use of these principles and attributes in elevating credible sources needs to be evaluated with a critical eye both towards applicability for the global community and across multiple social media platforms. The Phase 2 advisory committee agreed that attestation alone, while a first step, would be insufficient for establishing source credibility. The Phase 2 advisory committee therefore supported creating a standardized biographical statement or attestation for individual sources to use to consistently link to key attributes like licensure, expertise, and conflicts of interest across social media platforms. Ideally, these attributes should be able to be verified independently. Moreover, the Phase 2 advisory committee believes that source credibility must be reviewed regularly to both allow new sources to become credible and to remove sources that no longer meet credibility criteria. Specific criteria to be evaluated within each attribute should be considered within the context of local regulations (e.g., FTC regulations in the U.S. regarding advertisements). From an end-user standpoint, the Phase 2 advisory committee believes it is valuable for the consumer to understand the factors used to define the credibility of a source. The Phase 2 advisory committee also encouraged as much consistency as is feasible in the application of attributes across social media platforms. Finally, beyond the present goal of identifying credible sources, the Phase 2 advisory committee emphasized the need to further explore pragmatic and effective means of managing the larger issue of health-related mis- and disinformation online.

Assessment and Testing of the Processes

The first paper in this series extensively outlined a series of steps to be undertaken when assessing sources of information that cannot be afforded a preliminary assumption of credibility (Kington et al., 2021). Those sources are the focus of this paper. Social media platforms will need to develop standardized processes to assess how well a given source aligns with the principles and attributes that would allow for a judgement of credibility. These standardized processes might rely on primary data collected by the platforms themselves or could rely on secondary data provided by the source.

Recognizing the implementation challenges that are likely to occur, the Phase 2 advisory committee agreed that it is essential to iteratively test how algorithms perform in accurately flagging credible sources of valid health information, and, ultimately, how consumers make use of the results. The Phase 2 advisory committee also emphasized that testing of the assessment process itself will be critical both prior to wide-scale implementation of assessing sources (through use cases and pilot tests), and over time (by following potential credible sources longitudinally) to determine whether the processes are functioning as intended and whether there is evidence of inadvertent harm. The Phase 2 advisory committee believes that this testing should include global social media platforms. The Phase 2 advisory committee encourages social media platforms to develop transparent, standardized, digitally verifiable processes to assess how well a source aligns with the principles and attributes that would allow for a judgment of credibility. Lastly, the Phase 2 advisory committee encourages social media platforms to collaborate with research experts to assess the impact and validity of identifying credible sources and elevating them on their platforms.

Conclusion

Given increasing challenges to accessing high-quality health information on social media, elevating credible sources of health information online will assist consumers in finding information they can trust. The COVID-19 pandemic highlighted how vast amounts of misinformation and disinformation damaged individual and population-level health and even life expectancy. Social media platforms that elevate the most viewed posts, regardless of source credibility, potentially contribute to the viral spread of misinformation. As more people search for health information on social media, it is therefore critical that social media platforms provide indicators that identify and elevate the most credible sources of health information. Ideally, consumers would choose to consider sources deemed most credible in their personal evaluation of health information sources.

The advisory groups convened in both Phase 1 and Phase 2 of this work agree that research undertaken by social media platforms will be most valuable and useful if the research data and processes are shared externally in a transparent manner.

Building on the prior work of the advisory group convened by the NAM in 2021 that yielded the foundational principles and attributes for determining credibility of health information sources, this paper provides guidance for a wider range of potential sources, including the full range of non-profit and for-profit organizations and individual sources. The additional guidance provided in this paper in Phase 2 will further the goal of identifying and elevating credible sources of health information, although the Phase 2 advisory committee acknowledges that more work remains to be done.

In particular, the new principle on the importance of promoting diverse and equitable voices on social media will require support from funders, government agencies, and social media platforms to assist historically marginalized groups, including patient groups and community-based non-profits, to be able to offer social media content that is grounded in evidence and free of bias. Both grant makers and accredited organizations, who are subject to the Phase 1 principles, should support the development of credible health content by authentically representative groups or individuals through their research agendas, grantmaking, and partnerships between accredited organizations and community-based groups. Ideally, all types of groups—including patient organizations, community organizations, and individuals—will see a benefit to attesting to scientific rigor, objectivity, transparency, and inclusivity through greater visibility of their social media content.

While this paper provides principles and attributes on the full array of potential sources for social media platforms, source credibility is likely to be dynamic and periodic reassessment of source credibility may be required. In addition, source credibility may not be an adequate protection against misinformation if sources do not uniformly share credible content. The Phase 2 advisory committee encourages social media platforms to iteratively test how algorithms perform in accurately identifying credible sources of health information.

Ultimately, by applying and evaluating this committee's principles and attributes of credible sources, social media platforms may be able to elevate credible health information sources and potentially break the cycle of misinformation.

Appendix A: Project Team

Council of Medical Specialty Societies (CMSS)

• Julia Peterson, CAE, Chief Operating Officer

• Suzanne Pope, MBA, Project Manager

National Academy of Medicine (NAM)

• Laura DeStefano, Director of Strategic Communications & Engagement

World Health Organization (WHO)

• Andrew Pattison, MS, Team Lead, Digital Channels

• Monta Reinfelde, MS, Technical Officer, Digital Communications

Sigma Health Consulting, LLC

• Kristen E. D'Anci, PhD, Science Writer, Senior Manager

• Frances M. Murphy, MD, MPH, Expert Advisor, President & CEO

Funding Statement

The views expressed in this paper are those of the author and not necessarily of the author's organizations, the National Academy of Medicine (NAM), or the National Academies of Sciences, Engineering, and Medicine (the National Academies). The paper is intended to help inform and stimulate discussion. It is not a report of the NAM or the National Academies.