Exploring online health information quality criteria on social media: a mixed method approach

Abstract

Purpose

This article outlines a research study that ranked health information quality criteria on social media from experts’ perspectives.

Methodology

A mixed-method approach (qualitative-quantitative) used in current research. In the qualitative phase a literature review explored existing dimensions for evaluating social media content quality, focusing on identifying common dimensions and attributes. Furthermore, a quantitative method involving experts was utilized to rank the health information quality criteria for social media.

Results

The findings indicated various dimensions of health information quality in the literature. Out of 17 criteria, accuracy, credibility, and reliability had the highest ranks, while originality, value-added, and amount of data had the lowest ranks, respectively, according to experts.

Conclusion

The endeavor to bolster the dissemination of reliable health information on social media demands a sustained commitment to enhancing accountability, transparency, and accuracy, ensuring that users have access to information that is not only informative but also trustworthy.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-024-11838-8.

Introduction

Social media, encompassing a broad range of Internet-based applications built on the principles of Web 2.0, facilitate user-generated content creation and sharing [1, 2]. This encompasses a variety of platforms, including social networking sites to enhance participation, immediacy, and user interaction [3–6].

Social media have increasingly outpaced traditional media as the go-to source for health information due to their pervasive reach [7, 8]. They have revolutionized how health information is disseminated and exchanged, empowering users and professionals within the health sector through education and shared experiences [9–11]. Social media play a pivotal role in preparing patients to adopt new diagnostics and treatment modalities, with over 60% of people turning to them as their initial source of health information [12]. They have also been instrumental in reducing healthcare costs by providing readily accessible information [13, 14].

The utility of social media extends to various public health education objectives, for example managing chronic conditions, and bolstering patient self-management capabilities [2, 3, 15, 16]. They democratize access to health knowledge and promote evidence-based medicine by making high-quality health information more attainable and supporting the dissemination of health knowledge [17, 18]. Notably, there is a significant correlation between health literacy and the use of social media for health information [19–21].

Nonetheless, the vast expanse of health information on social media is marred by challenges in identifying trustworthy sources and users’ generally low awareness regarding the quality of the information they encounter [2, 4, 22]. The absence of specialized knowledge complicates assessing the quality of health information sourced from social media [23, 24]. Various tools and criteria like the HONcode, DISCERN, Silberg scale, CRAAP test, and Medline Plus have been developed to aid users in evaluating web-based medical resources [2, 25–27]; however, less emphasis has been placed on scrutinizing the content found on social media platforms. Health professionals express concerns about the prevalence of inaccuracies and misinformation, noting that 44% of problems faced by online health information seekers stem from inadequate evaluation of the content, despite 85% of these individuals reporting satisfaction with their search outcomes [28].

It is critical to enhance content quality evaluation from the end-user’s perspective to improve decision-making processes regarding health information on social media. Identifying and prioritizing quality criteria based on scientific evidence is vital. This research aimed to assist consumers, caregivers, and individuals in locating high-quality health information resources on social media more effectively. By establishing a hierarchy of criteria for assessing the quality of online health information in social media, this study endeavored to guide users toward selecting superior health resources that meet their informational needs.

Methodology

This study adopted a mixed-methods approach, leveraging two primary data sources to investigate the quality dimensions and criteria of health information on social media. Initially, a literature review was conducted to identify the critical dimensions of information quality within social media environments. This review spanned two electronic databases, namely Scopus and PubMed, employing a strategic search protocol. Relevant search terms, such as “health,” “information,” “quality,” and “social media,” were meticulously selected and utilized in conjunction with Boolean operators (AND, OR) to refine and target searches within the title, abstract, and keywords of these databases. After search in different databases, retrieved documents were evaluated using the GBI checklist and eligible articles were selected. Furthermore To ensure the quality, four reviewers of the research team screen retrieved resources. The selected articles were thoroughly studied and reviewed, and the required information was extracted in the narrative review section using the designed form. Also, Endnote resource management software was used to organize, study titles and abstracts, as well as identify duplicates. As an example, the search method in some selected databases was as follows.

Pubmed ((((“well-being“[Title/Abstract]) OR (health*[Title/Abstract])) AND (information[Title/Abstract])) AND (“social media“[Title/Abstract] OR “social platform“[Title/Abstract] OR “internet community“[Title/Abstract] OR “social media service“[Title/Abstract] OR “social media website“[Title/Abstract])) AND (“quality“[Title/Abstract]) AND (y_10[Filter]) retrieved number:1235.

To analyze the data, the articles were read several times and the textual data were manually analyzed with the content analysis method and the most important topics related to the categories of health information quality in social media, information quality evaluation criteria, dimensions of information quality were extracted.

The research team employed a narrative review methodology complemented by a forward chaining (citation searching) strategy. This involved starting with a seminal paper on the topic and tracing its citations forward to uncover relevant subsequent publications, primarily utilizing Google Scholar.

Furthermore, Delphi method was used to reach a general agreement about the components of health information in social networks in the next phase. The opinions of the people were approved in the first round of Delphi and the subsequent rounds were not conducted. The insights from the literature review informed the development of a data collection instrument for the study’s second phase. This instrument, a close-ended questionnaire employing a five-point Likert scale, was distributed among 94 experts selected through purposive snowball sampling. Eligibility for participation in this stage required the individual to have at least five years of experience and actively use social media to disseminate health information. Criteria selection and prioritization by the participants were then systematically ranked, laying the groundwork for a future health information quality assessment model specific to social media. Participant anonymity was maintained throughout the study, and each participant was assigned a numerical code representing their participation sequence. All stages of the research strictly safeguarded the participants’ characteristics, ensuring confidentiality when presenting the recorded findings.

Results

Several online information quality evaluation criteria were suggested in the literature before the rise of social media(). These tools were repeatedly used in subsequent studies to evaluate health information on social media. Notable examples are the JAMA benchmark [29], the DISCERN instrument [30], the HONCode [31], and the LIDA instrument [32]. The Journal of the American Medical Association (JAMA) utilizes a set of four criteria to evaluate the quality of information. These criteria include authorship, attribution, disclosure, and currency. The HONCode guidelines also provide specific guidance to publishers and authors of online health information. These guidelines are based on eight metrics, which include authority, complementarity, confidentiality, and advertisement policy. Furthermore, the DISCERN instrument is recognized as a reliable tool used by both health information providers and users to assess the quality of information related to treatment options. The DISCERN instrument consists of a five-point scale with 15 key questions, along with an overall quality assessment question that evaluates the reliability of online health information [31]. Conversely, the LIDAs evaluation criteria evaluate the effectiveness of health-related websites based on accessibility, readability, and usability metrics. In addition to these tools, the Elaboration Likelihood Model (ELM) [33] has been utilized to investigate the factors impacting consumers’ information choices via persuasive communications [34, 35]. According to the ELM, individuals may either critically analyze information to determine its true value or be influenced by superficial cues like source aesthetics without thorough examination of the content [23, 33].

The foundational work on information quality introduces a hierarchical framework based on utility for the consumer, characterized by four primary dimensions: intrinsic, accessibility, contextual, and representational [36]. Due to their comprehensive nature, these dimensions encapsulate the multifaceted construct of information quality [37]. Knight and Burn consolidated 12 recognized information quality frameworks from the past decade of information systems research, revealing commonalities in their dimension classifications. Despite the diversity in approach, these frameworks consistently emphasized traditional quality dimensions, such as accuracy, consistency, timeliness, completeness, accessibility, objectiveness, and relevancy, as summarized in Table 1. These shared dimensions underscore the convergence in the field’s understanding of information quality attributes [38].

Table 1.

The common dimensions of information quality [36–38]

Dimension	Definitions
Accuracy	extent to which data are correct, reliable, and certified free of error
Consistency	extent to which information is presented in the same format and is compatible with previous data
Security	extent to which access to information is restricted appropriately to maintain its security
Timeliness	extent to which the information is sufficiently up-to-date for the task at hand
Completeness	extent to which information is not missing and is of sufficient breadth and depth for the task at hand
Conciseness	extent to which information is compactly represented without being overwhelming
Reliability	extent to which information is correct and reliable
Accessibility	extent to which information is available or easily and quickly retrievable
Availability	extent to which information is physically accessible
Objectivity	extent to which information is unbiased, unprejudiced, and impartial
Relevancy	extent to which information is applicable and helpful for the task at hand
Usability	extent to which information is clear and easily used
Understandability	extent to which data are clear, without ambiguity, and easily comprehended
Amount of data	extent to which the quantity or volume of the available data is appropriate
Believability	extent to which information is regarded as true and credible
Navigation	extent to which data are easily found and linked to
Reputation	extent to which information is highly regarded in terms of source or content
Usefulness	extent to which information is applicable and helpful for the task at hand
Efficiency	extent to which data can quickly meet the information needs for the task at hand
Value added	extent to which information is beneficial and provides advantages from its use

In social media, the initial research predominantly equated information quality with content quality, often sidelining user perceptions [4, 39, 40]. The dynamic nature of social media necessitates a definition of information quality that incorporates user characteristics, tasks, and environmental context encapsulated by the “fitness for use” principle [4, 41]. This approach underscores the relational aspect of information quality, where user interaction and context play pivotal roles.

The integrity and reliability of health information on social media are frequently questioned, highlighting a significant challenge for users, policymakers, and healthcare providers [42]. Despite concerns over misinformation and the credibility of sources, user trust in the quality of health information remains largely predicated on perceived reliability and trustworthiness [43, 44]. The difficulty in verifying the authenticity and accuracy of health information sources on social media exacerbates this problem, often leading users to accept information at face value without rigorous scrutiny [22, 45].

The ambiguity surrounding the certainty of health information on social media stems from their potential for misinformation and unpredictable consequences [46–48]. The perceived reliability of online health information hinges on user-perceived accuracy, trustworthiness, and clarity, among other factors [49, 50]. Credibility assessments often rely on cues related to authorship and the source’s legitimacy, influenced by the platform’s ownership, objectives, and user engagement metrics, such as comments, shares, and likes [22, 23, 43, 51].

Agarwal and Yiliyasiv (2010) highlighted that traditional accessibility measures do not adequately address social media information quality issues [42]. Other studies used the bottom-up approach, derived from a comprehensive review of prior research, emphasizing the lesser relevance of accessibility in social media contexts and proposing intrinsic quality, representational quality, and contextual quality as the pivotal dimensions [4, 36, 52].

The criteria identified from the literature (Table 2) were utilized to construct a questionnaire for a sample of 94 experts. Based on Table 3, the demographic analysis revealed a majority of male participants, with a significant portion holding graduate degrees and substantial experience. Many participants had been engaged with social media for over eight years.

Table 2.

Adopted information quality criteria and measures for the social media context [36, 52]

criteria	Measure	Description
Intrinsic information quality	Completeness	Extent to which the information is not missing and is of sufficient breadth and depth
	Originality	How much information is not copied from other sources
	Objectivity	Extent to which information is unbiased and unprejudiced
	Novelty	If the information is innovative
	Accuracy	The degree to which data are correct, reliable, free of errors, and current
	Verifiability	The degree to which information can be checked for correctness
	Reliability	Extent to which information is correct and reliable
Contextual information quality	Amount of data	Extent to which the quality or amount of data is appropriate
	Relevancy	Extent to which information is applicable for the task in hand
	Credibility	Believability or the characteristic that makes people believe and trust someone or something
	User feedback	Users provide either an implicit or explicit quality evaluation of the content.
	Timeliness	Extent to which information is sufficiently up-to-date for the task in hand
	Understandability	Extent to which data are clear, without ambiguity, and easily comprehended
	Value added	Extent to which information is beneficial and provides advantages from its use
Representational information quality	Conciseness	Extent to which information presentation matches the information and is compact, represented without being overwhelming, has granularity, and appropriateness
	Consistency	The same format and compatible with previous data
	Accessibility	Extent to which information is available or easily and quickly retrievable

Table 3.

Demographics of research participants

Profile of respondents	Frequency	Percentage (%)
Gender
Female	23	24.46
Male	71	75.53
Age
25–34	52	55.31
34–44	31	32.97
45–54	9	9.57
More than 55	2	2.12
Education level
Graduate	3	3.19
Postgraduate	29	30.85
Ph.D	52	55.31
MD	8	8.51
Work time experience
3–10	9	9.57
11–20	79	84.04
More than 20	6	6.38
Experience using social media
1–4	14	14.89
5–8	36	38.29
More than 8	44	46.80

The survey, employing a 5-point Likert scale, asked the participants to prioritize health information quality criteria based on importance. The completion rate stood at 94%, with descriptive statistics provided in Table 4. Participants overwhelmingly ranked the accuracy dimension as paramount, while the amount of data dimension was deemed the least critical, illustrating the prioritization of quality over quantity in health information assessment.

Table 4.

The ranking of health information quality criteria in social media by importance

Criteria	Rank	Measure	N	Importance (Likert Scale)					Mean	Median	SD
				Very High	High	Medium	Low	Very Low
Intrinsic Information quality	1	Accuracy	94	84	7	2	1	0	4.85	5	0.68
	2	Reliability	94	81	10	0	2	1	4.78	5	0.30
	3	Objectivity	94	78	10	3	0	3	4.70	5	0.78
	3	Verifiability	94	76	13	2	1	2	4.70	5	0.93
	4	Completeness	94	5	84	3	1	1	3.96	4	0.63
	5	Novelty	94	7	81	3	2	1	3.96	4	0.16
	6	Originality	94	9	68	2	11	4	3.71	4	0.81
Contextual information quality	1	Credibility	94	85	5	0	2	2	4.79	5	0.13
	2	Understand ability	94	79	4	2	8	1	4.61	5	0.19
	3	Relevancy	94	9	81	0	2	2	3.98	4	0.24
	4	User feedback	94	10	74	1	5	4	3.86	4	0.75
	5	Timeliness	94	11	70	4	3	6	3.81	4	0.74
	6	Value added	94	9	68	3	8	6	3.70	4	0.68
	7	Amount of data	94	7	65	0	4	18	3.41	4	0.86
Representational Information quality	1	Consistency	94	73	8	3	8	2	4.51	5	0.21
	2	Accessibility	94	8	72	2	6	6	3.74	4	0.67
	3	Conciseness	94	13	64	3	7	7	3.73	4	0.82

Furthermore, the determination of experts’ agreement and the weight of each criterion was achieved by classifying the criteria according to statistical quartiles. Table 5 illustrates that accuracy, credibility, and reliability, which are positioned in the fourth quartile, hold the greatest weight. On the other hand, the first quartile encompasses four criteria, namely Conciseness, Originality, Value added, and Amount of data, which, according to experts’ opinions, carry less weight compared to the other criteria.

Table 5.

Quartiles and weight of online health information quality criteria on social media

Quartile	Quartile Value	Criteria	Experts’ Agreement	Weight
Larger Quartile (Q4)	4.850	Accuracy Credibility Reliability	96.808% 95.744% 96.808%	6.85% 6.76% 6.75%
Upper Quartile (Q3)	4.700	Objectivity Verifiability Understand ability Consistency Relevancy	93.617% 94.680% 88.297% 86.170% 95.744%	6.63% 6.63% 6.51% 6.37% 5.62%
Midle Quartile (Q2)	3.960	Completeness Novelty User feedback Timeliness Accessibility	94.680% 93.617% 89.361% 86.170% 85.106%	5.59% 5.59% 5.45% 5.38% 5.28%
Lower Quartile (Q1)	3.735	Conciseness Originality Value added Amount of data	86.170% 81.914% 81.914% 76.595%	5.26% 5.24% 5.22% 4.81%

Discussion

The current study emphasized the importance of experts’ opinion in identifying and ranking health information disseminated through social media. They strive to present information that is not only precise but also succinct, consistent, complete, and credible. Reliability emerged as a critical dimension, highlighting the necessity for information users to find understandable information that is devoid of excessive advertising and easy to navigate [23, 53, 54]. This study underscored the significance of credibility, objectivity, and verifiability, with high mean scores indicating that experts’ perceptions of these information quality dimensions vary across domains yet remain crucial. Previous research has echoed concerns over the accuracy of health information on social media, with particular alarm sounded over content related to life-threatening conditions, such as cancer and diabetes, which can significantly heighten patient anxiety [25, 55]. Interestingly, even non-life-threatening fields like dental care have been cited for poor information quality on social media, indicating the widespread nature of these concerns [2]. The average health information consumer often lacks the requisite knowledge or skills to accurately evaluate the credibility and quality of content on social media, tending instead to rely on subjective trust indicators and criteria [4, 56, 57]. Such reliance on subjective assessments can pose risks to user health [28].

This study suggests that further analysis might unveil additional health information quality dimensions relevant to the social media context that have not been deemed significant in the existing literature. Validating these findings through user feedback on their perceived health information quality rankings could pave the way for future research endeavors. This study illustrates that social media cannot be viewed as a monolithic entity; it comprises various platforms with information quality dimensions and requirements. Nonetheless, it also hints at the universality of health information quality dimensions across the social media landscape, suggesting an area ripe for further investigation across a broader array of social media platforms and among different professional groups to enhance the generalizability of these results.

The literature frequently characterizes health information on social media with terms such as “fake,” “misinformation,” “misleading,” and “inaccurate,” pointing to the pervasive problem of low-quality content [58, 59]. Consequently, there is a growing call for health professionals, particularly physicians, to scrutinize the information their patients are accessing on social media [13, 60]. This involves guiding patients and the public on assessing the quality of health information resources and actively engaging on social media platforms to counteract the spread of low-quality content, predominantly generated by non-expert users and generalist sources. By taking a proactive stance in social media health communications, health professionals can play a critical role in interrupting the cycle of misinformation and directing individuals toward reliable and trustworthy health information sources [2, 3, 13].

Due to the expansion of various interactive tools, institutions in charge of health information should pay attention to the periodic monitoring of this media and formulate information quality assurance criteria with their new features. On the one hand, experts from different health groups such as doctors, medical staff, librarians and information specialists should participate in compiling information content and on the other hand, they should pay attention to the user-oriented perspective in evaluating the quality of information.

Conclusion

In the contemporary landscape, social media platforms have emerged as pivotal sources of health information. Given the considerable variability in the quality of this information, developing robust methods for assessing credibility is imperative. This entails a multifaceted evaluation process, incorporating various tools, criteria, and markers to discern the quality of health information available on social media. By adhering to stringent criteria and seeking out reputable sources, users can navigate online health information more effectively, making more informed health-related decisions. The endeavor to bolster the dissemination of reliable health information on social media demands a sustained commitment to enhancing accountability, transparency, and accuracy, ensuring that users have access to information that is not only informative but also trustworthy.

Authors’ contributions

Hossein Ghalavand and Abdolahad Nabiolahi developed the theoretical formalism, performed the analytic calculations and performed the numerical simulations. Both authors contributed to the final version of the manuscript.

Funding

This study was supported by Abadan University of medical sciences, Research code: 1692.

Data availability

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

This research study was approved by Ethics Committee in Biomedical Research at Abadan University of Medical Sciences (Ethical code: IR.ABADANUMS.REC.1402.078). Every participant gave informed consent prior to taking part in the research after they were briefed on the study’s goals and advantages. Voluntary participation was encouraged. In order to protect subjects’ privacy, data collection method was conducted anonymously.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.