Quality Assessment of Information on Oral Cancer Provided at Arabic Speaking Websites

Abstract

Background:

Patients often use Inernet to explore information about their health and disease. This study aimed to evaluate the quality of information on oral cancer provided at Arabic websites.

Material and Method:

The translated Arabic keywords of “oral cancer” and treatment of oral cancer” and 6 search engines were used. The top 100 websites were selected. Irrelevant and duplicates websites were excluded. To evaluate the quality of websites and their information, DISCERN, HON, and JAMA benchmarks were used.

Results:

The majority (n = 64, 74.4%) of websites were founded by profit organizations. The mean overall rating was 2.23 (1.6) out of 5 and the median overall rating was 1 (1-3) based on DISCERN. According to HON, only 4.7% of the websites achieved a high score (≥75) and 37.2% of the websites gained a low score (<50). Based on JAMA benchmarks, 15% of the websites did not fulfill any criteria and only 2% of them fulfilled the four criteria proposed by the JAMA benchmarks.

Conclusion:

This study suggested the poor quality of web-based Arabic information on oral cancer. It is recommended to develop a websites based system by which enable to the Arabic websites related to oral cancer know their shortfalls, therefore, improve their quality according to evaluation tools which will ensure finding reliable data from the websites.

Introduction

The Internet has become one of main information to search for health and disease. A lot of patients incline to browse websites to clarify their health-related concerns, even before consulting the specialist. In a previous study, it was reported that 80% of US population were searching for online health information, including treatment methods (Castleton et al., 2011). In Saudi Arabia, among 344 diabetic patients, only 39% were Internet users, of whom 71.6% used the Internet for seeking health-related information (Jamal et al., 2015). In addition, it was found that more than three-quarters of the Arab participants (79%) did not use the Arabian health websites (Al Huziah et al., 2009). Mouth cancer is the 6^th prevalent cancer and considered among the ten top leading cause cancer related death in the world with an estimated overall annual incidence of nearly 443,000 new cases and 241,450 deaths worldwide (Forman et al.; Ferlay et al., 2015). Most of the new oral cancer cases are diagnosed in the developing world (Warnakulasuriya, 2009). The Middle East and North Africa region is constituted of 23 states, with a population of over 448 million people speaking Arabic. The incidence of cancer in the Middle East and North Africa region is on the increase, and it is currently the 4th leading cause of death (Organization and Organization, 2009). The incidence rates of oral cancer in these countries for females and males were 1.8 and 2.6, respectively, per 100,000 per annum in 2015 (Ferlay et al., 2015). Chewing habits, such as chewing Shamma, tobacco, and khat are extensively used in Arabic countries (Al-Ak’hali et al., 2017). There is a strong relationship between chewing Shamma and oral cancer (Basha et al., 2019).

The diagnosis of oral cancer may cause distress for the patient because the threat of life caused by oral cancer such as disability and pain and it is an effect on the physical and psychosocial status of the individual. Living with such potentially malignant disease is associated with modifiable risk factors such tobacco, excessive alcohol consumption and the use of betel quid, often requires effective health motivation and behavior changes, which demand that an individual has an awareness of their disorders and is provided with the data and solutions necessary to initiate and maintain these behavior changes (Schwarzer and Fuchs, 1995; Warnakulasuriya, 2010). Many researchers have evaluated the impact of online information on health behaviors changes, including disease treatment and tobacco smoking cessation (Strecher, 1999; Portnoy et al., 2008). There is huge information on oral cancer at Arabic websites. However, to the best of our knowledge, this is the first study that evaluated the quality and content of web-based Arabic information on oral cancer through evaluation of all Arabic internet websites have shown the original information related to oral cancer.

Materials and Methods

The the searching and identification of websites were done on 17 Jun 2019. Before conducting browsing, the author erased the cookie information and the browser. Browsing was done using incognito mode to prevent bias arising from preceding searches.

Searching and selecting websites

The most six popular search engines in Arabic, including “Google (http://www.google.com)”, “Yahoo (http://www.yahoo.com)”, “Bing (http://www.bing.com)”, “Ask (http://www.ask.com)”, “Aol. (http://www.aol.com)”, and “MSN (http://www.msn.com)” were used to identify the websites (Lebbos et al., 2014). The translated Arabic keywords of “oral cancer” and treatment of oral cancer” were used during the searches. The first 10 pages from each search engine were browsing visited within 24 hours to avoid any changes. Therefore, the first 100 consecutive websites identified from each engine were browsed using the first key word in Arabic language that was “oral cancer”. Then, similarly the first ten pages were browsed again using the second keyword in Arabic language, “treatments of oral cancer”. The default settings were kept and any advanced search options were avoided.

Exclusion criteria

At first, the repeated websites were eliminated. Then, extracted websites were excluded if:

a. did not use the Arabic language to present the information

b. mentioning oral cancer just by the hint only, exclusive audio, or visual-based

c. complete scientific articles or textbook

d. they were found to copied the report from another site

e. they included banner advertisements, sponsored links, or discussion forums

f. they denied direct access through keywords used

g. they contained no information on oral cancer

h. Workshops

Inclusion criteria

Websites that had a free entree, without a password requisite, presented in Arabic language, and provided information on oral cancer were involved.

Evaluation criteria

Websites that fulfilled the inclusion criteria were evaluated by two examiners independently and a common agreement was reached. The quality of the websites was evaluated using Discern plus Assessment (Charnock et al., 1999), JAMA Benchmarks (Silberg et al., 1997), and HONs assessment tools (Boyer et al., 2011). Intra and Inter-examiner calibration was done before assessing the quality of websites. The DISCERN, HON, and the JAMA benchmarks were selected as the validated tools to evaluate the quality of the medical websites. DISCERN is a questionnaire that provides users with a valid way of evaluating the quality of written information on different treatments for a health disorder (Charnock et al., 1999). This questionnaire includes 16 questions, divided into 3 sections. Questions 1–8 address the trustworthiness of the publication, which help to consider whether it can be trusted as a source of data about selected therapy, questions 9–15 focus on therapy options, and question 16 corresponds to the overall quality score at the end of the evaluation (Charnock et al., 1999). Each question is scored using a five-point Likert scale (1 indicates a poor quality and 5 a good quality). One expert (dentist) evaluated the consistency in marking websites using DISCERN. The JAMA benchmarks published by the Journal of the American Association include four standards: authorship, attribution, currency, and disclosure (Silberg et al., 1997). HON is a non-profit foundation that aims to assess the transparency and quality but not the precision of web-based health data. HON includes eight criteria: authority, complementarity, confidentiality, justifiability, attribution, financial disclosure, transparency, and advertising policy. HON page assistances the evaluator to evaluate the credibility, transparency, and quality of a health website by guiding the evaluator through questions related to the HON code principles and the Europe Quality Criteria for Health-related Websites. After replying a series of questions, the site will be given a grade in percent and indications regarding its level of production quality and transparency (Boyer et al., 2011). Two trained evaluators, both of whom were dentists, conducted the identification and evaluation of the websites.

Data were analyzed using SPSS (version 17) and were expressed in terms of frequency and percentages.

Results

The search process by using both keywords resulted in thousands of websites (Table 1). After eliminating the duplicates, only 184 sites were selected out of 600 websites (top 100 links from each search engine and keywords). In accordance with our exclusion criteria, 98 sites were then excluded (Figure 1). Therefore, 86 sites which met our inclusion criteria were included for further analysis (Figure 2).

Table 1.

Summary of Websites Classification According to Funding Background and Affiliation

Main Classification	Sub-classification	Distribution
		Frequency	%
Profit-based	Profit	64	74.4
	Non-profit	22	25.6
Profit-based expanded	Profit/Newspaper	15	17.4
	Profit/Social	40	46.5
	Profit/Medical Center	9	10.5
	Non-profit	22	25.6
Affiliation-based	Medical Center	9	10.5
	Newspaper	15	17.4
	Forum	9	10.5
	Cultural	30	34.9
	Medical Sites	23	26.7

Figure 1.

Diagram Shows the Inclusion and Exclusion of Websites for Evaluation

Figure 2.

Flow Chart of the Study

Search results obtained by each search engine

For all searches, Google introduced the highest related content; 87 sites, with less duplicating websites, compared to other search engines (Figure 2). Yahoo and Bing revealed the same number of sites; 20 sites for each engine with 61 and 75 duplicated sites, respectively. Ask, Aol, and MSN introduced 12, 21, and 24 related sites with 77, 78, and 90 duplicating websites, respectively (Figure 2). Highest excluded content was from MSN; 24 sites, followed by Aol; 20 sites. According to excluding criteria, 13, 16, 19, and 6 sites were respectively excluded from searches obtained from Google, Yahoo, Bing, and Ask introduced (Figure 2).

According to funding background, the profit organization websites hosted 64 sites (74.4%) and non-profit organizations hosted 22 sites (25.6%). According to the affiliation, the websites were divided to the following categories: Medical centers and clinics: 9 sites (10.5%), Newspapers: 15 sites (17.4%), forums: 9 sites (10.5%), Cultural sites: 30 sites (34.9%), and Medical sites: 23 sites (26.7%) (Table1).

Quality assessment

DISCERN

Based on the overall quality score and using DISCERN, the quality assessment of the 86 identified websites showed a mean overall rating of 2.23 (1.6) out of 5 and a median overall rating of 1 (1-3) (Table 2).

Table 2.

Shows Evaluation the Quality of the Medical Websites Using DISCERN Assessment

Domain	DESCREN Question	Score
		Mean (SD)	Median (IQR)
Reliability	Q1 Explicit aim (N=85)	2.68 (1.39)	2 (1-4)
	Q2 Attainment of aims (N=81)	2.05 (0.92)	2 (1-3)
	Q3 Relevance (N=86)	2.74 (0.97)	3 (2-3)
	Q4 Explicit source (N=86)	1.51 (1.11)	1 (1-1)
	Q5 Explicit date (N=86)	2.63 (1.27)	3 (1-4)
	Q6 Balanced and Unbiased (N=86)	2.8 (1.03)	2 (2-3)
	Q7 Additional sources (N=85)	1.96 (0.96)	2 (1-3)
	Q8 Areas of uncertainty (N=70)	2.66 (1.2)	3 (2-4)
Treatment Options	Q9 How treatment works (N=86)	2.73 (1.39)	3 (1-4)
	Q10 Benefits of treatment (N=86)	2.48 (1.23)	2.5 (1-3)
	Q11 Risk of treatment (N=86)	1.99 (1.22)	1 (1-3)
	Q12 Effects of no treatment (N=86)	2.14 (1.1)	2 (1-3)
	Q13 Effects on quality of life (N=86)	2.2 (1.19)	2 (1-3)
	Q14 All alternatives described (N=85)	2.39 (1.29)	2 (1-3)
	Q15 Shared decision (N=86)	2.1 (1.1)	2 (1-3)
Overall Rating (N=86)		2.23 (1.6)	1 (1-3)

The question about explicit aims scored median of 2 (1-4) and mean of 2.68 (1.39) out of 5. Most of the websites revealed the date of the information with a median of 3 (1-4) and mean of 2.63 (1.27), however a few number of websites have revealed the source of the information a median of 1 (1-1) and mean of 1.51 (1.11) out of 5 (Table 2).

Majority of the websites had given a correct account of how the treatment of oral cancer works a median of 3 (1-4). All alternatives correctly described a median of 2 (1-3). Benefits and risks of treatment were correctly given a median of 2.5 (1-3) and 1 (1-3), respectively. The effects on the quality of life due to oral cancer were mentioned a median of 2 (1-3) while details regarding the Shared decision for oral cancer patients were mentioned a median of 2 (1-3) (Table 2).

The average measure for intra examiner and inter-examiners assessment of DISCERN were at 0.823 and 0.843 (P < 0.001)] respectively. The score between 0.75 and 0.90 demonstrates good reliability (Koo and Li, 2016).

JAMA Benchmarks

The number and percentage of websites per obtained JAMA benchmark are demonstrated in Figure 3. Only 15 (17.4%) of the Arabic websites clarified the author of the presented content, and 11 (12.8%) of the websites mentioned the data sources, such as journals, textbooks and guidelines.

Figure 3.

Shows Evaluation the Quality of the Medical Websites Using the JAMA Benchmarks

More than half of the websites (64%) presented the last date of update and 51 (59%) of the websites disclosed “ownership” of the Web site With regard to the total number obtained in accordance with JAMA benchmarks; four benchmarks were met only in 2 websites (2.3%), three benchmarks met in 7 websites (8%), two benchmarks met in 39 websites (45%), one benchmark was met in 26 websites (30%), and no benchmark was observed in 13 websites (15%) (Supplementary Table 5). Therefore, 13 (15%) of oral cancer related Arabic websites did not fulfill any criteria posed by the JAMA benchmarks and only 2 (2%) of the sites fulfilled the four criteria (Figure 3).

HON

The quality assessment of the 86 identified websites according to HON showed that only 4.7% of the sites achieved high score (≥75) and 37.2% of the websites obtained low score (<50), and 58 % websites gained intermediate score (≥50-<75) (Figure 4).

Figure 4.

Shows Evaluation the Quality of the Medical Websites Using HON

Out of the 86 websites, 4 (4.7%) achieved scores >75% in HON assessment, of which three appeared in the top 10 based on the DISCERN assessment (Supplementary Table 7).

Discussion

According to DISCERN, JAMA benchmarks, and HON, this study showed poor quality of information provided by oral cancer-related Arabic websites. This finding is in line with those reported by other studies about information provided on oral cancer at English, French, and Spanish websites (Saithna et al., 2008; López-Jornet and Camacho-Alonso, 2009; Irwin et al., 2011; Vivien et al., 2017). Patients are most likely to visit sites, which are listed in the first top 10 websites by search engines. Most of these search engines corporate with sponsors’ links, that is, companies pay to be appeared at the top of the list for a particular search keyword, suggesting a bias towards information needed by patients from profit-making and therefore negative effect on the quality of provided data.

In this study, to overcome this limitation, most of profit and non-profit possible websites were included by browsing the first 100 websites and using 6 search engines. A non-profit organization is a group structured for a purpose other than making a profit, such as helping the community is concerned with money only as much as necessary to keep the organization operating. They can take the form of a corporation, partnership, an individual charitable contribution, or unincorporated association. they may include public hospitals and clinics, governmental agencies, political organizations, legal aid societies, volunteer service organizations, professional associations, and research institutes. A profit organization, as opposed to a non-profit one, aims at making money therefore, any Social forums contain any type of businesses whose primary goal is making money was classified as profit social websites and any Social forums including medical sites not contain trade advertisements or any business aim to making money was classified as nonprofit social websites (Frey, 2002).

Website failing to fulfill at least three of four criteria presented by JAMA benchmarks might be considered as doubtful (Silberg et al., 1997). According to the JAMA standards in this analysis, 90% of Arabic oral cancer sites were suspicious, which justifies the inability of 15% of Arab oral cancer sites in this study to ever meet any of the JAMA standards.

About 75% of the sites fulfilled two or three criteria of the JAMA benchmarks which more match the currency domain , and disclosure domain , contrary, the attribution domain achieved the lowest score in our assessment, following by the authorship domain as shown in JAMA benchmarks assessment (Figure 3), indicating that most of the Arabic websites did not mention the references and authorship, which were in line with results obtained by using DISCERN. However, the DISCERN assessment obtained a higher level score on the describing the treatment Q9 and the Explicit date Q5 (Table 2). Neglect to mention the references and authors at most of the Arabic oral cancer-related websites highly affected the quality of these websites. Therefore, it might then negatively influence the patients’ ability to make proper and independent decisions.

In conclusion, findings of this study indicated that information on oral cancer is easily accessible at Arabic websites. However, a huge number of these websites were not adequately qualified for patient education. It is recommended to develop websites based system by which enable to the Arabic websites related to oral cancer know their shortfalls, therefore, improve their quality according to evaluation tools which will ensure finding reliable data from the websites.