Content and Quality Analysis of Websites as a Patient Resource for Temporomandibular Disorders

Abstract

Objective

The purpose of this study was to evaluate the content and quality of internet information resources in Turkey about temporomandibular disorders (TMDs).

Methods

In July 2020, the keywords “jaw joint disease” (çene eklemi rahatsızlığı) and “jaw joint pain” (çene eklemi ağrısı) were searched on Google, Bing, YAHOO!, and Yandex. The first 20 websites were listed for 2 keywords on the 4 search engines. Scientific articles, product websites, repetitive sites, advertisements, and irrelevant websites were excluded from the list. The remaining 77 websites were assessed using the Quality Criteria for Consumer Health Information (DISCERN), Global Quality Score (GQS) and Journal of American Medical Association (JAMA) benchmarks. The topics related to TMDs that were thought to be important in informing the patient were determined and the Temporomandibular Disorder Content Score (TMDCS) was calculated to evaluate whether these contents were available on the website.

Results

The sources of the 77 websites included dentists in private practice (6.5%, n=5), hospitals/polyclinics (32.5%, n=25), universities (6.5%, n=5), and others (54.5%, n=42). The total DISCERN scores of all websites included were poor (average score 26.96). Mean scores of JAMA, GQS, and TMDCS were 1.75, 2.31, and 8.4, respectively.

Conclusion

The quality and reliability of the information on the websites related to TMDs are poor. Clinicians should be aware that patients may have access to unreliable or incomplete information. There is a need for improvement on websites about TMDs, especially by professionals through imparting more comprehensive and reliable information.

INTRODUCTION

The use of the internet has increased considerably for general purposes and for accessing health care information (1). Advantages such as ease of accessibility, the desire of patients to have more information without going to a healthcare professional, and being less time-consuming and more economical causes a significant increase in the rate of searching for medical information on the internet (2, 3). The information contained in official and reliable sources provides benefits such as directing individuals to the right healthcare professionals and health institutions regarding their health conditions and helping them understand the truth of misunderstood medical information (4–6).

However, incorrect information on some official and unreliable websites increases the level of anxiety about individuals’ health status and negatively affects their decision making. Therefore, concerns about the accuracy and reliability of this information on the internet have led to the development of tools that allow the given information to be evaluated scientifically. To help physicians and patients choose reliable websites providing health-related information, Quality Criteria for Consumer Health Information (DISCERN), Journal of American Medical Association (JAMA) benchmarks, LIDA (Minervation Inc.), and Health on the Verification tools such as Net Code of Conduct (HONcode) have been developed (7–9).

DISCERN, a verification tool, developed by Charnock et al. (9) in 1998 facilitates the production of new, high-quality, evidence-based consumer health information, enabling patients and information providers to evaluate the quality of written information on a subject. Another assessment tool, the JAMA benchmarks, which assess basic quality standards, such as the author, citation, disclosure, and currency of issues for health-related internet information, was published in 1997 by Silberg et al. (10). Similar to the other tools, the Global Quality Score (GQS), which evaluates the content quality of online resources, is a widely used but unconfirmed assessment tool (11).

The temporomandibular joint (TMJ) is one of the most complex joints in the human body. Disorders of the TMJ include problems affecting TMJ components, masticatory muscles, and all masticatory system functions. The first literature on temporomandibular joint disorders (TMDs) was published in 1918 (12). TMDs are defined according to the American Academy of Orofacial Pain as a group of disorders involving masticatory muscles, TMJ, and related structures (13). These disorders may cause symptoms such as tenderness in the masticatory muscles and TMJ, limitation and pain in the mandibular movements, and TMJ sounds, leading people to seek treatment. Trauma, stress, degenerative diseases, overwork of muscles, inflammation, and orthodontic irregularity are among the etiological factors (14–16). Although TMDs are not seen as a social health problem, they affect a significant part of society. An epidemiological study evaluating the prevalence of symptoms of TMJ disorders in the Turkish population, conducted on 1253 individuals, revealed a 31% prevalence of joint pain, an 8.4% prevalence of pain in the opening, and a 27.3% prevalence of joint noise (17). The incidence of TMJ disorders varies between 6% and 93%, depending on the population and clinical criteria (18).

The high rate of TMDs in the population and the wide use of internet sites as the first source for patient health suggests that websites have a critical role in directing patients to the related specialist and/or health institution. In addition, symptoms that are disregarded despite decreasing the quality of life may lead patients to seek answers to their questions on the internet rather than referring them directly to a physician. Therefore, this study aims to evaluate the content and quality of information about TMDs on websites in the Turkish language accessed in Turkey by using up-to-date information evaluation tools.

METHODS

Ethical approval was not required for this study because publicly available data were evaluated. An internet search was made using the 4 of the most popular search engines in Turkey on July 27, 2020: Google (www.google.com), Bing (www.bing.com), Yahoo! (www.yahoo.com), and Yandex (www.yandex.com) (18). The keywords were determined using the Google Trends application. The search setting was based on past “All categories/Turkey/Google web search” and has been limited in the past 5 years to avoid user restrictions and to expand their search results. Some keywords related to the main topic were analyzed using the app. The search was made in the Turkish language. After a comparative keyword search, “jaw joint disease” (çene eklemi rahatsızlığı) and “jaw joint pain” (çene eklemi ağrısı) were identified as keywords for the web search. The first 20 websites listed for 2 keywords on the 4 search engines, and in total, 160 websites were identified and listed. Exclusion criteria were scientific articles, product websites, repetitive sites, advertisements and irrelevant websites. According to the exclusion criteria, 83 websites were excluded from the study list. Two researchers (B.A. and F.Ç.D.), who received training for assessment tools (DISCERN, JAMA, and GQS), evaluated the remaining 77 websites.

Fourteen contents related to TMDs were determined based on the textbooks and guidelines published on this subject to evaluate the quality and sufficiency of the information (18–20). Each content’s presence was scored as 1 point. Then, Temporomandibular Disorder Content Score (TMDCS) was calculated as a total content score for each website with a maximum of 14 points.

DISCERN (Quality Criteria for Consumer Health Information) measurement tool was used for the evaluation of reliability and information quality of written training materials on selected websites (9). The tool consists of 16 questions, each representing a different quality criterion. DISCERN questions are organized into 3 parts. Questions 1 to 8 address the credibility of the publication and help users to decide if they are trustworthy sources on treatment selection. Questions 9 to 15 address specific details of information on treatment alternatives. In this context, Questions 9 to 11 refer to active treatments described in the publication; non-treatment options are addressed separately in Question 12. The scoring made for the 16th question corresponds to the collective quality evaluation of the website. Each question is scored on a scale of 1 to 5.

The information quality of selected websites was also evaluated using criteria known as JAMA benchmarks. The authorship of medical content that should be visible on a website, the display of citations or references, the date of creation and update, and the presence of ownership, sponsorship, advertising policies, or conflicts of interest features were evaluated. For each criterion, “yes” was evaluated as 1 point, and “no” as 0 points.

GQS, which was used to assess the quality of websites, rated using a 5-point scale (Table 1).

Table 1.

Global Quality Score (GQS) description

Score	Description
1	Poor quality, poor flow of the video, most information missing, not at all useful for patients
2	Generally poor quality and poor flow, some information listed but many important topics missing, of very limited use to patients
3	Moderate quality, suboptimal flow, some important information is adequately discussed but others poorly discussed, somewhat useful for patients
4	Generally good quality and flow, most of the relevant information is listed, but some topics not covered, useful for patients
5	Excellent quality and flow, very useful for patients

Statistical Analysis

IBM SPSS version 22.0 (SPSS Inc, Chicago, IL) was used for statistical analysis. The Shapiro-Wilk test was used to confirm the normal distribution of the data. Frequency and descriptive statistics included the number, percentage, and average values. The data collection and analysis were re-performed on randomly selected 35 websites by the same examiner after 2 weeks. Intraexaminer and interexaminer reliability were calculated using intraclass correlation coefficients (ICCs). The correlation between JAMA, GQS, DISCERN, and TMDCS was assessed by calculating Spearman correlation coefficients. Statistical significance was set at p<0.05.

RESULTS

ICC values indicated good intraexaminer repeatability (0.976–0.988) for both observers. The ICC range for examiners 1 and 2 was 0.969 to 1.000 and 0.984 to 1.000, respectively.

From the 160 websites listed, some were excluded because of duplication (75), irrelevance (1) advertisement (2), and no access (5). The sources of the remaining 77 sites were dentists in private practice (6.5%), hospitals/polyclinics (32.5%), universities (6.5%), and others (54.5%). The total DISCERN score of all websites included was poor (average score 26.96). No website reached an excellent or good score. More of the websites scored as poor or very poor (92.2%) (Table 2).

Table 2.

Association between the sources and DISCERN, JAMA, GQS, and TMDCS

	Total (n=77)	Dentist (n=5)	Hospital/polyclinics (n=25)	University (n=5)	Others (n=42)
Total DISCERN score (16–80)
16–26 (very poor)	51	3	17	4	27
27–38 (poor)	20	0	5	1	14
39–50 (fair)	6	2	3	0	1
51–62 (good)	0	0	0	0	0
63–80 (excellent)	0	0	0	0	0
Average DISCERN score	26.96	31.8	26.72	27.2	26.5
Average number of JAMA benchmarks satisfied (0–4)	1.75	2.2	1.48	1.6	1.88
Average GQS (1–5)	2.31	3.2	2.52	2.6	2.1
Avarage TMDCS (0–14)	8.4	9.4	8.28	9	8.29

DISCERN: Quality criteria for consumer health information; GQS: Global quality score; JAMA: Journal of American Medical Association; TMDCS: Temporomandibular disorder content score

The average score per DISCERN question among all websites was displayed in Table 3. When using the keywords “Temporomandibular Disorder” and “Temporomandibular pain,” most of the websites (n=74) were relevant according to DISCERN Question 3.

Table 3.

JAMA benchmarks and distribution of them between the sources

DISCERN questions	Mean score (1–5)
1. Are the aims clear?	1.92
2. Does it achieve its aims?	1.71
3. Is it relevant?	3.12
4. Is it clear what sources of information were used to compile the publication (other than the author or producer)?	1.18
5. Is it clear when the information used or reported in the publication was produced?	1.39
6. Is it balanced and unbiased?	2.25
7. Does it provide details of additional sources of support and information?	1.23
8. Does it refer to areas of uncertainty?	1.26
9. Does it describe how each treatment works?	1.96
10. Does it describe the benefits of each treatment?	1.84
11. Does it describe the risks of each treatment?	1.14
12. Does it describe what would happen if no treatment is used?	1.65
13. Does it describe how treatment choices affect the overall quality of life?	1.35
14. Is it clear that there may be more than one possible treatment choice?	1.87
15. Does it provide support for shared decision making?	1.27
16. Based on the answers to all of the above questions, rate the overall quality of the publication as a source of information about treatment choices.	1.81

DISCERN: Quality criteria for consumer health information; JAMA: Journal of American Medical Association

None of the websites covered all the JAMA benchmarks at once. Although the “disclosure” was the most achieved score, the attribution was the least. JAMA benchmarks and their distribution on websites are shown in Table 4.

Table 4.

JAMA benchmarks and distribution of them between the sources

JAMA Benchmarks	Dentist n	Hospital/ polyclinics n	University n	Others n	Total
					n	Percentage
Authorship	3	5	2	11	22	28.6
Attribution	2	1	0	9	12	15.6
Disclosure	3	25	5	29	62	80.5
Currency	2	6	1	30	39	50.6

JAMA: Journal of American Medical Association

The distribution of TMDCS between the sources was presented in Table 5. The most frequently mentioned subject was “Symptoms-Pain,” whereas the least mentioned subject was “Children/Adolescents.”

Table 5.

Distribution of TMD contents based on the sources

	Dentist (n=5)	Hospital/ polyclinics (n=25)	University (n=5)	Others (n=42)	Total
					n	Percentage
Definition-disorders of joint	4	16	4	23	47	61
Definition-disorders of mastication muscles	3	12	2	9	26	33.8
Anatomy and function of TMJ	2	11	3	17	33	42.9
Etiology-trauma	3	20	5	33	61	79.2
Etiology-Anatomic/systemic/pathologic	3	19	3	33	58	75.3
Etiology-psychologic	3	18	4	31	56	72.7
Diagnosis	3	13	2	20	38	49.4
Symptoms-pain	5	24	5	42	76	98.7
Symptoms-limitation of movement	5	23	5	33	66	85.7
Treatment-education/exercise	5	15	3	32	55	71.4
Treatment-surgical	4	12	4	21	41	53.2
Treatment-non surgical	4	19	4	32	59	76.6
Differential diagnosis	3	5	1	21	30	39
Children/adolescents	0	0	0	1	1	1.3

TMD: Temporomandibular disorder; TMJ: Temporomandibular joint

The correlation analysis of JAMA, DISCERN, TMDCS, and GQS values indicated that there was no significant relationship between JAMA and TMDs, and a very weak and weak relationship with the other 2 parameters. (DISCERN: r=0.238; GQS: r=0.318) The highest correlation was found between JAMA and TMDCS values (r=0.711) (Table 6).

Table 6.

Spearman correlation between DISCERN, JAMA, GQS, and TMDCS

		Total JAMA	GQS	Total DISCERN	Total TMDCS
Total JAMA	r	0.238*	0.318**	0.093
	p		0.038	0.005	0.423
GQS	r	0.238*		0.668**	0.711**
	p	0.038		0.000	0.000
Total DISCERN	r	0.318**	0.668**		0.529**
	p	0.005	0.000		0.000
Total TMDCS	r	0.093	0.711**	0.529**
	p	0.423	0.000	0.000

DISCERN: Quality criteria for consumer health information; GQS: Global quality score; JAMA: Journal of American Medical Association; TMDCS: Temporomandibular disorder content score

DISCUSSION

Nowadays, with rapidly developing technology, the internet is an important part of our lives. This directly increases the information obtained through the internet. Researchers point out that the use of the internet as an important resource for many issues causes the problems of reliability and/or accuracy of the source quoted or read (6). The absence of any control mechanism for all kinds of information published on the internet, as in printed sources, may cause this information to be spread rapidly and uncontrollability (6, 21). This may mislead patients and/or their relatives researching health problems and treatment alternatives. Moreover, awareness of the physicians about the possible knowledge of the patients about the diseases and their treatment choices may improve physician-patient communication. Therefore, we aimed to evaluate the content and quality of the Turkish information about TMDs which the patients in Turkey obtain. We used Google Trends to obtain the most popular keywords in Turkish. After the keywords were identified, the search for the determined keywords was made by a single researcher at the same time, and the links of the listed websites were transferred to a Word document. Thus, algorithms such as localization differences, grammar, and previous search history information have been standardized. As the location of the search may result in a different website list, the reader should keep in mind that the results will change when searching in a different country. Therefore, further studies should be conducted, which are based on the search results of that location.

Aldairy et al. (22) stated that in a regular internet search, patients were less likely to visit more than the first 20 site results displayed on the search engine. Therefore, we evaluated the first 20 websites for each keyword searched in the 4 search engines. Exclusion criteria were determined as scientific articles, product websites, repetitive sites, advertisements, and irrelevant websites. As the present study aims to evaluate the information source that is available for the patients and evaluating the scientific articles requires an almost different study design, we determined the scientific articles as an exclusion criterion. However, no scientific article was found. Moreover, the presentation of the advertisements is variable and independent from the algorithm of the search engine, whereas our purpose was reaching the most visited websites.

Park et al. (23) evaluated the content, quality, accuracy and comprehensiveness of websites related to TMDs. They reported that websites concerning TMDs were poorly organized and maintained. Similarly, Trüp et al. (24) stated that there is a discrepancy between quantity and quality of the available information on TMDs, and in general, there is a lack of evidence-based, high-quality information for patients seeking information related to TMDs on the websites. When we evaluated all the findings of our study in general, we found similar results in our study.

In the present study, we aimed to evaluate the prominent features of 3 different assessment tools. The DISCERN tool was developed to enable patients and information providers to evaluate the quality of written information about treatment options and to facilitate the production of high-quality, evidence-based patient information. DISCERN cannot measure the information accuracy of the content, as the evaluation of accuracy requires acknowledged sources of information. However, matters such as—if conflicting information on the subject is mentioned or not and whether the given references are clear and understandable—provide a detailed scoring (9). In the present study, we determined that the scope of written educational materials provided on the websites evaluated was clear, accessible, relevant, and impartial; however, the sources and the dates of the information were not clearly stated. For the general evaluation of the websites, the total average score of the DISCERN measurement tool, reliability, information quality, and general quality score means revealed moderate scores (9). Universities and hospitals also received poor values. It appeared that the articles on related websites mostly have commercial concerns, away from the academic style. Many websites (66.2%) scored at the lowest level of DISCERN. So, it can be concluded that the content and the quality of information on the websites searched with the keywords “jaw joint disease” and “jaw joint pain” are not sufficient in our country.

Nowadays the prevalence of TMDs is quite high and they affect patients socially and functionally, and it is difficult to differentially diagnose it in terms of localization and symptoms. According to Question 13 (which inquired about the effects of treatment options on the quality of life; the relationship with family, friends, and caregivers; and the effects on daily activities), only 28.6% of the websites mentioned this issue. Kindler et al. (25) reported that depression may be a risk factor in people with joint pain and that depression symptoms were encountered in 49.2% of people with joint pain. It is expected that the improvement and decrease in pain level as a result of TMD treatment will also be effective in reducing depression and somatization of the patients. It should be considered that TMDs have effects on the quality of life and should be mentioned on the websites. However, we observed that the quality of life was not examined adequately on the evaluated websites according to our study.

The JAMA assessment tool aims to critically evaluate the reliability, plausibility, and usefulness of health-related information on the internet. In this context, the authorship of the medical content of the website, citation or reference, date of creation or update, and ownership, sponsorship, advertising policies, or conflict of interest are evaluated comparatively. There was no website meeting all the JAMA benchmarks. It has been found that the biggest deficiency among the criteria evaluating websites was the citation/attribution (15.6%). Similar results have been found in the studies evaluating medical and dental websites using JAMA benchmarks (26–28).

Opinions on the subject evaluated may vary according to different sources of information. Different authors may make different comments on the subjects that are still at the hypothesis stage, or there may be a bias in some cases. To eliminate such problems, reference sources should be specified for the information provided when preparing a website. This issue is questioned with a criterion in both DISCERN and JAMA measurement tools (9, 10). We observed that 84.4% of the websites did not meet the “attribution” criteria of JAMA and the Question 4 of DISCERN. In contrast, websites that gave reference either provided the opinion of a single expert or did not cite the specific sources in the article. Providing reference sources for each new evidence-based information may avoid bias and facilitate access to accurate information.

TMDs were previously assumed to be a condition that only affects adults; however, recent epidemiological studies have reported that the frequency of TMDs’ signs and symptoms among the children is similar to the frequency among the adults. Although children and adolescents rarely complain of any symptoms, there is an increase in the frequency of TMD symptoms throughout life among the children and adolescents having symptoms (20). Frequent parafunctional habits such as bruxism in children are effective in the development of TMDs at older ages. The prevalence of TMDs in children has been reported as 16% in the deciduous dentition and 90% in the mixed dentition period. It has been reported that TMJ sounds increase with permanent dentition from deciduous dentition (29). Although the signs and symptoms among children and adolescences are not rare, TMDs in childhood and adolescence were briefly mentioned on only 1 website. It has been observed that the internet cannot be used as a source of information on this subject that families searched for their children.

It has been revealed that 40% to 75% of the cases in the adult population have symptoms of at least 1 joint dysfunction, and 33% of these cases have dysfunction symptoms such as facial pain and articular pain (30). The symptoms are observed between the ages of 17 and 30 years in the general population, while it is more pronounced between the ages of 20 and 40 years (31). In the present study, movement limitation (85.7%) and pain symptoms (98.7%) were found to be among the most frequently mentioned information contents.

TMDs can be divided into 2 groups as pain-related disorders (myalgia, arthralgia, and headache) and TMJ-related disorders (disc displacement and degenerative diseases) (32).

Pain is the most common symptom of temporomandibular disease (13, 33). In patients with TMDs, pain is observed in the TMJ and chewing muscles. Pain can be caused by musculoskeletal, vascular, neurovascular, neuropathic, psychogenic, and infectious diseases. Therefore, the fact that the pain is caused by TMDs should be confirmed by appropriate imaging methods and clinical examination concomitantly (34). Therefore, the information on the websites is an important element in providing information about possible differential diagnoses, raising awareness of the patients, and directing the patient to the right specialist. However, we found that 39% of the websites had provided information about differential diagnosis.

Although 54.5% of the evaluated websites were nonprofessional resources, (health sites, news sites ext.) the most emphasized content (29.3%) of these websites was “treatment: exercise and education.” Besides, it was observed that less than half of the websites provide information about TMJ anatomy and function and also a definition of disorders related to mastication muscles. It has been observed that there is a need for reliable, up-to-date, and evidence-based information sources about TMDs. The websites managed by professional organizations without any commercial concerns may help inform patients about TMDs correctly.

CONCLUSION

On the basis of the results of this study, the following conclusions were drawn:

1. None of the websites included all the contents, and most of them could not be considered as a patient resource for TMDs.

2. Because of the insufficiency of information, TMD-related websites cannot be a source of information about TMDs in children and adolescents.

3. It was observed that professionals organized or managed fewer websites about TMDs.

4. Professionals should prepare more comprehensive websites and make improvements on existing websites to improve informational resources for patients with TMDs or they should refer more scientific resources to patients.

Main points.

• The quality and content of information on TMDs available on the internet are low.

• The readability of websites about TMDs is generally poor or very poor.

• Reliability is a concern, and patients should interpret most of these sites carefully.