Quality of nutritional information on the Internet in health and disease

Abstract

Background:Quality assessment of nutritional information on the internet may prove vital prior to providing public guidance on searching relative information.

Methods:The most popular web sites on four different topics ("Mediterranean diet", "sports nutrition", "nutrition, dysphagia and children" and "herbs and common cold") were assessed with the use of two validated questionnaires (EQIP and DISCERN).

Results:Medical categories produced significantly lower total quality scores when compared to "Mediterranean diet" and "sports nutrition" categories. (F=7.189, P<0.001). Commercial web pages had a significantly lower credibility score compared to institutional and other web page types (H=17.987, P<0.001). Ranking order of each web page was related to its total quality score (P=0.04) but not to its credibility (P=0.241).

Conclusions:Monitoring the accuracy, comprehensiveness and consistency of health-related information on the internet is an important public health issue since there are popular web pages that are regarded as of high quality but they do not always provide reliable information. Health professionals and especially dieticians should provide consumer training on how to search for and appraise nutritional information from the internet.

An increasing number of people use the worldwide web as a source of information. This changing nature of information distribution can have important implications in health care. Several concerns have been raised in regard to the validity and consistency of available information and initiatives relative to the access and quality of health information on the Internet¹. Studies conducted to rate health information quality have been mainly focusing on disease conditions and their medical treatment options. However, a systematic approach to assess the quality of information on nutritional or alternative medicine issues would be necessary, since these issues are rather often neglected by doctors and the public increasingly rely on the Internet for information.

Quality does not necessarily refer to accuracy of information. It constitutes a broader term embracing website several credibility criteria, such as currency of the website, referencing of information and disclaimer, design and aesthetics, navigability and functionality and others. Several types of quality assessment schemes exist, such as lists of principles and quality criteria^2-4; guidance on assessing the presence of these criteria^5,6 site-marking schemes; and technical mechanisms^7-10. Despite the recent efforts to improve the quality of patient information, there is no rigorous method for assessing the quality of web-based information which is applicable to all information types and identifies actions to be taken as a result of the quality assessment^11,12. Thus, different methods or survey questionnaires are necessary in order to increase the credibility of quality assessment allowing useful conclusions. These would be best applied in a systematic way and in a variety of health topics by health professionals accredited in different specialty fields.

Methods

A quality assessment of the most viewed web pages was performed in four different areas of interest. More specifically, two popular Internet search engines (Google and Yahoo) were used to visit the first web sites appearing upon typing the key word phrases "Mediterranean diet", "sports nutrition", "nutrition AND dysphagia AND children" and "herbs AND common cold". The specific subjects were chosen for two reasons. The first is that the authors, as accredited health professionals in different speciality fields, have developed an interest on these issues. The second is that the relevance of these categories is low, allowing a broader spectrum evaluation of health and disease topics. Although Pubmed is probably the first choice for scientists seeking scientific information this is not the fact for lay people or consumers who use other search engines, like Google or Yahoo.

The first twenty Web sites from each search engine were then visited and preliminary assessed for suitability to be rated, since the search term can sometimes be found in isolation on a Web page, for example, in an online dictionary. Those pages with irrelevant content and those that failed to be downloaded were excluded from the study. Web sites were broadly classified in three categories: a) institutional (e.g., government, hospital, or university), b) commercial (e.g., sponsored site or private medical site) and c) the rest that were not in correspondence with the first two categories. Item scores were later analysed for each specific category.

Two survey questionnaires (EQIP and DISCERN) were used for the quality assessments. EQIP (Ensuring Quality Information for Patients) is a 20-item tool which has demonstrated good validity, reliability and utility when used by patient information management and healthcare professionals for a wide variety of written health care information material¹³. Moreover, EQIP exclusively identifies actions to be taken as a result of the quality assessment¹³. A grade scale between 0 and 100% is used with lower grades indicating poorer quality.

DISCERN is a quick and "convenient to use" validated survey instrument applicable to a variety of health information materials^7,14. It consists of 16 questions organized in three separate categories. In the first question group (eight questions), an estimation takes place to evaluate whether or not the material is reliable. In the second set (comprising seven questions) there is an evaluation of the information quality regarding the treatment choices suggested (where applicable). In the third and final part (last question) there is an overall quality evaluation of the information material. Examples of questions included in DISCERN and EQIP surveys can be found in table 3.

Table 3. Items with the higher and lower scores.

The reliability and quality of information is rated through use of a five-point Likert-type scale with lower grades indicating poorer quality. The first part of DISCERN was used for assessing the reliability of the web pages received for all four topics. Appropriate statistical tests (one way ANOVA and ANOVA by ranks) were used to determine possible differences on reliability scores between web pages of each topic area ("Mediterranean diet", "sports nutrition" etc.) and each type ("institutional", "commercial", and "other").

Five individuals (a dietician, a doctor, a biologist and two graduating students from the Department of Nutrition, Technological Institute of Crete) evaluated the related web pages using the aforementioned tools in September 2010. Average scores were obtained for web sites of each topic area and of each type. One way ANOVA was used to assess for differences of total quality scores between these categories. Correlation coefficients between ranking order in Google and Yahoo machines, and scores obtained from EQIP and DISCERN first sections were also produced.

EQIP and DISCERN's items with lower and higher scores as well as URLs with higher EQIP scores are presented.

Finally adherence of the web pages that have been evaluated with EQIP and DISCERN questionnaires to a "code of conduct" is assessed by checking whether a short of HONcode logo⁷ is displayed.

Results

Of a total of 160 web pages obtained, 42 (26%) were duplicates, 7 (4%) failed to download and 12 (8%) were unsuitable for full assessment.

EQIP ratings obtained for Web site content are listed by search category in Figure 1a, whereas Figure 1b shows EQIP instrument ratings of Web site content by Web site type. Both medical categories ("nutrition and dysphagia and children" and "herbs and common cold") had significantly lower EQIP scores when compared to "Mediterranean diet" and "sports nutrition" (F=7.189, P<.001) However, one way ANOVA test failed to reveal any statistically significant difference between the various web types (institutional, commercial and other) based on total EQIP scores (F=0.773, P=.465)

Figure 1. 1a: Total EQIP scores of each category, 1b: Total EQIP scores of each web type, 1c: First part of DISCERN scores in relation to four categories, 1d: First part of DISCERN scores in relation to each web type.

Web site credibility (accuracy, scope, source, relevance and currency of information) is demonstrated for each category and type by Figures 1c and 1d respectively, as assessed by the first part of the DISCERN instrument. Different categories did not present statistically significant differences as regards to the scores obtained from the first part of the DISCERN questionnaire (F=0.240, P=.868). However, one way ANOVA by Ranks showed that commercial web pages presented a significantly lower credibility score compared to institutional and other web pages (H=17.987, P<.001).

Correlation coefficient between web sites' ranking and EQIP and DISCERN's first part scores are shown in table 1. Ranking order of a web page is not statistically related to its credibility as assessed by the first part of the DISCERN survey ranking. On the contrary, ranking order was related to the total quality score as assessed by the EQIP instrument.

Table 1. Correlation coefficient between web sites ranking and EQIP and DISCERN first part scores.

URLs with the highest EQIP scores for each topic area are shown in table 2, whereas table 3 shows the items with significantly higher and lower scores.

Table 2. URLs with higher total EQIP scores.

Regarding adherence to a "code of conduct" only 4 pages displayed a logo that was suggesting a more official evaluation of their content. Although these pages had an above average score in both EQIP and DISCERN questionnaires further analysis was not conducted due to the small number of these pages.

Discussion

There is a wide range of differences between the various quality assessment studies of health Web sites¹⁵. A considerable diversity can be observed as regards to study methods and rigor, quality criteria, study population, and topic chosen. Operational definitions of quality are often inconsistent¹⁶. As a result, the conclusions drawn on the quality of health-related Web sites vary widely. The most frequently applied quality criteria include accuracy, completeness, readability, design, disclosures, and references provided¹⁵. It has been suggested that Internet users should prefer non-commercial (institutional) sites with referencing to scientific publications when searching for drug information¹⁷. These web pages are more reliable and provide information of higher standards. However, as it is shown here, there is space for improvement on comprehension ("Does the document contains easy to understand illustrations, diagrams or photos that are relevant to the subjects it covers?") and straightforwardness ("Does it personally address the reader?"). This is of outmost importance in order the popularity of these sites to be increased.

Initiatives undertaken few years ago to more ethical distribution of information via the internet seem to have loosened lately. The HONcode includes several ethical aspects, such as the author's credentials, the date of the last modification, sources and references, funding and the advertising policy. Still, only very few web pages offering nutritional information report an adherence to a "code of conduct". Thus consumers could not direct their internet searches based on these kinds of logos.

Another interesting finding of this study is the correlation of site ranking (presentation order in web machines) with their quality of information. Although ranking number does not necessarily equal to number of visitors, it consists a good indicator of public preference, since ranking effectively drives the likelihood of particular sites being recognized and visited¹⁶. However, monitoring health information on the Internet for accuracy, completeness, and consistency is still fundamental since there are web pages appearing high in the relevance list while being far from institutional or reliable. Moreover, quality measures such as display of authorship, attribution or references, currency of information, and disclosure did not differ between popular and less popular sites¹⁸.

Most relevant web pages present information in a logical order and have a respectful tone. However, they rarely draw attention to areas of uncertainty or provide any alternative treatment options or suggestions. This is of course expected for commercial web sites but we observe that it is also true for almost all non-institutional web sites.

A limitation of this study is that only the first 20 results from each search engine were used for initial evaluation, as previous searches had found that results appearing lower down in the relevancy lists were often duplications of earlier results¹⁹. It was also presumed that it is unlikely that lay searchers would scroll through pages and pages of results, since people mainly search using simple strategies in a search engine and chose results primarily from the first page of search results¹⁹.

Appropriateness of assessment tools was another consideration. The questionnaires used in this study are validated, convenient to use and allow for useful comparisons and conclusions^7,13. A wide range of tools has been developed to assist site developers in producing good quality sites and consumers in assessing the quality of sites. Rating instruments include codes of conduct, quality labels, user guides, filters, and third party certification. However, the value of these tools is unclear¹⁶. Health promotion and education needs to take into account the variety of consumer skills in both searching for and critically evaluating information.

Health professionals, especially dietitians are in an ideal position to provide consumer training on how to search for nutritional information in the Internet as they frequently advise consumers on nutritional and other health related issues and often come up with reproduced Internet information by patients. However, achievement of this goal requires training of dietitians in order to develop the necessary skills. Furthermore, the impact of such guiding on the way consumers search for Internet-based nutritional information and appraise it, remains practically unknown and therefore would need to be evaluated in the future. Therefore, the development of a health-promotion program designed to help dietitians or other health professionals guide consumers through seeking and appraising Internet-based nutritional information is both necessary and challenging.