In 1997 one of the first studies to evaluate the quality of health information on the internet was published.1 This article assessed the reliability of information for managing fever in children at home and found that the quality of information was poor. Four years after publication these findings were mentioned in 78 journals (from Journal Citation Report), and the message should therefore have reached a wide audience. We investigated the effects of the earlier findings by re-evaluating the quality of the original web pages four years later, as well as that of a more recent sample of pages, using the same methods.
Methods and results
On 28 June 2001 we searched articles through the Institute for Scientific Information's citation index for references to the earlier study.1 We searched for the 41 web pages evaluated in the original study to see if they still existed and if they did whether they had been substituted with new pages or their content had been modified. We compared the content with copies of the original pages, and we noted changes and assigned scores by using the guidelines and scoring system applied in the original study.1 We then repeated the 1997 search for new pages, found 40, and scored them as well.
Overall, 19 of the 41 (46%) original pages still existed. Of these, two had additional information leading, in one case, to a one point increase in completeness score. Five pages had been substituted with new content, leading to the score being increased in two of them (3 to 4 and 2 to 4 points). Three pages scored the same as the original pages (2, 3, and 5 points). Only one of the four pages that had adhered completely to the guidelines (5 points) remained. Thus, the overall improvement in the pages' scores was only 4 points and was not significant with the Wilcoxon signed rank test.
To compare completeness scores, we made the old and new pages more homogeneous by considering pages in English only (table). The average score of the 38 original pages was 2.6 (SD 1.19), whereas that of the 40 new web pages was 3.4 (SD 0.81). The difference was significant (P<0.001) when measured with the Mann-Whitney U test.
Only two (5%) of the new pages adhered to the guidelines for quality of content compared with three (8%) of the original pages. However, 18 (45%) of the new pages compared with only three (8%) of the original pages adhered to most of the guidelines, except for one indication. All of the new pages concurred with paracetamol, dressing the child lightly, giving extra fluids, and when to call the doctor but 13 failed to specify that an antipyretic should be given before a sponge bath to lower the hypothalamic set point. Furthermore, none of the new pages suggested aspirin for children, whereas two of the original pages had (despite possible severe consequences).
Comment
The quality of health information on the internet has improved over the past few years despite concerns over poor quality and its possible consequences.2 The importance of low quality or incomplete information has led researchers to evaluate web pages in different disciplines and to define lists of requirements for users to evaluate the quality of web pages.3–5 Despite these apparently unproductive attempts to amend the situation, it has improved. Monitoring health information on the internet for accuracy, completeness, and consistency is still fundamental.
Table.
Completeness scores for original and new web pages in English. Values are numbers (percentages)
| Completeness score* | 1997 (n=38) | 2001 (n=58) | Originals remaining (n=18) | New entries (n=40) |
|---|---|---|---|---|
| 0 | 1 (3) | — | — | — |
| 1 | 7 (18) | 3 (5) | 3 (17) | — |
| 2 | 7 (18) | 9 (16) | 3 (17) | 6 (15) |
| 3 | 17 (45) | 19 (33) | 5 (28) | 14 (35) |
| 4 | 3 (8) | 23 (40) | 5 (28) | 18 (45) |
| 5 | 3 (8) | 4 (7) | 2 (11) | 2 (5) |
0, did not comply with guidelines for quality; 5, complied with guidelines.
Footnotes
Funding: None.
Competing interests: None declared.
References
- 1.Impicciatore P, Pandolfini C, Casella N, Bonati M. Reliability of health information for the public on the world wide web: systematic survey of advice on managing fever in children at home. BMJ. 1997;314:1875–1879. doi: 10.1136/bmj.314.7098.1875. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Delamothe T. Quality of websites: kitemarking the west wind. BMJ. 2000;321:843–844. doi: 10.1136/bmj.321.7265.843. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Pandolfini C, Impicciatore P, Bonati M. Parents on the web: risks for quality management of cough in children. Pediatrics. 2000;105:e1. doi: 10.1542/peds.105.1.e1. [DOI] [PubMed] [Google Scholar]
- 4.Griffiths KM, Christensen H. Quality of web based information on treatment of depression: cross sectional survey. BMJ. 2000;321:1511–1515. doi: 10.1136/bmj.321.7275.1511. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Risk A, Dzenowagis J. Review of internet health information quality initiatives. J Med Internet Res. 2001;3(4):e28. doi: 10.2196/jmir.3.4.e28. [DOI] [PMC free article] [PubMed] [Google Scholar]