Skip to main content
NCBI home page
The BMJ logoLink to The BMJ
. 2001 Jun 2;322(7298):1366.

Quality of health information about depression on internet

Study's shortcomings may have affected findings

André Tomlin 1,2, Karin Lee Dearness 1,2, John Geddes 1,2
PMCID: PMC1120438  PMID: 11409402

Editor—Griffiths and Christensen evaluated the quality of web based information on the treatment of depression.1 Although they have tried to scrutinise the quality of a select group of websites, there are shortcomings in their methods that may have affected their findings.

Griffiths and Christensen used just one search engine (DirectHit) and one meta-search engine (MetaCrawler), which limited the comprehensiveness of their search. Although the popularity of search engines changes rapidly, we question the choice of DirectHit over more often used engines such as AltaVista or Google.2 Therefore, it is difficult to justify the assumption of Griffiths and Christensen that their search method provided the best available approximation to a list of depression sites.

We also question the strategy of using the single search term of “depression,” given that over 44% of internet users search using multiple keywords.3 The extreme narrowness of using a single keyword search was summed up eloquently by Brian Pinkerton, the founder of MetaCrawler, when he said: “Imagine walking up to a librarian and saying, ‘travel.’ They're going to look at you with a blank face.”4

Griffiths and Christensen attempt to assess the quality of information found by comparison with a guideline of the Agency for Health Care Policy and Research (AHCPR) guideline. Readers should be aware that tools, such as DISCERN (www.discern.org.uk), are readily available for appraising the quality of patient information. Furthermore, although the AHCPR guidelines were developed using an evidence based method, they were published in 1993 and may not necessarily reflect recent advances, alternative treatments, or local practice.

Smarter search engines such as SumSearch (http://sumsearch.uthscsa.edu/) try to refine and guide users through the literature, yet a global initiative from the World Health Organization may provide further direction for users. The WHO has proposed the creation of “.health” as a new internet top level domain to afford some measure of protection for internet users by identifying the holder of the domain name as one who adheres to known standards.5 While the feasibility of “dot health” is being debated, national initiatives, such as the National Electronic Library for Health (www.nelh.nhs.uk) are striving to bring together high quality information for patients and health professionals. One of the initial examples of which is information on depression that can be found at http://cebmh.com/depression.html.

Until initiatives such as dot health or high quality web searching tools become available, the onus will remain with users to search effectively and appraise the quality of the sites retrieved.

References

BMJ. 2001 Jun 2;322(7298):1366.

Level of evidence should be gold standard

S J Darmoni 1,2,3,4, M C Haugh 1,2,3,4, B Lukacs 1,2,3,4, J P Boissel 1,2,3,4

Editor—Several worldwide initiatives have defined criteria for assessing the quality of health information on the internet (box). Over 40 scoring tools are available,1-1 and studies examining the quality of health information available on the internet such as that by Griffiths and Christensen have used clinical guidelines as references.1-2,1-3

Initiatives to assess quality of health information

Health on the Net code (www.hon.ch/HONcode/Conduct.html)

Code of ethics of the Internet Healthcare Coalition (www.ihealthcoalition.org/ethics/ethics.html)

Netscoring (www.chu-rouen.fr/netscoring/)

MedCertain (www.medcertain.org), financed by the European Union

American Medical Association, (http://jama.ama-assn.org/issues/v283n12/pdf/jsc00054.pdf)

Hi-Ethics (www.hiethics.com)

The French health ministry and council of physicians have launched an initiative to define a French code of ethics for health oriented internet applications. One of the four working groups created aims to define criteria to assess the quality of the content of health sites on the internet (as distinguished from the quality of the site itself). The group differentiated information on the sites that is sensitive—for example, concerning the efficacy or toxicity of healthcare interventions—from that which is non-sensitive, such as doctors' addresses.

For sensitive information, the group recommended that an indication of the level of evidence for each piece of information should be the main criterion. This recommendation will not be mandatory for all health sites—for example, a website published by a patients' association, with information from patients and their carers, will not need to apply a level of evidence for the information given. None the less, the level of evidence should be indicated in documents, such as clinical guidelines, reports from consensus conferences, teaching materials, and technical reports when the information concerns the efficacy and toxicity of healthcare interventions.

CISMeF was created in 1995 at Rouen University Hospital, France, to catalogue internet health resources in the French language.1-4 In December 2000, of the 9600 resources catalogued (1914 documents: 589 clinical guidelines, 111 consensus conferences, 337 technical reports, and 664 teaching resources), only 63 (0.7%) indicated the level of evidence (59 clinical guidelines and four consensus conferences).

These results imply that we need to encourage publishers of sensitive health information to indicate the level of evidence for each piece of information. There is no reference method for evaluating the level of evidence, but this is not an excuse for not tackling the problem. With an increasing number of people accessing an increasing amount of health information on the internet, publishers have an ethical obligation to help their readers (health professionals, but more so, lay people). The conclusions of the French working group were that publishers of sites should be encourage to select a simple method (among the existing methods) for indicating the level of evidence for information on their site until a reference method has been validated.

References

  • 1-1.Jadad AR, Gagliardi A. Rating health information on the internet. Navigating to knowledge or to Babel? JAMA. 1998;279:611–614. doi: 10.1001/jama.279.8.611. [DOI] [PubMed] [Google Scholar]
  • 1-2.Griffiths KM, Christensen H. Quality of web based information on treatment of depression: cross sectional survey. BMJ. 2000;321:1511–1515. doi: 10.1136/bmj.321.7275.1511. . (16 December.) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 1-3.Impicciatore P, Pandolfini C, Casella N, Bonat M. Reliability of health information for the public on the world wide web: systematic survey of advice on managing fever in children at home. BMJ. 1997;314:1875–1878. doi: 10.1136/bmj.314.7098.1875. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 1-4.Darmoni SJ, Leroy JP, Thirion B, Baudic F, Douyere M, Piot J. CISMeF: a structured health resource guide. Methods Inf Med. 2000;39:30–35. [PubMed] [Google Scholar]
BMJ. 2001 Jun 2;322(7298):1366.

Authors' reply

Kathleen M Griffiths 1, Helen Christensen 1

Editor—We found that the quality of frequently accessed websites on depression was generally poor, with higher quality evident in sites owned by an organisation or with an editorial board.

Tomlin et al claim that our findings may have been affected by the search engines and search query we used to locate sites. We acknowledge that there are difficulties in being certain of the generality of our findings but doubt that Tomlin et al's suggestions represent improvements in methodology.

The key question is whether the sites we evaluated were representative of sites that are visited by people seeking information on depression. We selected depression sites using both a metasearcher that favoured sites returned by multiple “popular general purpose web search services”2-1 and an engine biased towards popularity that favoured sites known to be frequently accessed. Tomlin et al. advocate the use of AltaVista (which we did access via MetaCrawler) and Google. Google had not been launched at the time of our study but, when accessed recently, produced results overlapping substantially with those from our study.

The single word depression is one of the top 500 most frequently submitted search queries (excluding pornographic queries).2-1 Multiple word queries related to depression are much less frequently used and are narrower in meaning.2-2 (Surprisingly, the single word travel is currently the seventh most popular query.)

Tomlin et al note that the guidelines of the Agency for Health Care Policy and Research (AHCPR) may not reflect recent advances and alternative treatments. We believe that the guidelines remain valid, although they may need to be supplemented. We have recently produced an evidence based review of the effectiveness of medical, psychological, and alternative treatments written at year 8 reading level for the community.2-3 Although there is evidence to support the efficacy of several alternative treatments such as St John's wort and exercise, their inclusion is unlikely to alter substantially our conclusions. Information about alternative treatments on the sites is published elsewhere.2-4

The ACHPR guideline allowed us to directly assess the quality of the content of depression sites. DISCERN and other similar instruments identify criteria that may be indicators of quality. DISCERN and other promising indicators such as the evidence levels proposed by Darmoni et al need to be validated using a content assessment. The next challenge will be to avoid a situation in which authors meet the DISCERN or other criteria but do not provide the best information about effective treatments.

References

  • 2-1.Selberg E, Etzioni O. The MetaCrawler architecture for resource aggregation on the web. IEEE Expert. 1997;12:8–14. [Google Scholar]
  • 2-2.Mindel A, Mindel M. Wordtracker weekly top 500 keyword report. Available at: www.wordtracker.com; accessed 28 April 2000.
  • 2-3.Jorm AF, Christensen H, Griffiths KM, Korten A, Rodgers B. Help for depression: what works (and what doesn't). Canberra: Centre for Mental Health Research; 2001. [Google Scholar]
  • 2-4.Christensen H, Griffiths KM. Sites for depression on the web: a comparison of consumer, professional and commercial sites. Aust NZ J Public Health. 2000;24:396–400. doi: 10.1111/j.1467-842x.2000.tb01601.x. [DOI] [PubMed] [Google Scholar]

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Publishing Group

RESOURCES