Abstract
Objectives To determine the prevalence of false or misleading statements in messages posted by internet cancer support groups and whether these statements were identified as false or misleading and corrected by other participants in subsequent postings.
Design Analysis of content of postings.
Setting Internet cancer support group Breast Cancer Mailing List.
Main outcome measures Number of false or misleading statements posted from 1 January to 23 April 2005 and whether these were identified and corrected by participants in subsequent postings.
Results 10 of 4600 postings (0.22%) were found to be false or misleading. Of these, seven were identified as false or misleading by other participants and corrected within an average of four hours and 33 minutes (maximum, nine hours and nine minutes).
Conclusions Most posted information on breast cancer was accurate. Most false or misleading statements were rapidly corrected by participants in subsequent postings.
Introduction
Nearly half of women recently diagnosed as having breast cancer turned to the internet for information on health.1 Consumers are satisfied with their online experience and are making choices based on the information that they encounter.2,3 In some cases patients may not discuss with clinicians their use of treatments found online.4 On the other hand, clinicians are faced with patients who have been informed (or misinformed) by information posted on the internet. Clinicians, researchers, and healthcare consumers are therefore concerned about the accuracy of online health information.2,5-7
Internet cancer support groups offer the public a forum in which to share experiences, ask questions, and offer advice. Discussions have been categorised according to their content as related to quality of care, treatment, recurrence of disease, and alternative therapy.8 Rates of false or misleading information may vary depending on the definition of falsehood (for example, verified by multiple observers, possibly or definitely false?), forum studied, existence of a moderator, prevalence of health professionals, topic, and other factors. One study found that about 6% of postings to an online epilepsy forum were “objectively inaccurate.”9 A review of a German language brain tumour list found that “Normally, precisely formed questions were answered by medically correct and solid statements. When medical statements were incorrect, other participants did not hesitate to correct or criticise.”10
Professionals rely on the peer review process to screen out false or misleading information. Peer reviewed journals ensure that published articles are critically reviewed by experts. Similarly, open source software relies on the user community for quality control. A comparable peer review process may occur on online forums whereby false or misleading information is corrected quickly and reliably by subsequent postings. This possibility, which may be referred to as the “self correction hypothesis,” has been suggested in the literature.9,11,12 To our knowledge the self correction hypothesis has never been quantitatively tested in an unmoderated forum. We determined the prevalence of false or misleading statements found on a specific internet cancer support group site and whether these statements were identified and corrected by subsequent postings.
Methods
We selected the Breast Cancer Mailing List because it is unmoderated and therefore reflects only the opinion of voluntary participants without any kind of systematic interference from healthcare professionals. The list was formed in 1994 and facilitates communication between people affected by breast cancer. Participants include individuals with breast cancer; their caretakers, family, and friends; and a small number of healthcare professionals. At the time of our analysis, there were about 500 participants to the list, with an average of 310 distinct people uploading messages per month.
A person can join the Breast Cancer Mailing List by sending an email message to the mailing list administrators. After receiving a message of confirmation, the participant then receives all messages (postings) sent to the list's email address. The participant can post to the list using the same address. The list's archives contain all messages (postings) since its inception. At the time of our study the archives contained over 600 megabytes of information in 248 051 text email messages and were publicly available at http://bclist.petebevin.com/
We reviewed all messages posted to the list from 1 January to 23 April 2005. A “post” or “posting” was defined as the entire content of a message sent by a participant to the list. We reviewed the entire posting to identify false or misleading statements. A “thread” was defined as a group of postings under one subject. A thread started with the first posting of a new subject and ended when no more postings occurred in response to that subject. The postings in a thread were in chronological order. We excluded from our study postings that were incomplete at the cut-off date (23 April).
Three independent clinicians reviewed and classified the postings: a general practitioner (AE), a general internist (EVB), and a breast cancer surgeon (FMB). The general practitioner first identified statements that might be false or misleading. Factually incorrect statements and those that were likely to lead a medically naive reader to a false factual conclusion were defined as false and misleading, respectively. For the purposes of this study we did not consider any statement that was phrased as an opinion (I believe that...) or a question to be false or misleading. We did not require an explanation of why the original statement was incorrect to consider the false or misleading statement successfully identified and corrected. However, the correction had to relate to the specific statement that was false or misleading and give the correct facts. We recorded the location of the posting within the thread, the date and time the message was posted, and the number of postings until a participant identified the incorrect information. We considered a false or misleading statement as not identified when we found no evidence of its identification by other participants and the thread was exhausted.
EVB and FMB verified the information in the candidate postings identified by AE and validated or challenged the initial judgment. Reviewers EVB and FMB confirmed or dismissed all possibly false or misleading postings, based on review of relevant literature. Differences were resolved by consensus.
Results
Overall, 4600 postings, organised into 1378 threads, were reviewed. Table 1 shows a summary of the total number of postings reviewed per month. AE identified a total of 32 (0.7%) candidate statements containing false or misleading information in 30 threads. Of these 32, 10 (31%) were confirmed to be false or misleading by the other two reviewers on the basis of the relevant literature and consensus among the three independent reviewers. The statements were posted by seven different participants, none of which identified themselves as health professionals. A list of the statements confirmed as false or misleading is on bmj.com.
Table 1.
Number of threads and postings reviewed
| Variable | January | February | March | April* | Total |
|---|---|---|---|---|---|
| No of threads | 448 | 328 | 359 | 243 | 1378 |
| No of postings | 1487 | 1131 | 1197 | 785 | 4600 |
| Average No of postings per thread | 3.32 | 3.45 | 3.33 | 3.23 | 3.34 |
| No of authors | 308 | 313 | 308 | 313 |
Cut-off point of study was 23 April.
What is already known on this topic
Healthcare consumers search the internet for information on health
Online information affects patient's decisions about treatment
Despite the publication of many quality measures, no validated, usable measures exist that can reliably identify false or misleading information online
What this study adds
Given a sufficiently active forum, participants can identify and correct most false or misleading statements quickly and reliably without requiring professional review
Online forums can police themselves
Table 2 summarises the participants' level of activity before and during the study period. The participants who had posted or corrected false or misleading information were among the historically most active users of the list. On average these nine participants ranked among the top 1.6% of participants based on the number of postings and in the top 9.5% of participants during the study period.
Table 2.
Participants' level of activity before and during study period
|
Participant
|
No of false or misleading postings
|
Historical level of activity (12 430 participants)
|
Level of activity during study (313 participants)
|
|||
|---|---|---|---|---|---|---|
| Produced by participant | Identified by participant | No of postings | Activity ranking | No of postings | Activity ranking | |
| A | 2 | 1 | 1051 | 16 | 51 | 29 |
| B | 2 | 1 | 772 | 30 | 64 | 23 |
| C | 0 | 1 | 272 | 148 | 175 | 4 |
| D | 1 | 0 | 190 | 207 | 18 | 54 |
| E | 2 | 1 | 188 | 208 | 186 | 2 |
| F | 1 | 0 | 128 | 261 | 61 | 25 |
| G | 1 | 0 | 85 | 303 | 48 | 31 |
| H | 0 | 1* | 18 | 370 | 8 | 64 |
| I | 1 | 0 | 5 | 383 | 5 | 67 |
| Mean | 301 | 214 | 68.4 | 33.2 | ||
This single posting corrected three related but distinct false or misleading statements, therefore total is 10 false or misleading statements, seven corrections.
Seven of the 10 false or misleading statements were identified and corrected by six participants, three of whom had also posted false or misleading statements. The average time before a false or misleading statement was identified was 4 hours and 33 minutes from when the posting appeared on the forum (range 2h3min-9h9min). For the false or misleading statements that were identified and corrected, the average number of postings before identification was 2 (range 1-3).
Discussion
Few postings on the internet cancer support group Breast Cancer Mailing List contained false or misleading information. If false or misleading information was posted to the mailing list, it was identified and corrected by a subsequent posting in more than two thirds of cases. Multiple participants posted and corrected false statements, rather than a single expert participant. One individual did, however, post three of the seven corrections that related to a single topic (expiry of drugs).
In contrast to our results, one study found that conclusions drawn by authors on a mailing list for painful hand and arm conditions were rarely questioned.13 This finding was not, however, quantified. One possible explanation for this discrepancy is that our study was carried out 11 years later. In 1994, the internet was relatively new and our findings may reflect a maturing medium in which participants are more likely to critically evaluate information. In addition, because participants may have already experienced phases of the disease, they can provide accurate information.14 Perhaps there is more shared experience among patients with breast cancer than among people affected by limb pain. In addition, most of the false or misleading statements that we identified were not likely to lead to harm.
Our study was limited by the fact that a single reviewer determined the statements that might be false or misleading. We may therefore have missed some false or misleading statements. For this reason we did not require that a statement be factually incorrect and allowed misleading statements, as defined above. Furthermore, reviewers were not blinded to the study hypotheses. An additional limitation of our study is that we analysed a single, albeit large, internet cancer support group. Our findings may not generalise to other online forums. Unlike most previous studies, however, we chose a large, unmoderated list that truly reflects self correction, rather than the knowledge of a moderator. More research is needed to determine if our findings generalise beyond the Breast Cancer Mailing List to other online communities and other health topics.
Ideally consumers would have access to accurate online information without direct professional guidance, so that the limited time they have with clinicians could be used more efficiently. This requires that online resources present accurate information. At this time, no known effective strategies exist to ensure that online information is accurate. Our findings suggest that, given a forum, the internet can police itself.
Supplementary Material
Editorial by Jadad et al
Details of false or misleading statements are on bmj.com
This article was posted on bmj.com on 2 March 2006: http://bmj.com/cgi/doi/10.1136/bmj.38753.524201.7C
Contributors: AE, FM-B, and EVB planned the study, collected the data, and drafted and revised the manuscript. EVB is guarantor.
Funding: Supported in part by US National Library of Medicine (grant No 5K22LM008306: to EVB) and the National Cancer Institute (grant No 1K08CA91895: to FM-B.). All authors are independent of the funding agencies.
Competing interests: None declared.
Ethical approval: Not required.
References
- 1.Satterlund MJ, McCaul KD, Sandgren AK. Information gathering over time by breast cancer patients. J Med Internet Res 2003;5: e15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Fox S, Rainie L. The online health care revolution: how the web helps Americans take better care of themselves. Washington DC: Pew Internet and American Life Project: Online, 2000: 3-7.
- 3.Hope and the media in advanced cancer patients. American Society of Clinical Oncology 36th annual meeting; New Orleans, 2000.
- 4.Eisenberg DM, Kessler RC, Rompay MIV, Kaptchuk TJ, Wilkey SA, Appel S, et al. Perceptions about complementary therapies relative to conventional therapies among adults who use both: results from a national survey. Ann Intern Med 2001;135: 344-51. [DOI] [PubMed] [Google Scholar]
- 5.Biermann JS, Golladay GJ, Greenfield ML, Baker LH. Evaluation of cancer information on the Internet [see comments]. Cancer 1999;86: 381-90. [PubMed] [Google Scholar]
- 6.Jadad AR, Gagliardi A. Rating health information on the internet: navigating to knowledge or to Babel? JAMA 1998;279: 611-4. [DOI] [PubMed] [Google Scholar]
- 7.Price SL, Hersh WR. Filtering web pages for quality indicators: an empirical approach to finding high quality consumer health information on the world wide web. Proc AMIA Symp 1999: 911-5. [PMC free article] [PubMed]
- 8.Weinberg N, Schmale J, Uken J, Wessel K. Online help: cancer patients participate in a computer-mediated support group. Health Soc Work 1996;21: 24-9. [DOI] [PubMed] [Google Scholar]
- 9.Hoch DB, Norris D, Lester LE, Marcus AD. Information exchange in an epilepsy forum on the world wide web. Seizure 1999;8: 30-4. [DOI] [PubMed] [Google Scholar]
- 10.Mursch K, Behnke-Mursch J. Internet-based interaction among brain tumour patients. Analysis of a medical mailing list. Zentralbl Neurochir 2003;64: 71-5. [DOI] [PubMed] [Google Scholar]
- 11.Ferguson T. Heatlh online: how to find health information, support groups and self help communities in cyberspace. Reading, MA: Addison-Wesley, 1996.
- 12.Feenberg AL, Licht JM, Kane KP, Moran K, Smith RA. The online patient meeting. J Neurol Sci 1996;139(Suppl): 129-31. [DOI] [PubMed] [Google Scholar]
- 13.Culver JD, Gerr F, Frumkin H. Medical information on the internet: a study of an electronic bulletin board. J Gen Intern Med 1997;12: 466-70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Till JE. Evaluation of support groups for women with breast cancer: importance of the navigator role. Health Qual Life Outcomes 2003;1: 16. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.