Abstract
As consumer use of the Internet for medical information grows, continuing evaluation of the medical content on the Internet is needed. We evaluated Internet sites describing breast magnetic resonance imaging (MRI), an emerging technology tool in breast cancer diagnosis and screening. We searched Google for sites describing breast MRI and abstracted the affiliation, content, media type, readability, and quality of 90 most popular unique sites. Over half (56%) of the sites were commercially sponsored. The content varied by site and included medical and procedural facts, information about clinical trials, grants and journal articles, as well as human interest stories. Most (82%) sites described potentially beneficial uses of breast MRI, such as further evaluation of newly diagnosed breast cancers (58%); screening women at high risk for breast cancer (54%); evaluation of abnormal breast findings (48%); screening women with dense breasts (48%) or implants (27%); and surveillance for breast cancer recurrences (24%). Approximately half (56%) of the sites described the limitations of breast MRI, most commonly false positive findings (44%) and costs (24%). Website quality, including the display of contact information, sponsorship, currency of information, authorship, and references varied. The reading level was close to high school graduate. Internet sites describing breast MRI were mostly commercially sponsored, more often described the potential beneficial uses of the procedure than its limitations, and were of variable quality and high reading level. With the lack of enforceable standards for display of medical information on the Internet, providers should encourage patients to direct their searches to the most credible sites.
Keywords: breast cancer, Internet, magnetic resonance imaging
Approximately 60% of the US population uses the Internet for health related purposes (1,2); however, the quality of medical information on the Internet is variable (3–6) and often fails to reach quality standards (3). Studies have found breast cancer-related websites to have unbalanced information (7), variable quality (5,6), and popularity that is driven by the information type rather than medical accuracy (4). As emerging technologies may be promoted for nonevidence based uses (8), we sought to evaluate the type and accuracy of information available to the general public on the Internet about magnetic resonance imaging (MRI) of the breast. Breast MRI emerged in the late 1980s (9), and over the last two decades has been evaluated as a screening and diagnostic imaging tool for women with (10–14) and without (15–19) prior breast cancer.
MATERIALS AND METHODS
Data Collection
On January 06, 2006, using http://www.google.com, we searched the Internet for the term “breast MRI.” Our search revealed 2,960,000 links. After abstracting the top 100 sites and reviewing sites 101–200, we found duplicate links, limited content, and often sites that were unrelated or minimally related to breast MRI. We limited our abstraction to the first 100 sites with the goal of simulating a search that may be reasonably conducted by a lay person (20). We printed the information displayed on each page of the sites and abstracted the information onto a standardized abstraction form, developed specifically for this project. The items on the abstraction form were based on our initial review of 20 sites and on earlier published recommendations for website quality (3). An abstractor reviewed all sites (KK) and the study investigator (LN) reviewed 10% of the sites for quality assurance.
The abstraction form included the following variables: site affiliation, content, media type, readability, and website quality. We categorized the specific content described on the sites based on their descriptions of the potential uses for and limitations of breast MRI. We included potential uses if described with or without supporting scientific evidence. The reading level was assessed by analyzing the first page of each site using the Flesch-Kincaid Grade Level Score program (http://office.microsoft.com/en-us/help/HP101485061033.aspx) (Microsoft Word, Version 2003). Website quality was determined according to the five previously described criteria: (3) display of contact information; display of ownership, sponsorship, advertising policies, and/or conflicts of interest; currency of the information or date of its most recent update; display of authorship; and display of references for the information provided.
RESULTS
Among the 100 sites abstracted, eight were repeats and two were no longer available with the link provided. Of the 90 unique sites included in this analysis, 50 (56%) were commercial and 40 (44%) were noncommercial. The 40 noncommercial sites were sponsored by professional medical organizations (n = 17), university/medical centers (n = 16), nonprofit organizations (n = 4), and governmental agencies (n = 3). All 90 sites were devoted to broader health care topics and not restricted to information about breast MRI. Sites presented the following content: medical facts (88%), procedural details (56%), results of clinical trials (45%), information on ongoing clinical trials (17%), links to journal articles or abstracts (11%), human interest stories (5%), descriptions of grant proposals (4%), and other (10%). All 90 sites contained text, while 35 sites also included images related to the material reported, and three included video. The mean Flesch-Kincaid reading level was at approximately 12th grade education (n = 11.6 years, range 8–12).
Most (82%) sites included a description of the potentially beneficial uses for breast MRI, such as following: further evaluation of newly diagnosed breast cancers (58%); screening women at high risk for breast cancer (54%); further evaluation of abnormal breast findings (48%); screening women with dense breast tissue (48%); screening women with breast implants (27%); and surveillance for breast cancer recurrence (24%). Screening in the general population was listed as an appropriate indication for breast MRI in 2% of the sites. Approximately half (56%) of the sites included a description of the potential limitations of breast MRI, such as: false-positive findings (44%); costs (24%); and false-negative findings (i.e., failure to detect invasive breast cancer and/or ductal carcinoma in situ) (20%). A few sites described the potential for anxiety, claustrophobia, and interference by metal objects.
Site quality varied across the sites with respect to displaying contact information (70%), ownership/sponsorship/conflict of interest (70%), currency of information (69%), authorship (43%), and references (27%). We found no significant differences in any of the measures among commercial versus noncommercial sites and in analyses that excluded sites that only described ongoing studies and/or published findings.
DISCUSSION
We found that Internet sites describing breast MRI were mostly commercially sponsored, more often described the potential beneficial uses than limitations, and were of variable quality and high reading level. Most sites described the uses that were consistent with existing medical literature including screening women at high risk for breast cancer (15,16,19), as well as evaluating prior abnormal findings and newly diagnosed breast cancers (11,13,21). Although some sites reported the potential of using MRI to screen for contralateral breast cancer at the time of diagnosis, these data had not yet been published at the time of this review (14). A few sites recommended screening women at average risk for breast cancer, an indication that is not supported by existing data.
Our findings are consistent with prior literature reporting inadequacies in the information reported on the breast cancer-related sites (5–7,22). Jorgensen and Gotzsche (7) reviewed the information available on websites about screening mammography and found that the information presented was unbalanced and more often discussed the pros of screening than the potential harms. Berland et al. (22) evaluated websites describing a number of diseases and found that about 90% of the breast cancer-related content was accurate, although some information was not given sufficient coverage (such as alternative types of breast cancer treatment). Meric (4) reviewed breast cancer websites and found that the type of information presented rather than its quality drove the popularity of such sites, and thus determined the likelihood that a site would be identified in a search. Interestingly, despite the availability of information about breast MRI on sites such as National Cancer Institute and the American Cancer Society, these did not emerge in our search. This finding is supported by a prior study reporting that private websites were twice as likely as public sites to be visited for medical information (23), probably due to methods used by sites to optimize their popularity (4,24).
The mean reading level of the sites reviewed in our study was at the 12th grade education, representing an inappropriately high level given the estimated 5th grade reading level of the general US population (25). As our software had an upper limit (of 12th grade), it is likely that the actual average reading level was college and beyond. This gap between the actual health literacy and the readability of the websites poses a potential problem, and demonstrates that no significant progress has been made since a 2001 study finding that all English-language Web sites were written at a high school reading level (22).
Our study has limitations. Selection bias may have resulted from analyzing the first 100 sites out of a large list of search results. However, as the goal of our study was to evaluate information on the Internet that would be accessible by a lay person, it is unlikely that individuals conducting an Internet search for medical information by using a tool such as Google would extend their search beyond the first few results, and very unlikely that they would individually review more than 100 sites (20). Further, we found that the more distant sites were largely unrelated to breast MRI and/or contained very little substantive information. Another limitation is that while we used the most popular search engine, (26) the information on the Internet is not static. Therefore, it is possible that using other engines or conducting the search on a different day may have produced different results (22). We re-ran the search in May 2007 and found no significant differences, with the exemption of finding a few more links to the National Cancer Institute and the American Cancer Society, as well as links to national media organizations. Most of these new links described the then-recent findings reporting the potential benefits of using breast MRI in the setting of a newly diagnosed breast cancer (14).
In summary, we found that Internet sites describing breast MRI were mostly commercially-sponsored, were more likely to describe its potentially beneficial uses of the procedure than its limitations, and were of variable quality and of high reading level. With ongoing emerging technologies and increased use of the Internet for medical information, it is important that credible medical sites are given preference in searches and that standardized format is used by sites providing medical information. However, with the lack of enforceable standards for display of medical information on the Internet, providers should encourage patients to direct their searches to the most credible sites.
APPENDIX.
Acknowledgments
Grant support for this study was provided by the National Cancer Institute U19 CA0991931 (LN) and K05 CA104699 (JGE).
References
- 1.Hesse BW, Nelson DE, Kreps GL, et al. Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first health information national trends survey. Arch Intern Med. 2005;165:2618–24. doi: 10.1001/archinte.165.22.2618. [DOI] [PubMed] [Google Scholar]
- 2.Diaz JA, Griffith RA, Ng JJ, Reinert SE, Friedmann PD, Moulton AW. Patients’ use of the internet for medical information. J Gen Intern Med. 2002;17:180–5. doi: 10.1046/j.1525-1497.2002.10603.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the internet: Caveant lector et viewor—Let the reader and viewer beware. JAMA. 1997;277:1244–5. [PubMed] [Google Scholar]
- 4.Meric F, Bernstam EV, Mirza NQ, et al. Breast cancer on the world wide web: cross sectional survey of quality information and popularity of websites. BMJ. 2002;324:577–81. doi: 10.1136/bmj.324.7337.577. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hoffman-Goetz L, Clarke JN. Quality of breast cancer sites on the world wide web. Can J Public Health. 2000;91:281–4. doi: 10.1007/BF03404290. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ekman A, Hall P, Litton J. Can we trust information on the internet? —a comparison of interactive cancer risk sites. Cancer Causes Control. 2005;16:765–72. doi: 10.1007/s10552-005-1722-3. [DOI] [PubMed] [Google Scholar]
- 7.Jorgensen KJ, Gotzsche PC. Presentation on websites of possible benefits and harms from screening for breast cancer: cross sectional study. BMJ. 2004;328:148–51. doi: 10.1136/bmj.328.7432.148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Deyo RA, Patrick DL. Hope or Hype: The Obsession With Medical Advances and the High Cost of False Promises. Vol. 1. New York: AMACom Div American Mgmt Association; 2005. [Google Scholar]
- 9.Lee JYM, Morris EA. Breast magnetic resonance imaging: historical overview. In: Morris EA, Liberman L, editors. Breast MRI: Diagnosis and Intervention. New York: Springer; 2005. pp. 3–6. [Google Scholar]
- 10.Morrow M. Magnetic resonance imaging in breast cancer: is seeing always believing? Eur J Cancer. 2005;41:1368–9. doi: 10.1016/j.ejca.2005.03.016. [DOI] [PubMed] [Google Scholar]
- 11.Hylton N. Magnetic resonance imaging of the breast: opportunities to improve breast cancer management. J Clin Oncol. 2005;23:1678–84. doi: 10.1200/JCO.2005.12.002. [DOI] [PubMed] [Google Scholar]
- 12.Kinkel K, Vlastos G. MR Imaging: breast cancer staging and screening. Semin Surg Oncol. 2001;20:187–96. doi: 10.1002/ssu.1033. [DOI] [PubMed] [Google Scholar]
- 13.Orel SJ, Schnall MD. MR Imaging of the breast for the detection, diagnosis, and staging of breast cancer. Radiology. 2001;220:13–30. doi: 10.1148/radiology.220.1.r01jl3113. [DOI] [PubMed] [Google Scholar]
- 14.Lehman CD, Gatsonis C, Kuhl CK, et al. MRI evaluation of the contralateral breast in women with recently diagnosed breast cancer. N Engl J Med. 2007;356:1295–303. doi: 10.1056/NEJMoa065447. [DOI] [PubMed] [Google Scholar]
- 15.MARIBS study group. Screening with magnetic resonance imaging and mammography of a UK population at high familial risk of breast cancer: a prospective multicentre cohort study. Lancet. 2005;365:1769–78. doi: 10.1016/S0140-6736(05)66481-1. [DOI] [PubMed] [Google Scholar]
- 16.Saslow D, Boetes C, Burke W, et al. American Cancer Society guidelines for breast screening with MRI as an adjunct to mammography. CA Cancer J Clin. 2007;57:75–89. doi: 10.3322/canjclin.57.2.75. [DOI] [PubMed] [Google Scholar]
- 17.Warner E, Causer PA. MRI surveillance for hereditary breast-cancer risk. Lancet. 2005;365:1747–9. doi: 10.1016/S0140-6736(05)66520-8. [DOI] [PubMed] [Google Scholar]
- 18.Warner E, Plewes DB, Hill KA, et al. Surveillance of BRCA1 and BRCA2 mutation carriers with magnetic resonance imaging, ultrasound, mammography, and clinical breast examination. JAMA. 2004;292:1317–25. doi: 10.1001/jama.292.11.1317. [DOI] [PubMed] [Google Scholar]
- 19.Kriege M, Brekelmans CT, Boetes C, et al. Efficacy of MRI and mammography for breast cancer screening in women with a familial or generic predisposition. N Engl J Med. 2004;351:427–37. doi: 10.1056/NEJMoa031759. [DOI] [PubMed] [Google Scholar]
- 20.Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002;324:573–6. doi: 10.1136/bmj.324.7337.573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Esserman L, Wolverton D, Hylton N. Magnetic resonance imaging for primary breast cancer management: current role and new applications. Endocr Relat Cancer. 2002;9:141–53. doi: 10.1677/erc.0.0090141. [DOI] [PubMed] [Google Scholar]
- 22.Berland GK, Elliot MN, Morales LS, et al. Health information on the internet: accessibility, quality, and readability in English and Spanish. JAMA. 2001;285:2612–21. doi: 10.1001/jama.285.20.2612. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Miller EA, West DM. Characteristics associated with use of public and private web sites as sources of health care information: results from a national survey. Med Care. 2007;45:245–51. doi: 10.1097/01.mlr.0000244509.60556.49. [DOI] [PubMed] [Google Scholar]
- 24. [accessed 22 January 2009];Search engine criteria. http://www.submitawebsite.com/blog/seo-ranking-tips.html.
- 25.Kirsch IS, Jungeblut A, Jenkins L, Kolstad A. Adult Literacy in America: A First Look at the Findings of the National Adult Literacy Survey. Washington DC: National Center for Education Statistics; 1993. Available at: http://nces.ed.gov/pubsearch/pubinfo.asp?pubid=93275. [Google Scholar]
- 26. [accessed on 05/02/08];Search engine statistics and market share. http://www.submitawebsite.com/search_engine_statistics.htm.