How well does the Internet answer patients’ questions about inflammatory bowel disease?

Abstract

BACKGROUND

The Internet is an increasingly important source of health information.

OBJECTIVE

To assess how well common websites answered patients’ questions regarding inflammatory bowel disease (IBD).

METHODS

Thirty websites were identified and evaluated. Based on a previous survey of patient information needs, a comprehensive question list was developed in the three following areas: medical information (seven items), medical treatment (six items) and self-management (eight items). The websites were evaluated for the amount of information they provided to answer each question using two standard measures of information quality – the DISCERN and the Ensuring Quality Information for Patients scales.

RESULTS

Four particularly strong websites, scoring highest (on a scale from 1 to 5) in terms of IBD information, were the Crohn’s and Colitis Foundation of America (mean information score 4.3), About.com (4.2), HealthCentral (3.8) and WebMD (3.8). These websites also scored well on the DISCERN and the Ensuring Quality Information for Patients quality scales. Most websites provided at least adequate information on common symptoms, complications, treatments and what is known (or not known) about the causes of IBD. However, many web-sites did not provide adequate information about prognosis, possible side effects of treatment and risks of developing cancer. Information regarding self-management was covered to a very limited extent.

CONCLUSION

Websites could be strengthened by providing more of the information patients deem to be important, and by more clearly identifying sources of information and the date the information was updated. Most websites would benefit from more attention given to reducing the reading level and improving the organization of material.

The Internet is a resource that is increasingly accessible, fast and comprehensive for a variety of queries. People rely on the Internet as a primary source of information for many items, including their health. Among adults in the United States (US) and Europe, 61% and 52%, respectively, have searched online for health information (1,2). Furthermore, information acquired from the Internet may influence how patients manage their health themselves or with the aid of their doctors (3,4). Fifty-nine per cent of patients recently diagnosed with an illness reported that the information they found online led them to ask a doctor new questions or to get a second opinion (4). As the trend away from paternalistic medicine toward more patient-centred practice continues, it is clear that information patients access through the Internet will affect overall management and patient care.

Inflammatory bowel disease (IBD) is a complex illness that affects many aspects of people’s lives. The multifaceted nature of IBD management coupled with the demanding time constraints faced by physicians makes it difficult to exchange all of the information patients need during time-limited medical appointments. This is particularly true at the time of diagnosis when there are many questions to be answered. These time-limited interactions may have resulted in 86% of those with other chronic conditions using the Internet for information and education (4). The Internet has the potential to provide information quickly and be a useful adjunct to medical consultations. However, to be effective, websites should provide up-to-date, accurate, evidence-based information in an understandable format that addresses the questions that are important to patients. A recent survey (5) conducted in an IBD specialty clinic indicated that the most common sources of IBD information for patients were a gastroenterologist (59%), the Internet (54%) and a primary care physician (54%). Internet use was higher among younger patients, particularly those younger than 40 years of age and those with higher education. The most commonly used websites were associated with health care institutions or public organizations such as the Crohn’s and Colitis Foundation of America (CCFA). Of the 69% who used the Internet to gather information about IBD, 78% indicated that it improved their understanding of the disease, and 77% indicated that it improved their understanding of treatment options.

The goal of the current study was to evaluate the adequacy of information provided by popular websites to answer specific patient questions concerning IBD. Our study complements previous studies (6–8) evaluating Internet information regarding IBD by assessing a broader range of information areas based on questions that patients consider to be important. We identified these questions in a recent survey of patients diagnosed with IBD within two years (9). Rather than using a checklist that simply identified the presence or absence of information in each area, our aim was to evaluate how well the websites dealt with each topic. A second goal was to evaluate, with established measures of quality, the websites patients were likely to visit. Identifying the strengths and weaknesses of existing web-sites may contribute to improving the information available to patients and their families in the future. Furthermore, we can direct health care providers to the current best sites (from both an information and a readability perspective) so they can provide this information to their patients.

METHODS

Website selection

The purpose of the present study was not to identify every possible website with information but, rather, to identify typical websites that would be identified in a search. ‘Google’ was used as the search engine because it is the most widely used search engine in North America and the world by a large margin. Information from specialty industry sources (www.searchengineland.com/comscore-us-most-searches-china-slowest-34217) indicated that Google was used in December 2009 for 67% of searches worldwide, while its closest competitor, ‘Yahoo’, was used for 7%. Figures for North America were similar. On July 24, 2008, a search was conducted using the key words “inflammatory bowel disease”. A refined search was also conducted using the <for patients> link when the list of websites from the first search was retrieved. The first 30 websites that were retrieved through these two searches were considered for inclusion in the study. The first 30 websites of each search were targeted because a recent study (10) of online consumers found that 62% of search engine users click on a search result within the first page of results, and 90% of search engine users click on a result within the first three pages of search results – numbers that have both been increasing over the past few years. Exclusion criteria included duplicates, excerpts from books and articles (not websites), non-IBD-related websites and websites containing too little information to be regarded as a comprehensive resource (defined as less than 500 words). Of the 30 websites retrieved in the <for patients> search, 21 were included: five had too little information, one was about irritable bowel syndrome, one was an abstract and two were links to one of the selected websites. Of the 30 websites retrieved in the general search, eight were included, five were duplicates from the <for patients> search, two were links with one of the selected websites, six had descriptions of books, two were for articles and seven had too little information. An additional website – KidsHealth For Kids – was selected because the teen and parent version had been selected in the other searches and was believed to be helpful in evaluating all three.

In applying these selection rules, it is important to note that the websites identified would vary from one time to another depending on Internet activity and the decision rules applied by the search website. Because the search originated in Canada, two Canadian websites were identified that may not have appeared if the search had originated in the US. The search results would also likely differ somewhat if different search terms, such as “Crohn’s disease” and “ulcerative colitis”, were used. The study was not intended to be an exhaustive evaluation of all websites with information about IBD but, rather, an evaluation of typical information that would be retrieved in a search by a member of the public and, having examined the trends in the websites evaluated, it is most likely that the same issues that were identified would arise across other IBD-related websites also.

Comprehensiveness of IBD information relevant to patients

There are many different types of information about IBD that patients may desire. Based on previous experience and surveys of patient information needs (9), a comprehensive patient question list was developed in the following three areas: medical information concerning IBD (seven items), medical treatment of IBD (six items) and self-management of IBD (eight items). A survey of patients diagnosed with IBD within three to 24 months was administered to patients presenting at six different gastroenterology clinics within the city of Winnipeg, Manitoba. It was found that patients rated information in each of these topic areas as very important (9). The survey sample encompassed a wide range of age and education levels, was balanced in terms of sex and disease type (Crohn’s disease versus ulcerative colitis) and consisted of patients with varying degrees of disease activity. The information from each website addressing each topic (excluding external links) was copied into a word processing file for further evaluation. Information on each topic was assessed on a five-point rating scale ranging from 1 (no information), to 3 (adequate information with basic detail), to 5 (complete and comprehensive information with detailed discussion) (11). The ratings for all of the topics were averaged to create a mean information score for each website. The first rater (SP) was a medical student between his second and third year of training (four-year program). He did not have previous experience with IBD and was considered to have a level of knowledge similar to a well-educated patient with a strong science background. To provide an assessment of the completeness of the information from the perspective of a person knowledgeable in the IBD content area, the material on each topic was also evaluated by a gastroenterology fellow (MT) in postgraduate year 6 to assess the degree of agreement on scoring with a person more experienced with the content area. Interobserver agreement was assessed by comparing the ratings of the two evaluators for each question using the intra-class correlation coefficient (12).

Measures of website quality

Overall, website quality was assessed using the DISCERN (13) and Ensuring Quality Information for Patients (EQIP) (14) scales. DISCERN is a 15-question instrument used to assess the quality of consumer health information concerning treatment choices (written materials including material on the Internet). Each question receives a score of 1 to 5, with 1 indicating no success in meeting the question criteria, 2 to 4 indicating partial meeting of the question criteria, and 5 indicating complete fulfillment of the question criteria. The scale has detailed scoring material available on the Internet (www.discern.org.uk). For each website, the scores of the 15 questions were averaged to produce an overall website score out of 5. The DISCERN scale is widely used (including in the three recent studies of Internet information on IBD [6–8]) and has good psychometric properties (13,15). EQIP is a 20-question tool specifically designed to assess the presentation quality of health care information. The questions cover language, style and design (including figures), with content answered as ‘yes’, ‘partly’ or ‘no’. These responses are converted using a formula to produce a per cent score out of 100.

Readability

The reading level of the text from each website was assessed using the Flesch-Kincaid Grade Level score (16), available in the word processing program ‘Microsoft Word’ (Microsoft Corporation, USA). The Flesch-Kincaid Grade Level score analyzes the relative numbers of syllables, words and sentences in a passage of text to calculate a readability score that indicates the grade level at which a person would be able to understand the text.

RESULTS

The websites reviewed in the present study are presented in Table 1. The search strategy identified a wide variety of web-sites including general sources of information for the public (eg, Wikipedia), general health-related websites (eg, eMedi-cine and WebMD), hospitals and health centres (eg, Cleveland Clinic, Mayo Clinic), a national association (eg, the CCFA) and IBD-specific websites (eg, MayoIBD). Many of the sites focused on adults while others were focused on the needs of children and their parents. The reading grade level of the sites varied considerably with lows in the grade 9 level and highs at the 13 and 14 years of education level, which could be difficult for significant proportions of the population. Given the different areas of focus of the websites, one would not expect that they would all provide as much detail in responding to the information needs of adults with IBD. On the other hand, it would be reasonable to evaluate the sites according to the DISCERN and EQIP criteria, which are designed to apply to any website. The mean information score for each website was strongly influenced by the number of IBD information topics covered by the website (r=0.92, P<0.01). The relationship between the amount of IBD information provided on the web-site and quality measures was evaluated by calculating Pearson correlation coefficients. The mean information score correlated with the DISCERN score (r=0.76, P<0.01) and the EQIP score (r=0.54, P<0.01). The DISCERN and EQIP scores were also correlated with one another (r=0.76, P<0.01). The DISCERN instrument is focused on treatment information while the EQIP scale focuses on broader characteristics of the website including its visual appearance and organization. The assessment of interobserver agreement indicated that evaluators had a high level agreement on the mean information score for each website (r=0.94, P<0.01).

TABLE 1.

Characteristics of the 30 websites evaluated

Internet website	Order in search results*	Reading level, grade	IBD information score, mean	Topics covered, n	DISCERN score†	EQIP score‡
About.com	13	10.4	4.2	21	2.9	55.6
BioBasics	(G8)	10.5	1.4	5	2.3	42.3
The Canadian Encyclopedia	(G14)	12.8	1.3	8	1.7	38.5
Canadian Society of Intestinal Research	(G3)	12.6	2.8	16	2.7	55.3
CedarsSinai	11	10.8	2.2	16	1.9	41.7
CHealth.org	(G17)	10.2	2.8	15	3.5	75.0
ChildrensHospitalBoston	12	11.5	2.1	14	2.2	52.6
CincinnatiChildrens	30	11.4	1.9	14	2.2	60.5
ClevelandClinic	15	12.2	2.7	16	2.9	61.1
Crohn’s & Colitis Foundation of America	16	10.7	4.3	21	4.1	80.6
DrugDigest	19	12.0	2.3	14	3.7	73.7
eMedicine	(G25)	13.5	2.5	16	3.3	52.6
eMedicineHealth	3 (G21)	12.5	2.3	14	2.9	60.5
FamilyDoctor	2 (G20)	9.0	1.7	11	2.6	61.1
HealingWell	29	12.1	3.5	17	3.4	65.8
HealthCentral	28	11.2	3.8	20	3.3	68.4
HealthScout	10	12.7	1.9	10	2.1	52.8
KidsHealth For Kids	§	9.2	1.5	10	2.2	70.6
KidsHealth For Parents	1 (G5)	12.1	1.7	10	2.1	58.8
KidsHealth For Teens	(G30)	12.3	1.7	13	2.0	50.0
LabTestsOnline	20	13.1	1.4	8	2.8	50.0
MayoClinic.com	6	11.0	3.0	15	3.3	63.2
MayoClinic.org¶	25	12.8	2.3	14	2.9	58.3
MedicineNet	8	12.2	3.2	17	3.2	68.4
MedlinePlus	9	10.9	1.9	13	2.3	55.3
MountSinaiHospital	(G11)	11.8	2.4	14	3.1	73.7
RushUniversityMedicalCenter	23	12.3	1.9	13	2.2	50
WebMD	5 (G23)	10.2	3.8	19	3.6	65.8
Wikipedia	(G1)	14.3	2.8	14	3.0	52.6
WomensHealth	4 (G24)	9.4	2.9	16	3.2	73.7

^*Order is from the ‘Google’ “for patients” search when not in brackets or from the ‘Google’ general search when in brackets and with the letter G. Some sites appeared in both searches;

^†The DISCERN score is a rating from 1 to 5 averaged across 15 questions;

^‡The Ensuring Quality Information for Patients (EQIP) score is based on a scale from 0 to 100;

^§The KidsHealth for Kids site was evaluated because the parent and teen versions were identified in the general search;

^¶Two websites from the Mayo Clinic resulted from the patient search and were evaluated independently. IBD Inflammatory bowel disease

A description of the amount of information provided on the websites to answer patients’ questions concerning IBD is provided in Tables 2, 3 and 4. The tables present both the proportion of websites providing no information on a particular topic and the mean score (on a scale of 1 to 5). The mean information score was based on the ratings of the first evaluator (SP) across all 30 websites. Considering medical information regarding IBD (Table 2), most websites provided at least adequate information on the common symptoms of IBD (mean rating of 3.4 on the five-point scale), complications (3.3) and what is known (or not known) about the causes of IBD (3.6). On average, less information was provided on long-term prognosis (2.8), risk of developing cancer (2.7), how IBD or medications used to treat it may affect fertility (2.0), or the risk of children developing IBD (2.6). Considering medical treatments for IBD (Table 3), on average, websites provided adequate information regarding medications and surgery, but limited information on possible side effects of medication and on how to manage symptoms of IBD, with only basic details concerning each of these topics provided. Information that patients judge to be important concerning self-management of IBD (Table 4) was covered to a very limited extent by the websites, with modest amounts of information provided about diet and sources of support in coping with IBD. Many of the websites contained no information about self-management. The intraclass correlation coefficient (the rightmost column of Tables 2, 3 and 4) suggested that there was an acceptable level of agreement between the two raters on each topic area. There was one lower value on the topic of ‘common symptoms of IBD’, with a correlation (r) of 0.50. A comparison of the ratings of the first (SP) and second evaluator (MT) on this topic indicated that their ratings were within one point of one another on 25 of 30 websites and within two points on the remaining five. The low correlation may have been due, in part, to the limited range of ratings on this topic because almost all of the websites achieved a rating of at least 3 (adequate information). Rater MT, a gastroenterology resident, generally rated the sites as having higher amounts of information on the topic of common symptoms of IBD.

TABLE 2.

Medical information regarding inflammatory bowel disease (IBD): Amount of information on each topic across the websites evaluated

Information topic	Proportion of websites with no information (n=30), %	Information score, mean	Intraclass correlation coefficient comparing two raters
Common symptoms of IBD	0	3.4	0.50
Complications that may arise from IBD	7	3.3	0.86
What is known (or not known) about the causes IBD	0	3.6	0.80
Long-term prognosis/outcome of IBD	7	2.8	0.72
Risk of developing cancer	27	2.7	0.91
How IBD or the medications used may affect fertility	57	2.0	0.78
The risk that children of persons with IBD have of developing IBD	20	2.6	0.86

The information score is based on a five-point rating scale ranging from 1 (no information), to 3 (adequate information with basic detail), to 5 (complete and comprehensive information with detailed discussion). All correlations were statistically significant at P<0.01

TABLE 3.

Information regarding medical treatment of inflammatory bowel disease (IBD): Amount of information on each topic across the websites evaluated

Information topic	Proportion of websites with no information (n=30), %	Information score, mean	Intraclass correlation coefficient comparing two raters
Medication treatments for IBD	3	3.7	0.82
Possible side effects of medication treatments	30	2.9	0.83
Surgical treatments that may be required for IBD	3	3.5	0.73
How to manage symptoms of IBD	7	2.7	0.69
How to adjust medications when symptoms cause problems	27	1.9	0.77
When to contact your doctor	20	2.7	0.80

The information score is based on a five-point rating scale ranging from 1 (no information), to 3 (adequate information with basic detail), to 5 (complete and comprehensive information with detailed discussion). All correlations were statistically significant at P<0.01

TABLE 4.

Information regarding self-management of inflammatory bowel disease (IBD): Amount of information on each topic across the websites evaluated

Information topic	Proportion of websites with no information (n=30), %	Information score, mean	Intraclass correlation coefficient comparing two raters
Changes to diet that may be helpful with IBD	10	2.8	0.87
What foods offer the best nutritional value	83	1.2	0.82
What nutritional deficiencies you may be at risk for	43	2.3	0.91
When to use nutritional supplements	60	1.5	0.92
Informing family members about IBD	87	1.2	0.89
How to manage time away from work/school	90	1.3	0.96
Insurance that may be available if sick days run out	90	1.3	1.00
Sources of support in coping with IBD	17	2.6	0.82

The information score is based on a five-point rating scale ranging from 1 (no information), to 3 (adequate information with basic detail), to 5 (complete and comprehensive information with detailed discussion). All correlations were statistically significant at P<0.01

To further consider how website information concerning IBD could be strengthened in the future, in addition to providing more information in response to patients’ questions, it is useful to consider the criteria for the quality scales. Table 5 lists the criteria in the DISCERN scale for written patient information concerning treatments and the average score across the websites. Considering that the highest score is 5, there was room for improvement in almost every area evaluated.

TABLE 5.

Mean scores for the items in the DISCERN scale across the websites evaluated

DISCERN question	DISCERN score, mean
Are the aims clear?	2.60
Does it achieve its aims?	3.71
Is it relevant?	3.30
Is it clear what sources of information were used to compile the publication (other than the author or producer)?	1.80
Is it clear when the information used or reported in the publication was produced?	2.43
Is it balanced and unbiased?	2.70
Does it provide details of additional sources of support and information?	4.10
Does it refer to areas of uncertainty?	2.77
Does it describe how each treatment works?	3.30
Does it describe the benefits of each treatment?	3.63
Does it describe the risks of each treatment?	2.93
Does it describe what would happen if no treatment is used?	1.20
Does it describe how the treatment choices affect overall quality of life?	1.93
Is it clear that there may be more than one possible treatment choice?	2.97
Does it provide support for shared decision making?	2.70

Each question is rated on a five-point scale ranging from 1 (question criterion not fulfilled), to 2 to 4 (criterion fulfilled to some extent), to 5 (criterion completely fulfilled)

The EQIP criteria are described in Table 6. There is considerable overlap between these criteria and those in DISCERN. The EQIP scale indicated that most websites use a respectful tone, present the information logically, and explain complex words and jargon, but they may benefit from using shorter sentences. The majority of sites described treatment options and the benefit of treatment; however, only one-half identified the risks and side effects of treatment, and indicated other treatment options. Many websites indicated the date the document was produced, but fewer indicated the source of the information. It is understandable that few websites provided contact information for health services (as is often the case in written information provided in a specific clinic, for example) because it would not be feasible to provide information about local resources on nationally and internationally used websites. Similarly, on websites that may be accessed by a wide range of people, it is not always appropriate to address readers personally because the reader’s circumstances are unclear.

TABLE 6.

Mean scores for specific items in the Ensuring Quality Information for Patients (EQIP) scale across the websites evaluated

EQIP question	EQIP score, mean
The document mentions which subjects will be covered and covers them	0.70
Use of everyday language, explains complex words or jargon	0.75
Use of short sentences (<15 words on average)	0.18
The document personally addresses the reader	0.50
The tone is respectful	0.98
The design and layout are satisfactory (excluding figures or graphs)	0.83
Figures or graphs are clear and relevant	0.66
Information is presented in a logical order	0.87
The document has a designated space for the reader’s notes	NA
Specific contact details for health services	0.22
Date of issue or revision provided	0.72
Names the persons or organization that produced the document	0.60
The document states if and how patients were involved/consulted in its production	0.0
Contains reference to quality of life issues	0.30
Use of generic names for all medications or products	0.85
Provides specific details of other sources of reliable information/support	0.68
Describes the purpose of treatments	0.77
Describes the benefits of treatments	0.82
Describes the risks and side effects of treatments	0.57
Describes alternatives for treatments	0.50

Each item on the scale was scored on a scale ranging from 0.0 to 1.0, with the values 0 (no), 0.5 (partly) and 1 (yes). The mean was calculated across the 30 websites evaluated. NA Not applicable

DISCUSSION

Four particularly strong websites, which scored highest in terms of IBD information, were the CCFA (mean information score 4.3 out of 5), About.com (4.2), HealthCentral (3.8) and WebMD (3.8). These websites also scored well on the DISCERN and EQIP quality scales. The range of DISCERN scores was similar to that reported in recent studies of IBD websites (7,8).

In considering the reading level determined in our study, there was a broad range in terms of readability scores that were skewed toward a higher grade level. The grade levels in our study were similar to those reported in two recent studies of Internet information on IBD (7,8). It is generally recommended that health information for the public be targeted at a reading level of approximately grade 8 – perhaps even grade 5 – given the low literacy of many people in the community (17). In practice, however, it is difficult to achieve these levels with medical information. Persons with serious literacy problems may struggle to understand written information even when it is at a more basic reading level (18). Developing material at desirable readability levels is complicated by the complex information about IBD that is required, and the use of medical terms that increase the length and difficulty of words. Some of these difficulties may be offset by providing an extensive glossary and illustrations, or videos that can act as easier-to-understand supplements to the written material provided. While it may be difficult to attain grade 8 reading levels, for example, attention to reading level in the continuing development of educational materials is likely to improve quality. The sites with the highest mean IBD information scores tended to have lower reading levels; the one website that focused on children (KidsHealth kids) had a lower reading level.

Tailoring information in a manner that is easily understood by patients (19) is a significant challenge in improving the information available to the public via the Internet. Some types of information are more readily available than others. For example, considering medical information about IBD (Table 2), information regarding common symptoms of IBD, complications that may arise, and long-term outcome or prognosis is available in many resources for patients. On the other hand, information on risk of developing cancer may be less clear and should be communicated to patients in ways that are informative without producing undue anxiety. For example, information about cancer might emphasize the minor risk and the importance of surveillance colonoscopy. Definitive information about the cause of IBD is not available; however, written material for patients might discuss some of the areas that are being considered in the research on causes. Given that much about IBD is still unknown, and research in IBD is a particularly active field, it would be helpful for websites to have a plan for regular reviews of their information and to provide the date of the latest review. This would assure patients that the information they are reading is current. Information regarding the effects on fertility of IBD and the medications used in fertility treatment, and the risk that children of parents with IBD have, may be of particular interest to patients of child-bearing age or with children.

Information concerning the medical treatment of IBD (described in Table 3) may also be challenging to communicate to patients in straightforward ways. Furthermore, it is likely that not all experts would agree on the best recommendations in each of these areas (such as whether medication should be adjusted without consultation with the physician and when to contact the physician). While information on medication side effects may be widely available, patients are often concerned that this information is communicated in ways that are difficult to interpret, such as listing side effects without indicating the proportion of patients who are likely to experience them (19). Similarly, while patients rated many topics concerning self-management of IBD to be very important (Table 4), adequate answers may not be available for all of these questions, and experts may disagree about the best advice to provide.

The description of treatments on many websites would be strengthened by providing additional information concerning the benefits and risks of the common treatments, how they work, treatment choices, how treatments influence quality of life and areas in which there is some uncertainty concerning treatment. It would also be helpful to provide information about the sources of information used in development and when this information was produced.

A limitation of the present study is that the websites identified for evaluation were selected on a particular day and using the specific search term “inflammatory bowel disease”. A search performed one year later, or using alternative search terms such as “Crohn’s disease” and “ulcerative colitis”, would identify different websites. However, we believe that the quality issues identified in the present study apply to a wide range of possible websites, as suggested by a comparison between our study and other research on Internet information. In comparing our findings with those of recent studies of Internet information on IBD (6–8), there were several similarities. A group from The Netherlands (8) evaluated English language websites using a 35-point checklist based on European evidence-based consensus statements concerning the diagnosis and management of IBD. While these websites, many of which were European based, frequently covered definition of disease, signs/symptoms and causes, more than 70% were missing information deemed to be important on a number of both medical and surgical management issues. A team from Dalhousie University (Halifax, Nova Scotia) and the University of Alberta (Edmonton, Alberta) (7) developed a detailed measure of the information provided on Crohn’s disease and ulcerative colitis based on a review of the literature pertaining to patient concerns and the opinions of gastroenterologists. The recent study by this group focused on a detailed analysis of information provided about medication treatments and, as with The Netherlands study, among the websites reviewed, the majority did not provide sufficient information on many aspects of medical management deemed to be important (7). The CCFA and WebMD websites were identified by all of the studies to be in the top six for information content. In spite of the different websites identified, the studies identified similar strengths and weaknesses in the patient-oriented information available on the Internet. Information concerning symptoms, medical treatments and surgical treatments was more often covered adequately. Information concerning prognosis, risk of cancer, medication side effects and management of symptoms was less adequately covered. Our study provided additional information on issues of self-care and quality of life – issues that are particularly important to patients. Many of the websites provided very little information in these areas. All of the studies found large variations in the quality and comprehensiveness of websites. The best quality websites were often not among the first 10 identified by the Internet search engine. Taken together, our study and other recent studies identify a number of areas in which the quality of patient-oriented websites could be improved.

In developing information and decision aids for patients, many approaches have been developed that are likely to be useful. The International Patient Decision Aid Standards Collaboration (20) described criteria that are important in developing patient decision aids, and can also be applied to developing and evaluating patient educational material. Key processes in this area include determining what information patients need and want, developing educational material based on available evidence, arranging for peer review of draft material by patients and professional experts not involved in the development of the educational materials, and field testing of materials by users and practitioners.

When dealing with a health question or concern, the majority of people continue to see a health professional, usually their physician, as a key source of information (1). The use of the Internet as a source of information is steadily increasing, particularly with more frequent access using wireless devices. High-quality, comprehensive, unbiased information on the Internet has the potential to complement the information provided in consultation with health professionals and to enable people to be more active in caring for their health.