Abstract
Background
The use of stereotactic body radiation therapy (SBRT) for the treatment of stage I lung cancer has been associated with improvements in population-based survival in Western Europe. As patients are increasingly accessing the Internet for information on health-related topics, we evaluated the quality, usability and readability of online patient information about SBRT in three adjacent Western European countries.
Materials and methods
We conducted a web search and analysis between May – June 2011. Thirteen key terms were entered into the Google search engine. We analyzed websites from the Netherlands, Germany and the United Kingdom, by using the DISCERN instrument, a tool designed to assess the quality of health information on treatment choices. In addition, websites’ usability and readability were examined.
Results
We identified a total of 20 websites. None of the websites received an excellent or good quality rating and only two were rated as fair (both from the United Kingdom). DISCERN scores rated 55% (N = 11) of the websites as poor, and 35% (N = 7) as very poor. Dutch websites had the lowest scores. Five websites attained relatively high usability scores, and none had sufficient readability scores.
Conclusions
Despite the fact that SBRT is widely used as a standard therapy in elderly patients with stage I NSCLC in the Netherlands, only limited high quality information is available on the Internet. Improvements in quality information available on the Internet are required in order to increase patient participation in decision-making.
Keywords: Stereotactic body radiation therapy, lung cancer, Internet, quality, usability, readability, decisionmaking, patient participation
INTRODUCTION
The treatment of stage I non-small cell lung cancer (NSCLC) increasingly represents a therapeutic challenge as a growing proportion of patients either are elderly or have significant medical co-morbidity [1-2]. Surgery is commonly considered to be the standard of care, but a large group of older patients do not undergo this treatment. In the Netherlands, for example, surgery is performed in only 43% of Dutch patients aged 75 years and older, as opposed to 79% of younger patients [3]. Until recently, up to 40% of patients who were aged 75 years or older did not receive any curative-intent treatment [4]. The recent introduction of stereotactic body radiation therapy (SBRT) has led to local control rates in excess of 90% being achieved in stage I NSCLC, with only limited high-grade toxicity observed [5-7]. This contrasts with post-surgical 30-day mortality rates of nearly 7% reported in elderly patients [3, 8-9]. The introduction of SBRT in the Netherlands has been associated with survival improvements at a population level [4,10]. This suggests that medical professionals involved in lung cancer treatment are increasingly aware of the merits of this technique. Despite surgery being the standard treatment for operable early stage NSCLC, patients who are at medium or high risk for surgery are increasing electing to undergo SBRT in some countries [11-12].
An increasing number of patients will become eligible for SBRT, and as patients are expected to more actively participate in health decisions, there is a need to ensure that patients have access to reliable information on SBRT [13-15]. The importance of appropriate and understandable information on radiotherapy for cancer patients has been highlighted [16]. Patients who are eligible to undergo SBRT may receive information regarding treatment option from their pulmonologist and/or surgeon and radiation oncologist. However, patients also widely use the Internet as an information source, particularly in the area of cancer treatments [17-19]. Several studies demonstrated that cancer patients who use the Internet to search for information about their disease are better able to cope with their illness and feel better informed and prepared [17, 20]. Consequently, it seems that online information could have beneficial benefits for patients who search for information on treatment of stage I NSCLC.
Some information about both surgery and conventional radiotherapy is already available on the Internet for patients. However, this information does not appear to be systematically developed with the aim of support patients’ decisions and participation, as has been the case for prostate and breast cancer [21]. Anecdotal comments from Dutch patients suggested that the information on SBRT was of limited value in meeting their information needs and supporting their decisions. We undertook the present study to evaluate the quality of online information in selected Western European countries with broadly similar systems for financing, where SBRT is widely available. One was the United Kingdom (UK), where SBRT has been available for about 3 years [22-23]. The other country identified for study was Germany, where SBRT has been available for a period of more than 10 years [24-25]. In the Netherlands, SBRT has been an available treatment option for approximately 8 years.
MATERIALS AND METHODS
Search strategies
This present study was conducted from May to June 2011. We used the search engine Google (www.google. nl, www.google.co.uk, www.google.de) with the following search terms: cancer (kanker; Krebs), lung cancer (longkanker; Lungenkrebs), non-small cell lung cancer (niet-kleincellige longkanker; nicht kleincelligem Lungenkrebs), early stage (vroeg stadium; frühe Stadien), radiotherapy (radiotherapie; Strahlentherapie), radiation (bestraling; Bestrahlung), stereotactic radiotherapy (stereotactische radiotherapie; stereotaktische Strahlentherapie), SBRT (SBRT, SBRT), Gammaknife (Gammaknife), precision radiation (precisiebestraling; Präzisionstrahlung), RapidArc (RapidArc), side-effects (bijwerkingen; Nebenwirkungen) and patients (patiënten; Patienten).
In our search for Dutch-language websites, we employed an additional search on the websites of the 14 hospitals or radiotherapy centers at which SBRT is used to treat primary lung tumors. Eligible websites had to provide information on SBRT. Excluded were websites that were unrelated to SBRT or only provided a list of links to other websites.
Evaluation of the quality of the websites
The quality of the information on the websites was independently assessed by three researchers using the DISCERN tool and handbook [26-27]. Briefly, the DISCERN tool has been designed to assess the quality of consumer health information on treatment choices and has been shown to be an originally reliable and valid instrument [28-31]. The instrument consists of 15 key questions and one overall quality rating question. The first section (questions 1-8) examines the reliability of the information. The second section (questions 9-15) considers the quality of the information on treatment choices. The third section (question 16) consists of an overall quality rating. Each question can be scored from 1 to 5 depending on how well it adheres to the criterion in question. We assigned scores according to the DISCERN marking system (topic addressed=5, partially addressed=3, not addressed=1), and generated a summary score by calculating the each item’s scores (ranges from 15 to 75), with low scores indicating poor quality and high scores indicating good quality. Using the total DISCERN score, we grouped the websites into categories of excellent (63-75), good (51-62), fair (39-50), poor (27-38) and very poor (15-26) [33]. The DISCERN tool is displayed in the appendix.
Evaluation of website usability
The usability of the different websites was assessed using items developed by Cline and Haynes [32]. These design characteristics include: ease of use, logical organisation, links between sites, and aesthetic and format characteristics. We allocated a score ranging from 0 to 7 to each website. The appendix shows the usability questions.
Evaluation of website readability
The readability of the reviewed websites was evaluated by calculating the Flesch Reading Ease scale (FRE) [33]. This scale was originally designed for the English language, but modified formulas were developed for the Dutch and German websites by Brouwer et al. and Amstad et al, respectively [34-35]. We chose the FRE because it has been recognised as a validated method of evaluation used in many studies [36-38]. The FRE uses the length of sentences and the number of polysyllabic words to determine a score from 0 to 100, with lower scores correlating to texts with a higher difficulty [33]. If the text on the website contained less than 300 words, the FRE was calculated for the entire text. In case the text contained over 300 words, three passages, each ranging from 90 to 120 words, were selected from the beginning, middle and end of the text. We edited the text to remove any titles, headings, citations and references and used Microsoft Word 2007 to determine the number of words and sentences. We calculated the readability score using the formulas of the three instruments. See Table 1 for the formulas and scoring index.
Table 1.
Readability
Fresch readability index A = 195 66, 7 · W 2 · Z
The Netherlands (Brouwer 1963) : x = 195 – 66.78*AWL – 2*ASL
Germany (Amstad 1978): x = 180 – ASL AWL*58.5
United Kingdom (Fresch 1948): x = 206.835-1.015*ASL – 84.6*AWL
ASL = average sentence length = number of words / number of sentences
AWL = average word length = number of syllables / number of words
| 90 – 100 | Very easy |
| 80 – 89 | Easy |
| 70 – 79 | Fairly easy |
| 60 – 69 | Standard |
| 50 – 59 | Fairly difficult |
| 30 – 49 | Difficult |
| 0 – 29 | Very confusing |
Data analysis
The quality and usability of the websites were individually assessed by three researchers (WH, NV and CH), following several joint meetings to discuss their findings and a consensus was reached on DISCERN scores and usability scores. In case different scores were given, the researchers accessed the websites together and a final consensus score was derived after discussion. The readability scores were calculated using the different formulas. We entered the data from the quality, usability and readability ratings in Microsoft Excel and analyzed the numbers using SPSS 15.0. Intraclass correlation coefficients (ICC) were calculated for each instrument separately to determine inter-rater reliability between the three researchers.
RESULTS
Search results
We identified a total of 20 websites providing information about SBRT. Ten websites were Dutch, 6 were German and 4 were English. Table 2 gives a short description of each website.
Table 2.
Search results websites
| Name website | Abbr. | URL& last view | Description |
|---|---|---|---|
| Netherlands | |||
| KWF Kankerbestrijding | KWF | www.kwfkankerbestrijding.nl Last view: 28-06-2011 |
General cancer organization; information on lung cancer in general, different treatment options (surgery, radiotherapy and chemotherapy), information on radiotherapy in general and stereotactic radiotherapy. |
| Stichting nationaal fonds tegen kanker | NFK | www.tegenkanker.nl Last view: 28-06-2011 |
Foundation for information about and stimulating of research for regular and alternative treatments; information on lung cancer in general, short description on non-small cell lung cancer, different treatment options mentioned. |
| Kanker wie helpt? | KWH | www.kankerwiehelpt.nl Lat view: 28-06-2011 |
Website of integral Cancer Registration; information on lung cancer, examination methods, NSCLC and SCLC, treatment options, radiotherapy, stereotactic radiotherapy |
| Radiotherapie Nederland | RNL | www.radiotherapienederland.nl Last view: 28-06-2011 |
Website of a radiotherapist (M. van de Pol);information on radiotherapy, radiation, internal and external radiation, stereotactic radiotherapy |
| Radiotherapie Groningen | RGR | www.radiotherapiegroningen.nl Last view: 25-06-2011 |
Website of radiotherapy department University Medical Center Groningen; information on lung cancer in general, treatment options, second opinions, scientific research, NSCLC, SCLC, radiotherapy, stereotactic radiotherapy |
| NKI-AVL | NKI | www.nki.nl Last view: 25-06-2011 |
Website Dutch Cancer Institute; Information on different cancer types, lung cancer, treatment options, radiotherapy |
| VUmc | VMC | www.vumc.nl Last view: 29-06-2011 |
Website VU University Medical Center Amsterdam; information on cancer and radiotherapy purpose of the radiotherapy, explanation of a tumor cell, preparation for treatment, stereotactic radiotherapy |
| Catharina ziekenhuis | CZE | www.cze.nl Last view: 28-06-2011 |
Website Catharina Hospital in Eindhoven; information on radiotherapy, external and internal radiotherapy, stereotactic radiotherapy by lung cancer |
| UMC St Radboud | MCN | www.umcn.nl Last view: 28-06-2011 |
Website St. Radboud University Medical Center in Nijmegen; information on lung cancer in general, complaints and fast diagnostics, radiotherapy, stereotactic radiotherapy. |
| Erasmus MC | EMC | www.erasmusmc.nl Last view: 28-06-2011 |
Website Erasmus University Medical Center; general information on cancer, radiotherapy, treatment options, stereotactic radiotherapy. |
| Germany | |||
| Robert Janker-Klinik | RJK | www.robert-janker-klinik.de Last view: 29-06-2011 |
Website radiotherapeutic institution Robert Janker; information on radiotherapy and stereotactic radiotherapy specific for lung cancer |
| Curado | CUR | www.curado.de Last view: 29-06-2011 |
General German health website; information on all types of diseases, information on lung cancer en different treatment options. |
| Krebsgesellschaft | KGS | www.krebsgesellschaft.de Last view: 28-06-2011 |
Website of the German cancer society; general information on different cancer types and general information on radiotherapy |
| Radonc.med.tumuenchen.de | RAD | www.radonc.med.tumuenchen.de 14 Last view: 27-06-2011 |
Website radiotherapeutic institution of the university of München; general information on different types of radiotherapy |
| Radiologische Allianz | RAL | www.radiologischeallianz.de Last view: 28-06-2011 |
Website German radiotherapeutic society; general information on radiotherapy, stereotactic radiotherapy |
| Deutsches Cyberknife Centrum | DCC | www.deutsches-cyberknife-zentrum. de Last view: 28-06-2011 |
Website German Cyberknife Centrum; information on the cyberknife system and indications for this treatment. |
| United Kingdom | |||
| National lung cancer forum for nurses | NLF | www.nlcfn.org.uk Last view: 27-06-2011 |
Website of the National lung cancer forum for nurses; general information on radiotherapy and stereotactic radiotherapy |
| Cyberknife Centre London | CCL | www.cyberknifecentrelondon.co.uk Last view: 27-06-2011 |
Website of Cyberknife Centre in London; general information on cyberknife, indications for this treatment and patient support. |
| Cancer research UK | CRU | www.cancerresearchuk.org Last view: 27-06-2011 |
Website of English Cancer Research; general information on radiotherapy, stereotactic radiosurgery, specific information on stereotactic radiotherapy for brain tumors. |
| London Lung Cancer Centre | LCC | www.londonlungcancer.co.uk Last view: 27-06-2011 |
Website of London Lung Cancer Centre; general information on lung cancer, treatment options, radiofrequency ablation/cyberknife and stereotactic radiosurgery. |
Quality of websites
Table 3 presents the DISCERN scores for all websites as assessed by the researchers. Of the 20 relevant websites identified, none of the websites received an excellent or good rating (score 51-75). Only 2 websites received a fair rating (score 39-50), namely CRU and NFL, both from the UK, with both a total score of 41. A total of 55 percent of the websites (N = 11) were rated as poor (score 27-38) and 35% (N = 7) as very poor (15-26). The lowest scored websites were NFK, VMC and CCL, respectively.
The mean DISCERN item quality ratings are also outlined in Table 3. The majority of the items were rated under a score of 3 out of 5, indicating a poor quality rating. Especially in the Netherlands, the tool indicated that 13 out of the 15 items were of poor quality; and the main score per item was only 1.8. Only the items “balanced and unbiased information” and “the description of the treatment” had moderate scores overall. The German websites overall had somewhat better scores; the average score per item was 1.9. Especially the items “relevant information”, “the description of the treatment” and “describing the benefits of the treatment” had relatively high scores. The UK websites` overall main score was highest, namely an average of 2.1, with the items “balanced and unbiased information”, “details for additional sources of support” and “information and the description of the treatment” receiving the highest scores. Overall, the Western European websites showed lowest quality on the aspects: “Is it clear what sources of information were used to compile the publication”, “Does it describe what would happen if no treatment is used”, “Does it describe how the treatment choice affect overall quality of life” and “Does it provide support for shared decision-making”.
Table 3.
DISCERN scores
| Site | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | Tot | M |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| The Netherlands | |||||||||||||||||
| KWF | 1 | 0 | 3 | 1 | 1 | 3 | 1 | 1 | 4 | 2 | 1 | 1 | 1 | 2 | 1 | 23 | 1.5 |
| NFK | 1 | 0 | 2 | 1 | 1 | 2 | 1 | 1 | 4 | 1 | 1 | 1 | 1 | 1 | 1 | 19 | 1.3 |
| KWH | 1 | 0 | 2 | 1 | 4 | 3 | 5 | 1 | 4 | 3 | 2 | 1 | 1 | 1 | 1 | 30 | 2.0 |
| RNL | 1 | 0 | 2 | 1 | 4 | 4 | 4 | 1 | 5 | 1 | 1 | 1 | 1 | 1 | 1 | 28 | 1.9 |
| NKI | 1 | 0 | 2 | 1 | 1 | 3 | 5 | 1 | 4 | 1 | 1 | 1 | 1 | 1 | 1 | 24 | 1.6 |
| RGR | 4 | 4 | 4 | 1 | 1 | 3 | 4 | 1 | 5 | 4 | 1 | 1 | 1 | 1 | 1 | 36 | 2.4 |
| VMC | 1 | 0 | 2 | 1 | 1 | 3 | 1 | 1 | 5 | 1 | 1 | 1 | 1 | 1 | 1 | 21 | 1.4 |
| CZE | 2 | 3 | 2 | 1 | 1 | 3 | 1 | 3 | 5 | 4 | 1 | 1 | 1 | 2 | 1 | 31 | 2.1 |
| MCN | 2 | 3 | 3 | 1 | 1 | 3 | 5 | 1 | 4 | 3 | 4 | 1 | 2 | 1 | 1 | 35 | 2.3 |
| EMC | 1 | 0 | 4 | 1 | 2 | 2 | 1 | 1 | 4 | 4 | 1 | 1 | 2 | 4 | 1 | 29 | 1.9 |
| Mean | 1.5 | 1.0 | 2.6 | 1.0 | 1.7 | 2.9 | 2.8 | 1.2 | 4.4 | 2.4 | 1.4 | 1.0 | 1.2 | 1.5 | 1.0 | 24.6 | 1.8 |
| Germany | |||||||||||||||||
| RJK | 1 | 0 | 4 | 1 | 1 | 3 | 1 | 1 | 4 | 4 | 1 | 1 | 1 | 2 | 3 | 28 | 1.9 |
| CUR | 1 | 0 | 2 | 2 | 4 | 3 | 1 | 2 | 4 | 4 | 1 | 1 | 1 | 1 | 1 | 28 | 1.9 |
| KGS | 2 | 4 | 3 | 4 | 5 | 3 | 4 | 1 | 3 | 1 | 1 | 1 | 1 | 1 | 1 | 35 | 2.3 |
| RAD | 1 | 0 | 4 | 1 | 1 | 3 | 1 | 2 | 4 | 4 | 3 | 1 | 1 | 3 | 1 | 30 | 2.0 |
| RAL | 1 | 0 | 3 | 1 | 1 | 3 | 5 | 1 | 4 | 1 | 2 | 1 | 1 | 1 | 1 | 26 | 1.7 |
| DCC | 1 | 0 | 2 | 1 | 1 | 2 | 1 | 1 | 4 | 4 | 2 | 1 | 2 | 4 | 1 | 27 | 1.8 |
| Mean | 1.2 | 0.7 | 3.0 | 1.7 | 2.2 | 2.8 | 2.2 | 1.3 | 3.8 | 3.0 | 1.7 | 1.0 | 1.2 | 2.0 | 1.3 | 29.0 | 1.9 |
| United Kingdom | |||||||||||||||||
| NLF | 5 | 5 | 4 | 1 | 1 | 3 | 5 | 1 | 4 | 2 | 5 | 1 | 2 | 1 | 1 | 41 | 2.7 |
| CCL | 1 | 0 | 2 | 1 | 1 | 3 | 1 | 1 | 3 | 2 | 1 | 1 | 1 | 2 | 1 | 21 | 1.4 |
| CRU | 2 | 2 | 4 | 1 | 1 | 5 | 5 | 4 | 5 | 3 | 3 | 1 | 1 | 1 | 3 | 41 | 2.7 |
| LCC | 1 | 0 | 2 | 1 | 1 | 3 | 1 | 3 | 2 | 1 | 2 | 1 | 1 | 3 | 1 | 23 | 1.5 |
| Mean | 2.3 | 1.8 | 3.0 | 1.0 | 1.0 | 3.5 | 3.0 | 2.3 | 3.5 | 2.0 | 2.8 | 1.0 | 1.3 | 1.8 | 1.5 | 31.5 | 2.1 |
| Total | 1.7 | 1.2 | 2.9 | 1.2 | 1.6 | 3.1 | 2.7 | 1.6 | 3.9 | 2.5 | 2.0 | 1.0 | 1.2 | 1.8 | 1.3 | 28.4 | 1.9 |
Usability
Table 4 presents the usability scores for all websites as assessed by the researchers. Twelve out of the 20 websites appraised were considered always easy to navigate around and 12 were organized in a logical way. Two third of the websites (N = 13) combined text, audio and visual formats and almost all websites (N=17) had colour coordination, lack of clutter and legibility of text. Of concern was that only 4 websites had useful links between sites to help locate specific information.
Table 4.
Accessibility scores
| Website | Q1 | Q2 | Q3 | Q4 | Q5 | Total (%) |
|---|---|---|---|---|---|---|
| The Netherlands | ||||||
| KWF | 2 | 2 | 0 | 0 | 1 | 5 (71) |
| NFK | 2 | 2 | 0 | 0 | 1 | 5 (71) |
| KWH | 2 | 2 | 0 | 0 | 1 | 5 (71) |
| RNL | 2 | 2 | 0 | 1 | 0 | 5 (71) |
| NKI | 1 | 0 | 1 | 1 | 1 | 4 (57) |
| RGR | 1 | 1 | 0 | 1 | 1 | 4 (57) |
| VMC | 1 | 0 | 0 | 1 | 1 | 3 (43) |
| CZE | 1 | 1 | 0 | 1 | 0 | 3 (43) |
| MCN | 1 | 1 | 0 | 0 | 1 | 3 (43) |
| EMC | 2 | 2 | 1 | 0 | 0 | 5 (71) |
| MEAN | 1.5 | 1.3 | 0.2 | 0.5 | 0.7 | 4.2 (60) |
| Germany | ||||||
| RJK | 2 | 2 | 0 | 1 | 1 | 6 (86) |
| CUR | 1 | 2 | 0 | 1 | 1 | 5 (71) |
| KGS | 2 | 2 | 1 | 0 | 1 | 6 (86) |
| RAD | 1 | 1 | 0 | 1 | 1 | 4 (57) |
| RAL | 2 | 2 | 0 | 1 | 1 | 6 (86) |
| DCC | 2 | 1 | 0 | 1 | 1 | 5 (71) |
| MEAN | 1.7 | 1.7 | 0.2 | 0.8 | 1.0 | 5.3 (76) |
| United Kingdom | ||||||
| NLF | 1 | 2 | 0 | 1 | 1 | 5 (71) |
| CCL | 2 | 2 | 0 | 1 | 1 | 6 (86) |
| CRU | 2 | 1 | 1 | 0 | 1 | 5 (71) |
| LCC | 2 | 2 | 0 | 1 | 1 | 6 (86) |
| MEAN | 1.8 | 1.8 | 0.3 | 0.8 | 1.0 | 5.5 (79) |
| TOTAL | 1.6 | 1.5 | 0.2 | 0.7 | 0.9 | 4.8 (69) |
Five out of the 20 websites attained high scores of 6 out of a possible 7. These scores were given to German (RJK, KGS, RAL) and English (CCL and LCC) websites. There was some variation in median score by country. Germany and the UK scored better (5.3 and 5.5, respectively). The range of scores was widest in the Netherlands (mean: 4.2; range 3-5).
Readability
The mean Fresch Reading Ease score for Dutch websites was 34.6 (range 1.94 to 52.3), for English websites 38.3 (range 22.4 to 55.5) and for German websites 36.6 (range 27.0 to 47.6), indicating that all websites contained texts that were difficult to read. None of the websites had a score ≥60, which is the minimum score that indicates standard reading level (see Table 5).
Table 5.
Readability scores
| Website | |
|---|---|
| The Netherlands | |
| KWF | 45.1 |
| NFK | 52.3 |
| KWH | 40.3 |
| RNL | 14.9 |
| NKI | 40.6 |
| RGR | 51.7 |
| VMC | 21.9 |
| CZE | 33.0 |
| MCN | 44.1 |
| EMC | 1.94 |
| MEAN | 34.6 |
| Germany | |
| RJK | 28.9 |
| CUR | 47.6 |
| KGS | 40.0 |
| RAD | 38.5 |
| RAL | 27.0 |
| DCC | 37.5 |
| MEAN | 36.6 |
| United Kingdom | |
| NLF | 46.1 |
| CCL | 22.4 |
| CRU | 55.5 |
| LCC | 29.1 |
| MEAN | 38.3 |
Intraclass correlation coefficient analysis
For the DISCERN rating, there was an ICC agreement of 0.78. For the usability rating an ICC of 0.85 was calculated. Overall, there was a general substantial agreement as the overall ICC determined between the three researchers was 0.81.
DISCUSSION
Principal results
The main aim of our study was to evaluate the quality, usability and readability of information available on SBRT in three Western European countries. After reviewing a total of 20 websites that provided information for patients on SBRT, our main finding was that only limited information was available on SBRT as a curative treatment option in early stage NSCLC. In addition, the information which was available was found to be suboptimal as websites had relatively poor scores for quality, usability and readability.
Our findings are not surprising as previous studies on other cancer types also found low quality health information for patients [39-42]. The findings of our study may partly reflect the fact that that SBRT is a relatively new treatment option for early stage NSCLC. However, the survival benefits observed with outpatient SBRT make it important to ensure that patients have access to comprehensive information to enable to participate in treatment decision-making [4]. It is essential for hospitals and organisations involved in providing patient information on the Internet to frequently re-evaluate the information on the websites, in order to ensure that it is accessible and up-to-date. This will result in more adequate and up-to-date information that is more likely to meet the needs of patients.
Another finding of our study was that the websites with the highest DISCERN scores, often had poor readability scores. This indicates that the websites were too difficult for patients to understand, despite the fact that relatively good quality of information was provided. Specifically, the reading level of analyzed websites was ill-suited to the reading level of a majority of patients, a finding reported previously [30, 43-47]. If information available on websites is too difficult for patients to comprehend, it can specifically exclude readers with low literacy levels, a group who also has less self-management capacities [48-49]. Therefore it is important for persons posting information about cancer and treatment options on the Internet to consider basic literacy skills.
Despite the availability of SBRT in all regions of the Netherlands, only 6 out of the 14 hospitals treating patients with SBRT provided information about this treatment modality on their website. However, these 6 websites were of relatively high quality, in comparison to websites of patient organisations in the Netherlands. In contrast, our findings suggest that the high quality German and British websites were not hospital websites, but rather websites designed to provide health care information for patients. This contrasts with a previous report suggesting that government and academic websites in the UK had higher quality ratings than other types of sites [43]. In addition, most Dutch and German websites provided relatively good quality information on the details of treatment (for example RNL and RJK). This might be caused by the relative long period of time SBRT has been available in the Netherlands and Germany. Another notable finding was that websites in the UK generally had a higher quality compared to both Dutch and German websites. The latter may reflect a priority in the UK to provide high-quality information online for patients.
Limitations
An important limitation is that our study only addresses websites that originated from the Netherlands, Germany and UK, even though many patients may search and access information on a range of international websites. Our search was limited to only three countries with healthcare systems similar to that in the Netherlands. As SBRT utilization is increasing rapidly in countries such as the United States [50], it seems obvious that more websites will provide information to patients. Furthermore, websites investigated in our study may have been modified since the study. Therefore, our findings are likely not completely representative of the present situation. Nevertheless, our study of websites is a unique reflection of the state of the art of information available in Western Europe at a time of change in patterns of care arising from the introduction of SBRT. Periodic review is required in order to ensure that patient information needs are optimally met. Another limitation was that usability was defined as the ease of finding information, but usability for elderly patients or users with disabilities (those who may find it difficult to use the Internet), is not known [51]. In addition, the DISCERN tool evaluates whether or not a website is biased, but does not score the accuracy of information on the website. As patients usually gather information from multiple websites, using the DISCERN tool that only rates individual websites may have limitations. However, the DISCERN tool rates information from a subjective point-of-view, and can effectively discriminate between high-quality and low-quality information without the need of expert knowledge in a particular health area [52]. Despite these limitations, we believe that our study does identify areas for improving the quality of websites.
Conclusions
Our study found that the overall quality, usability and readability of information on SBRT for stage I NSCLC, were poor in all three European countries. Many websites evaluated did not confirm to the standards as defined in the context of decision-making and patient participation [21], our findings suggest that major improvements are needed in websites dedicated to providing lung cancer information to ensure that patient information is complete, transparent and user-friendly for the increasing number of patients accessing the Internet to search for information on SBRT as a treatment option for early stage NSCLC. As more patients seek to be involved in decisions regarding treatment options for early stage lung cancer, it is crucial to provide good quality websites on SBRT to facilitate decision-making by patients. In addition, it is necessary to periodically re-evaluate the content of the online information provided to patients.
ACKNOWLEDGEMENTS
None
APPENDIX
DISCERN Instrument
SECTION 1
1. Are the aims clear?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
2. Does it achieve its aims?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
3. Is it relevant? For patients with lung cancer searching for information on stereotactic radiotherapy
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
4. Is it clear what sources of information were used to compile the publication (other than the author or producer)?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
5. Is it clear when the information used or reported in the publication was produced?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
6. Is it balanced and unbiased?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
7. Does it provide details of additional sources of support and information?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
8. Does it refer to areas of uncertainty?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
SECTION 2
9. Does it describe how each treatment works?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
10. Does it describe the benefits of each treatment?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
11. Does it describe the risks of each treatment?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
12. Does it describe what would happen if no treatment is used?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
13. Does it describe how the treatment choices affect overall quality of life?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
14. Is it clear that there may be more than one possible treatment choice?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
15. Does it provide support for shared decision-making?
| No | Partially | Yes | ||
| 1 | 2 | 3 | 4 | 5 |
SECTION 3
16. Based on the answers to all of the above questions, rate the overall quality of the publication as a source of information about treatment choices.
| Low | Moderate | High | ||
| Serious or extensive shortcomings | Potentially important but not serious shortcomings | Minimal shortcomings | ||
| 1 | 2 | 3 | 4 | 5 |
Accessibility variables (Cline and Haynes, 2001)
1. Does the website facilitate navigation through large quantities of information while maintaining simplicity of technology, operation and format?
[ ] Yes always (2)
[ ] Yes sometimes (1)
[ ] No (0)
2. Does the website have logical organisation? (essential for locating information quickly)
[ ] Yes always (2)
[ ] Yes some (1)
[ ] No (0)
3. Are there useful links between sites to help locate specific information?
[ ] Yes (1)
[ ] No (0)
4. Does the website combined text, audio and visual formats to adapt to user preference?
[ ] Yes (1)
[ ] No (0)
5. Is there colour coordination, lack of clutter and legibility of text?
[ ] Yes (1)
[ ] No (0)
REFERENCES
- 1. Sigel K, Bonomi M, Packer S, Wisnevsky J. Effect of age on survival of clinical stage I non-small-cell lung cancer. Ann Surg Oncol 2009; 16:1912-17. [DOI] [PubMed] [Google Scholar]
- 2. Battafarano RJ, Picirrillo JF, Meyers BF, Hsu HS, Guthrie TJ, Cooper JD, Patterson GA. Impact of comorbidity on survival after surgical resection in patients with stage I non-small cell lung cancer. J Thorac Cardiovasc Surg 2002; 123:280-87. [DOI] [PubMed] [Google Scholar]
- 3. Wouters MW, Siesling S, Jansen-Landheer ML, Elferink MA, Belderbos J, Coebergh JW, Schramel FM. Variation in treatment and outcome in patients with non-small cell lung cancer by region, hospital and volume in the Netherlands. Eur J Surg Oncol 2010; 36:S83-92. [DOI] [PubMed] [Google Scholar]
- 4. Palma D, Visser O, Lagerwaard FJ, Belderbos J, Slotman BJ, Senan S. The impact of introducing stereotactic lung radiotherapy for elderly patients with stage I NSCLC: A population-based time-trend analysis. J Clin Oncol 201-2010; 28:5153-59. [DOI] [PubMed] [Google Scholar]
- 5. Baumann P, Nyman J, Hoyer M, Gagliardi G, Lax I, Wennberg B, Drugge N, Ekberg L, Friesland S, Johansson KA, Lund JS, Morhed E, Nilsson K, Levin N, Paludan M, Sederholm C, Traberg A, Wittgren L, Lewensohn R. Stereotactic body radiotherapy for medically inoperable patients with stage I non small cell lung cancer – a first report of toxicity related to COPD/CVD in a non randomized prospective phase II study. Radiother Oncol 2008; 88:359-67. [DOI] [PubMed] [Google Scholar]
- 6. Timmerman R, Paulus R, Galvin J, Michalski J, Straube W, Bradley J, Fakiris A, Bezjak A, Videtic G, Johnstone D, Fowler J, Gore E, Choy H. Stereotactic body radiation therapy for inoperable early stage lung cancer. JAMA 2010; 303:1070-76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Lagerwaard FJ, Haasbeek CJA, Smit EF, Slotman BJ, Senan S. Outcomes of risk-adapted fractioned stereotactic radiotherapy for stage I non-small cell lung cancer. Int J Radiat Oncol Biol Phys 2008; 70:685-92. [DOI] [PubMed] [Google Scholar]
- 8. Finlayson E, Zhaohui F, Birkmeyer JD. Outcomes in octogenarians undergoing high-risk cancer operation: a national study. J Am Coll Surg 2007; 205:729-34. [DOI] [PubMed] [Google Scholar]
- 9. Rivera C, Dahan M, Bernard A, Falcoz PE, Thomas P. Surgical treatment of lung cancer in the octogenarians: results of a nationwide audit. Eur J Cardiothorac Surg 2010; 39:981-86. [DOI] [PubMed] [Google Scholar]
- 10. Haasbeek CJ, Visser O, Palma D, Lagerwaard FJ, Slotman BJ, Senan S. Improvements in survival of elderly patients with stage I NSCLC in the Netherlands between 2003-2009. Ann Oncol, 2011; in press. [DOI] [PubMed] [Google Scholar]
- 11. Onishi H, Shirato H, Nagata Y, Hiraoka M, Fujino M, Gomi K, Karasawa K, Niibe Y, Takai Y, Kimura T, Takeda A, Ouchi A, Hareyama M, Kokubo M, Kozuka T, Arimoto T, Hara R, Itami J, Araki T. Stereotactic Body Radiotherapy (SBRT) for Operable Stage I Non-Small-Cell Lung Cancer: Can SBRT Be Comparable to Surgery. Int J Radiat Oncol Biol Phys 2011; 81(5):1352-8. [DOI] [PubMed] [Google Scholar]
- 12. Lagerwaard FJ, Verstegen NE, Haasbeek CJ, Slotman BJ, Smit EF, Paul MA, Senan S. Outcomes of stereotactic ablative radiotherapy (SABR) in potentially operable patients with stage I non-small cell lung cancer. Int J Radiat Oncol Biol Phys. Epub ahead of print. [DOI] [PubMed] [Google Scholar]
- 13. Coulter A, Elwyn G. What do patients want from high-quality general practice and how do we involve them in improvement? Br J Gen Pract 2002; 52:S22-26. [PMC free article] [PubMed] [Google Scholar]
- 14. Stacey D, Samant R, Bennett C. Decision making in oncology: a review of patient decision aids to support patient participation. CA Cancer J Clin 2008; 58:293-304. [DOI] [PubMed] [Google Scholar]
- 15. Joosten EA, DeFuentes-Merillas L, de Weert GH, Sensky T, van der Staak CP, de Jong CA. Systematic review of the effects of shared decision-making on patient satisfaction, treatment adherence and health status. Psychother Psychosom 2008; 77:219-26. [DOI] [PubMed] [Google Scholar]
- 16. Nijman JL, Sixma H, van Triest B, Keus RB, Hendriks M. The quality of radiation care: The results of focus group interviews and concept mapping to explore the patient’s perspective. Radiother Oncol. 2011; Epub ahead of print. [DOI] [PubMed] [Google Scholar]
- 17. Eysenbach G. The impact of the Internet on cancer outcomes. CA Cancer J Clin 2003; 53:356-71. [DOI] [PubMed] [Google Scholar]
- 18. Heft PR, Eckles RE, Johnson-Calley CS, Daugherty CK. Use of the Internet to obtain cancer information among cancer patients at an urban county hospital. J Clin Oncol 2005; 23:4954-62. [DOI] [PubMed] [Google Scholar]
- 19. Lee CJ, Gray SW, Lewis N. Internet use leads cancer patients to be active health care consumers. Patient Educ Couns 2010; 81:S63-69. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Rice RE. Influences, usage, and outcomes of Internet health information searching: multivariate results from the Pew surveys. Int J Med Inform 2006; 75:8-28. [DOI] [PubMed] [Google Scholar]
- 21.IPDAS: Criteria for judging the quality of patient decision aids (2005). 28-06-2011, from http://ipdas.ohri.ca.
- 22. Franks KN, Jain P, Dahele M. Response on behalf of the UK Stereotactic Body Radiotherapy Consortium to Brock et al. Clin Oncol (R Coll Radiol) 2009; 21:731-32. [DOI] [PubMed] [Google Scholar]
- 23. Needham A, Hutton D, Baker A. The introduction of lung stereotactic body radiotherapy in the UK… it’s now a reality! J Radiother Pract First View 2011; 1-9. [Google Scholar]
- 24. Wulf J, Haedinger U, Oppitz U, Thiele W, Mueller G, Flentje M. Stereotactic radiotherapy for primary lung cancer and pulmonary metastases: a noninvasive treatment approach in medically inoperable patients. Int J Radiat Oncol Biol Phys 2004; 60:186-96. [DOI] [PubMed] [Google Scholar]
- 25. Zimmermann FB, Heinitz H, Schill S, Grosu A, Schratzenstaller U, Molls M, Jeremic B. Stereotactic hypofractionated radiation therapy for stage I non-small cell lung cancer. Lung Cancer 2005; 48:107-14. [DOI] [PubMed] [Google Scholar]
- 26.European Commission – Europe 2002: quality criteria for health related websites. Brussels; 2002. [Google Scholar]
- 27. Charnock D. The DISCERN Handbook. Abingdon, Oxford: Radcliffe Medical Press; 1998. [Google Scholar]
- 28. Griffiths KM, Christensen H. Website quality indicators for consumers. J Med Internet Res. 2005; 7:e55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Ademiluyi G, Rees CE, Sheard CE. Evaluating the reliability and validity of three tools to assess the quality of health information on the Internet. Patient Educ Couns 2003; 50:151-55. [DOI] [PubMed] [Google Scholar]
- 30. Lau L, Hargrave DR, Bartels U, Esquembre C, Bouffet E. Childhood brain tumour information on the Internet in the Chinese language. Childs Nerv Syst 2006; 22:346-51. [DOI] [PubMed] [Google Scholar]
- 31. Aldairy T, Laverick S, McIntyre GT. Orthognathic surgery: is patient information on the Internet valid? Eur J Orthod. 2011; doi:http://dx.doi.org/10.1093/ejo/cjr04. [DOI] [PubMed] [Google Scholar]
- 32. Cline RJ, Haynes KM. Consumer health information seeking on the Internet: the state of the art. Health Educ Res 2001; 16:671-92. [DOI] [PubMed] [Google Scholar]
- 33. Flesch R. A new readability yardstick. J Appl Psychol 1948; 32:221-33. [DOI] [PubMed] [Google Scholar]
- 34. Brouwer RHM. Onderzoek naar de leesmoeilijkheden van Nederlands proza. Pedagogische Studiën 1963; 40:454–64. [Google Scholar]
- 35. Amstad T. Wie verständlich sind unsere Zeitungen? Dissertation, University of Zürich; 1978. [Google Scholar]
- 36. Stinson JN, Tucker L, Huber A, Harris H, Lin C, Cohen L, Gill N, Lukas-Bretzler J, Proulx L, Prowten D. Surfing for juvenile idiopathic arthritis: perspectives on quality and content of information on the Internet. J Rheumatol 2009; 36:1755-62. [DOI] [PubMed] [Google Scholar]
- 37. Stinson JN, White M, Breakey V, Chong AL, Mak I, Low KK, Low AK. Perspectives on quality and content of information on the Internet for adolescents with cancer. Pediatr Blood Cance. 2011; 15;57:97-104. [DOI] [PubMed] [Google Scholar]
- 38. Ahmed OH, Sullivan SJ, Schneiders AG, McCrory PR. Concussion information online: evaluation of information quality, content and readability of concussion-related websites. Br J Sports Med 2011; doi:10.1136/bjsm.2010.081620. [DOI] [PubMed] [Google Scholar]
- 39. Nilsson-Ihrfelt E, Fjällskog ML, Blomqvist C, Ahlgren J, Edlund P, Hansen J, Malmberg L, Villman K, Andersson G. Breast cancer on the Internet: the quality of Swedish breast cancer websites. Breast 2004; 13:376-82. [DOI] [PubMed] [Google Scholar]
- 40. Al-Bahrani A, Plusa S. The quality of patient-oriented Internet information on colorectal cancer. Colorectal Dis 2004; 6:323-26. [DOI] [PubMed] [Google Scholar]
- 41. Selman TJ, Prakash T, Khan KS. Quality of health information for cervical cancer treatment on the Internet. BMC woman`s health 2006; 6:9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Ream E, Blows E, Scanlon K, Richardson A. An investigation of the quality of breast cancer information provided on the Internet by voluntary organizations in Great Britain. Patient Educ Couns 2009; 76:10-15. [DOI] [PubMed] [Google Scholar]
- 43. Ansani NT, Vogt M, Henderson BA, Mck aveney TP, Weber RJ, Smith RB, Burda M, Kwoh CK, Osial TA, Starz T. Quality of arthritis information on the Internet. Am J Health Syst Pharm 2005; 62:1184-89. [DOI] [PubMed] [Google Scholar]
- 44. Oermann MH, Lowery NF, Thornley J. Evaluation of Web sites on management of pain in children. Pain Manag Nurs 2003; 4:99-105. [DOI] [PubMed] [Google Scholar]
- 45. Oermann MH, Gerich J, Ostosh L, Zaleski S. Evaluation of asthma websites for patient and parent education. J Pediatr Nurs 2003; 18:389-96. [DOI] [PubMed] [Google Scholar]
- 46. Friedman DB, Hoffman-Goetz L, Arocha JF. Readability of cancer information on the Internet. J Cancer Educ 2004; 19:117-22. [DOI] [PubMed] [Google Scholar]
- 47. Bergman J, Konijeti R, Lerman SE. Myelomeningocele information on the Internet is accessible and of variable quality, and requires a high reading level. J Urol 2007; 177:1138-42. [DOI] [PubMed] [Google Scholar]
- 48. Thomson MD, Hoffman-Goetz L. Readability and cultural sensitivity of web-based patient decision aids for cancer screening and treatment: a systematic review. Med Inform Internet Med 2007;32:263-86. [DOI] [PubMed] [Google Scholar]
- 49. Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med 2011; 155:97-107. [DOI] [PubMed] [Google Scholar]
- 50. Pan H, Simpson DR, Mell LK, Mundt AJ, Lawson JD. (2012). Asurvey of stereotactic body radiotherapy use in the United States. Cancer 2012; 117(19):4566-72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Web Accessibility Initiative (2005). 28-06-2011, from http://www.w3.org/WAI/eval/preliminary.html.
- 52. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol 1999; Community Health 53:105-11. [DOI] [PMC free article] [PubMed] [Google Scholar]