Skip to main content
PMC home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2018 Jul 17;28(1):e12882. doi: 10.1111/ecc.12882

Optimising eHealth tools for older patients: Collaborative redesign of a hospital website

Minh Hao Nguyen 1,✉, Nadine Bol 1, Julia C M van Weert 1, Eugène F Loos 1, Kristien M A J Tytgat 2, Debby Geijsen 3, Ellen Drenth 4, Meriam Janse 4, Ellen M A Smets 5
PMCID: PMC6588263  PMID: 30015998

Abstract

Most hospital websites have not been developed in collaboration with patients and, therefore, rarely take into account the preferences and abilities of older patients. This study describes the systematic redesign of an existing hospital website in a co‐design process with patients and professional stakeholders (e.g. researchers, physicians, nurses, department heads, policymakers, website designers), with the aim to make it more user‐friendly for older patients with colorectal cancer (CRC). The redesign process consisted of three phases, where (I) both existing content and design were evaluated among CRC patients; (II) a prototype website was developed based on these insights; which (III) was evaluated again before making final adjustments. Mixed research methods were used for the redesign process. Specifically, insights from existing literature, outcomes from qualitative and quantitative empirical studies conducted by our team, and expert knowledge from relevant stakeholders, were collected and discussed in multidisciplinary consensus meetings, and served as input for the redesigned website. While the existing website was evaluated poorly, the qualitative evaluation of the prototype website in phase 3 showed that the newly redesigned website was usable for older CRC patients. A practical roadmap on how to collaboratively redesign and optimise existing eHealth tools to make them suitable for and operational in clinical settings is provided.

Keywords: cancer, eHealth, older patients, patient education, tailoring, website development

1. INTRODUCTION

Currently, the majority of newly diagnosed patients with cancer are aged ≥65 years, and the number of newly diagnosed cancer patients of this age is rapidly increasing worldwide (American Cancer Society, 2013; Dutch Cancer Society, 2011). Due to age‐related factors, such as cognitive and sensory decline, these patients are the most at‐risk population for poor communication with providers (Sparks & Turner, 2008). As a complementary source of cancer information, the Internet can be a particularly valuable channel for patient education efforts targeted at the older patient population, as older adults are increasingly going online to seek medical information (Pew Research Center, 2014, Statistics Netherlands, 2013, 2016). Using online medical information as a complementary information source prior to medical consultations with the health care provider, might lead to better informed and more confident patients, benefit the patients’ fulfillment of information needs, and perhaps even allow for more efficient use of clinical time (Albada, Van Dulmen, Lindhout, Bensing, & Ausems, 2012; Fleisher, Bass, Ruzek, & McKeown‐Conn, 2002; Gerber & Eiser, 2001). The use of online sources might be particularly relevant for older patients, as processing and memorising medical information from consultations is considered more difficult for this group (Jansen et al., 2008).

Despite the increasing number of online medical sources, even of credible sources such as hospital websites, most have not been developed in collaboration with patients and rarely take into account the preferences and abilities of older adults (Becker, 2004; Bolle et al., 2016). Although physicians and other experts strive to ensure that the content of the information is accurate, this content does not always match with the topics that are discussed during consultations or the patients’ information needs (Fleisher et al., 2002; Rideout, Neuman, Kitchman, & Brodie, 2005). In addition, less attention is given to how this information can be optimally presented to older patients or whether the usability of the website suits the computer literacy of the older audience (Bolle et al., 2016). Particularly in older patients, age‐related factors such as cognitive and sensory decline (e.g. reduced processing speed, loss of vision and hearing) might negatively influence experience with health websites. Therefore, both content and design need to be considered when developing websites that are to be accessed by older patients (Loos & Romano Bergstrom, 2014; Pernice & Nielsen, 2002).

This paper aims to describe the systematic redesign of an existing Dutch hospital website in a co‐design process with patients and professional stakeholders (e.g. researchers, physicians, nurses, department heads, policymakers and website designers), in order to offer older patients with cancer a usable website. By sharing the lessons learned, we provide future researchers and practitioners with systematic guidance to collaboratively redesign health websites in a multidisciplinary setting. In contrast to many other papers which have reported the development of an eHealth tool from scratch, this paper illustrates how an existing tool (i.e. a hospital website) can be redesigned and optimised. This can be particularly useful, as many existing eHealth tools have great potential (Kreps & Neuhauser, 2010), but remain unused or underused (Crutzen, Kohl, & De Vries, 2012; Eysenbach, 2005; Rogers, Lemmen, Kramer, Mann, & Chopra, 2017; Van't Riet, Crutzen, & De Vries, 2010). A cost‐effective solution for clinical practice could be to redesign and optimise such eHealth tools and make them suitable for and operational in a clinical setting.

2. METHOD

2.1. Context: The GIOCA website

The existing website of the multidisciplinary outpatient clinic Gastro Intestinal Oncological Centre in Amsterdam (GIOCA) at the Academic Medical Center (AMC) in The Netherlands was redesigned and optimised for older colorectal (CRC) patients. Nearly half of the GIOCA patients reported looking up information online before consultation, of which half consulted the existing GIOCA website as part of their search. The existing website functioned as an information source for both (prospective) patients and healthcare providers to refer patients to the GIOCA, and was part of an overarching hospital website. The redesigned website was, however, intended for (prospective) patients only.

Colorectal cancer mainly occurs in older adults (American Cancer Society, 2014; Cancer Research UK, 2016; Netherlands Comprehensive Cancer Organisation IKNL, 2017). The GIOCA specialises in secondary and tertiary fast diagnostics, with patients having four to six consultations (e.g. history taking, discussing diagnosis and treatment plan) with various healthcare providers (e.g. gastroenterologist, surgeon, oncologist, radiation oncologist, nutritionist and oncological nurse) in just 1 day. Most patients are highly anxious while awaiting their diagnosis (Bronner, Nguyen, Smets, van de Ven, & van Weert, 2018), implying that they may be unable to optimally process and recall all the imparted information (Kessels, 2003; Shapiro, Boggs, Melamed, & Graham‐Pole, 1992). Using the redesigned website prior to patients’ hospital visit is likely to benefit interpersonal patient‐provider communication and to support older patients in processing the information given during the day at GIOCA. Furthermore, they can use the website to look up information afterwards.

2.2. Stakeholders

The redesign of the existing GIOCA website was carried out by a multidisciplinary project team, with assistance from a web designer specialised in digital interfaces for older adults. The project team consisted of five academic scholars (NB, EL, HN, ES, JW), two oncology nurse practitioners (ED, MJ), three colorectal surgeons (JK, PT, AV), one gastroenterologist (KT) and one radiation oncologist (DG). All medical specialists were working at GIOCA at the time of the project. Furthermore, the project team was supported by several research assistants (MA, FA, AH, RR, CS, HT). In addition, the department of patient relations and the department of communication of the AMC ensured that the GIOCA website adhered to the current hospital policies regarding (online) patient education. Finally, the project team was supported by input from various patients and nonpatient samples that provided their opinions on needs and preferences regarding adequate online cancer information provision (see “Results” for details). Both patients and nonpatients evaluated the content and design of the website during the redesign process.

2.3. The redesign process in three phases

Using a multimethod approach, we collected insights from existing literature, and qualitative (e.g. usability study) and quantitative (e.g. experiment) empirical studies conducted by the project team, and expert knowledge from all relevant stakeholders, which were later discussed in multidisciplinary consensus meetings. The redesign process of the website consisted of three phases, each divided into two substeps that focused on (a) the content or (b) the design of the website. A detailed overview of the redesign process is shown in Figure 1.

Figure 1.

Figure 1

Open in a new tab

Redesign process in three phases

“Content” refers to the information made available by the website, whereas “design” refers to the presentation of the information on the website (e.g. the lay‐out and navigational structure). In phase 1 of this project, the content and design of the preexisting GIOCA website were evaluated (Tables 1 and 2). In phase 2, a redesigned prototype version of the website was created based on the outcomes in phase 1 (Table 1). In the final phase, the usability of the prototype website was evaluated among older patients. Based on these insights, both the content and design of the website were optimised (Tables 1 and 2).

Table 1.

Overview and timeline of the redesign process in three phases

Objective Method & Data analysis Results & Insights
Phase 1: Evaluation of the preexisting website
1a. Content2013/01–2013/10 To evaluate the content of the preexisting GIOCA website; To identify additional relevant information topics to include on the website Comparison of consultation content (audiotaped; n = 37) and website content; Data were transcribed and coded using ATLAS.ti; Results were discussed in multidisciplinary consensus meetings (In)consistent content was identified between consultations and website content; Additional important information was identified that was not yet covered on the website
1b. Design2014/06–2014/07 To evaluate the usability of the preexisting GIOCA website, focusing on the design structure (navigation areas, information buttons, web pages) Qualitative usability study with think‐aloud protocol in older cancer patients (n = 9); Patients were given search tasks, application tasks, and evaluation tasks while using the website Problems with navigation structure of the website were identified; Font colours and sized needed to be consistent throughout the website; Amount of information on one web page needed to be reduced and dosed
Phase 2: Development of redesigned prototype website
2a. Content2014/05–2015/09 To rewrite the website content based on results and insights from phase 1; To develop illustrations and videos to support the textual website content Insights from phase 1 about the textual website content functioned as the basis, which was checked by the department of patient relations; Illustrations and videos were co‐developed with input from (non‐)patients (n = 15, n = 5) and professionals, with certain choices being guided by existing literature This phase resulted in cognitive illustrations that supported the textual website content where needed, and six conversational styled videos featuring patients and healthcare providers that reflected the textual information on the website
2b. Design2014/09–2015/09 To develop a clear navigation structure and design for the redesigned prototype website; To develop a tool to tailor the mode of information presentation on the redesigned prototype website Insights from the qualitative evaluation study in phase 1 functioned as the basis; Existing website design guidelines and Google Analytics visitor statistics from the preexisting website guided decisions; A study was conducted to test the effectiveness of the mode tailoring tool in a nonpatient sample (n ≈ 563; Nguyen et al., 2017; Nguyen et al., 2018); A website designer specialised in digital interfaces for older adults was consulted, and patients evaluated every test version of the prototype website; Multidisciplinary consensus meetings were held This phase resulted in a redesigned navigation structure of the website; specifically information topics that were considered important were central in the navigation menu; A horizontal built‐in bar was developed and integrated in the website that allowed users to self‐tailor the mode of information presentation (via text, illustrations and/or video)
Phase 3: Usability testing and final adjustments to the redesigned website
3a. Content2015/10–2015/11 To examine the satisfaction with the comprehensibility of the content of the redesigned prototype website Qualitative usability study with think‐aloud protocol in older CRC patients (n = 10); Patients were given search tasks, application tasks, and evaluation tasks regarding the content and design while using the website Based on suggestions from patients, final (minor) textual changes were made to the website (e.g. word choice).
3b. Design2015/10–2015/11 To evaluate the usability of the redesigned prototype website, focusing on the design structure (navigation areas, information buttons, web pages) Technical errors and difficulties regarding navigational structure were identified and adjusted in the final version of the website (e.g. adjusting the home button)
Open in a new tab

Table 2.

Comparison between preexisting website and redesigned prototype website

Phase 1 (n = 9) Phase 3 (n = 10)
Preexisting website Evaluation and improvements Redesigned prototype website Evaluation and improvements
Task
 1. Evaluation task: To give a first impression of the website + Homepage well‐organised (n = 2); ‐ Different fonts and colours made homepage look disorganised (n = 3); ‐ Too much text on homepage (n = 2); ‐ Did not like homepage (n = 2); ‐ Fonts too small (n = 1); ‐ Disliked photo medical team (n = 1) Too much text on the homepage and the use of different fonts and colours made homepage feel disorganised. New website lay‐out contained three main colours and one font type and size. Photo of medical team was replaced + Positive first impression (n = 10); + Website looked clear (n = 6); + Complimented navigation menu (n = 3); + Good to see logo of hospital to ensure credibility source (n = 1) Not applicable
 2a. Search task: To show what page(s) the patient would visit to get information about (treatment at) GIOCA + Able to find useful web pages (n = 2), using in‐text hyperlinks on homepage to get to other pages; ‐ Were not able to find useful web pages (n = 7); ‐ Did not know where to find information (n = 3); ‐ Browsed in irrelevant menus (n = 2) Primary pages could not be found by most patients. A better navigation pane (only on horizontal left) was developed that contained the most important information topics in the main menu items. In‐text hyperlinks, these were included on the homepage. Content was often clear, but the organisation could be improved. Length of text was reduced by using pull‐out menu's and font size was made slightly larger and darker + Able to find useful pages (n = 10); + Fast‐diagnostics procedure and preparation information (n = 6); + CRC and treatment‐related information (n = 3); + Both CRC and preparation information (n = 1) All patients were able to find the pages that we identified as being most important. Patients have different preferences for information modality, particularly when they are not satisfied with the text, it is important to have other modalities of information (e.g. illustrations and/or video) to be able to meet patient needs
 2b. Evaluation task: To evaluate the content and ease of finding that informationa + Web page content clear (n = 7); + Easy to find information (n = 3); ‐ Web page content irrelevant (n = 3); ‐ Too much textual information on one page (n = 2); ‐ Fonts too small and too light (n = 1) + Easy to find information because of clear navigation (n = 6); + Web page content clear (n = 6); + Preferred video (n = 3); + Liked illustrations with text (n = 2); + Information about how to prepare for consultations (e.g. questions) (n = 2); ‐ Text too long and difficult, but could relate to video (n = 1)
 3a. Application task: To go to the homepage ‐ Could not find home button (n = 9); ‐ Used back button in browser (n = 4); ‐ Used wrong navigation menu items (n = 6); ‐ No idea how to return (n = 1) None of the patients could find the home button. Home button was considered useful, but location of the home button is not intuitive and clear. Home button was moved to top left corner on website and made explicit with ‘HOME’ + Could find home button (n = 8); ‐ Could not find home button due to technical error and used back button in browser (n = 2) All patients could find the home button, despite the technical error. The location of the home button was clear, but the English word ‘HOME’ might not be understood by all Dutch patients. The technical error was fixed and a “home” icon was added to the menu to ensure all patients would identify it as the home button
 3b. Evaluation task: To assess the ease of finding the homepage ‐ Home button unclear (n = 4); ‐ Would never find button (n = 2); ‐ Move button to top left corner (n = 2); ‐ Button useful if possible to find (n = 2) + Easy to find home button (n = 8); ‐ English word “HOME” might not be clear to everyone (n = 1)
 4a. Search task: To find information on patient stories + Were able to find web page (n = 1); ‐ Were unable to find web page (n = 8); ‐ Used wrong menu on the right (n = 5); ‐ Used correct left menu, but failed to choose correct menu item (n = 3); ‐ Declined search task (n = 1) Important information could not be found by most patients. A better navigation pane (only on horizontal left) was developed that contained the most important information topics in the main menu items. All audiovisual information was developed in the form of a patient testimonial + Were able to find web page (n = 8); ‐ Could not find home button due to technical error and used back button in browser (n = 1); ‐ Was able to find web page, but could not find video mode due to technical difficulty (n = 1); ‐ Could not find “close” button video (n = 3); ‐ Were not able to find web page (n = 2) Most patients could find the information within an acceptable time span and found the navigation clear and logical. Some patients stumbled upon technical errors or difficulties undermining them in finding the correct information. These were all repaired or adjusted, such as relocating the “close” button
 4b. Evaluation task: To assess the ease of finding patient storiesb + Easy to find (n = 3); + Easy to understand (n = 3) ‐ Not an important element (n = 3); ‐ Very important element (n = 1); ‐ Not clear where to find (n = 2); ‐ Liked more patient stories (n = 2) + Easy to find (n = 7); + Easy to understand (n = 7); + Patient stories ease the mind (n = 1); ‐ Not applicable to own situation (n = 1); ‐ Focus web page should jump to video when selected (n = 2); ‐ Use different thumbnail video (n = 1)
 5a. Evaluation task: To give an overall impression of the website + Website is satisfying (n = 3); + Website is well‐organised (n = 4); ‐ Textual information too long (n = 1); ‐ Dose information on web page (n = 1); ‐ No clue where to find information (n = 1) Suggestions of patients mainly related to the textual information on the web pages being too long and the overall unstructured organisation of the website. The amount of textual information is therefore dosed by using “pull‐out” menus. Where relevant, pictures of healthcare professionals are also included + Positive evaluations, “good”, “clear”, “well‐organised” (n = 9); + Clear navigation (n = 4); + Information offered in different modes (n = 2); + Information complete (n = 2); + Font type and size (n = 7); ‐ Font type for “old people” (n = 1); Most patients were positive about the website, its navigation, and liked that information was being offered in different modes (i.e. text, illustrations and/or video). Most patients saw the website as a good tool to prepare for their consultation. Some suggestions were beyond the scope of this website (e.g. viewing medical records online), and were therefore not taken into consideration
 5b. Evaluation task: To suggest missing elements of the website + No missing elements (n = 2); ‐ Stories of CRC patients instead of other types (n = 1); ‐ Explicitly state that hospital empathises with patients (n = 1); ‐ Need for relevant information only (tailored) (n = 1); ‐ Pictures of healthcare professionals you will meet (n = 1); ‐ Website more organised and textual information shorter (n = 1) + No missing elements (n = 7); ‐ Medical records online (n = 1); ‐ Less medical jargon (n = 1); ‐ Search function missing (n = 1)
Open in a new tab

A synthesis is given of the qualitative usability studies in phase 1 and 3. For a detailed description, please e‐mail the authors. The number of patients is stated between parentheses. Positive points are indicated with an “+”, and negative points with an “‐”.

aWhen patients could not find the web page, we directed them to this page to evaluate the content. bIn phase 3 (redesigned prototype website), the task was slightly adjusted to look up information in the form of a video to test the mode‐tailoring tool.

All studies were approved by the medical ethical review board of the AMC (reference numbers: W12_019 #12.17.0028 and W13_053 #13.17.0069) and the ethics committee of the Amsterdam School of Communication Research (reference numbers: 2014‐CW‐64 and 2014‐CW‐110). All stakeholders (patients, nonpatients, physicians, nurses) involved provided informed consent.

3. RESULTS

An overview of the redesign process and timeline by phase is presented in Table 1, and is described below.

3.1. Phase 1: evaluation of the content and design of the preexisting website

In phase 1, both the (a) content and (b) design of the original GIOCA website were evaluated by the project team in two subsamples of patients.

3.1.1. Evaluating the content (1a)

To provide patients with adequate information about GIOCA before and after their visit to GIOCA, it is essential that such information is matched to the information given during the GIOCA visit. Moreover, aligning different sources of information is also important to achieve optimal synergy effects of health information provision (Linn, Van Weert, Smit, Perry, & Van Dijk, 2013). Therefore, we assessed the content of the preexisting GIOCA website and the extent to which this content matched the content of what was discussed during the GIOCA consultations. To this end, CRC patients visiting the GIOCA were asked for permission to audiotape their consultations during their visit. These consultations comprised intake and test result consultations with a physician and an oncological nurse separately.

In total, 37 audiotaped consultations of 11 CRC patients were recorded (M age = 62.36, SD age = 10.76, range 42–77 years; 81.8% male); 3–4 consultations per patient were available (M = 3.36, SD = 0.51; M length = 19:15, SD length = 8:46). Two research assistants (FA, AH) qualitatively analysed the 37 transcripts by double coding 13 transcripts and separately coding 12 transcripts each in ATLAS.ti (ATLAS.ti, 2017). Coding was based on the original content structure of the preexisting GIOCA website, providing us with an initial code book. New codes were assessed after each transcript, discussed with the second author, and were added to the code book. Next, the coded transcript texts were compared to the existing texts on the original GIOCA website, categorising transcript texts into (a) information that is similar to the original website information, (b) information that is incongruent to the original website information and (c) information that is not on the original website. These results were discussed in multidisciplinary consensus meetings with researchers and physicians to decide which website information needed to be adjusted and which information needed to be added to the website.

3.1.2. Evaluating the design (1b)

The second step involved evaluating the design of the preexisting GIOCA website (e.g. navigation areas, information buttons, web pages), by testing the usability of this website in a qualitative think‐aloud study (Figure 2). Nine patients from the GIOCA (M age = 73.22, SD age = 6.22; range 66–82 years; 55.6% male; 44.4% lower educated and 44.4% higher educated) were asked to evaluate the preexisting GIOCA website as part of a larger think‐aloud study (Bolle et al., 2016). Patients were recruited from PanelCom, a panel of people who previously participated in studies from our group and consented to be contacted for future studies (PanelCom, 2017). The usability study was conducted either in a private room at the GIOCA clinic or at the patient's home. Patients were explicitly instructed and prompted to think aloud when they read and navigated through the website. Sessions were videotaped to link the website navigation data to patients’ verbalisations. The think‐aloud procedure was guided by an interview protocol containing various tasks (i.e. search tasks, application tasks and evaluation tasks) (Bolle et al., 2016). Examples of tasks included “Imagine you would like to know about other patients’ experiences with the GIOCA clinic. Please look for such information” (search task), “Please go to the homepage” (application task) and “What do you think about the findability of the information on the website?”; “What made it easy/difficult to find the information?” (evaluation tasks).

Figure 2.

Figure 2

Open in a new tab

Preexisting website

The usability of the original GIOCA website was assessed by the efficiency (task completion time), effectiveness (quality of the solution) and user satisfaction (overall evaluation of navigability, content and aesthetic appeal) (Frøkjær, Hertzum, & Hornbæk, 2000). Search and application tasks were used to capture the efficiency (i.e. how much time it took older patients to return to the home page) and effectiveness (i.e. if older patients were able to return to the homepage and, if so, how many clicks were needed), and evaluation tasks were used to explore user satisfaction (i.e. how did older patients value the website navigation and design).

The evaluation tasks showed that most patients (n = 7) were satisfied with the content of the website (e.g. the content was clear, the illustrations were considered valuable), but were critical about the design and navigation of the website (see task 1 in Table 2). With the first impression of the homepage, several patients (n = 3) noticed the different fonts and colours used on the website and navigational areas, and indicated that the website looked disorganised with different navigation panels (left, right and top). One patient suggested to use a darker font colour and a larger font size. Several patients repeatedly mentioned that there was too much text on the web pages and suggested to limit the amount of text on one page and to dose the information.

The search tasks showed that most patients could not successfully find information. Only two patients managed to find relevant information during an open search task (task 1 in Table 2); both these patients used the hyperlinks in the text instead of the navigation menu. The other patients found no relevant information and explained that they did not know where to find the information they were looking for, or were lost browsing in irrelevant navigation menus. Moreover, when giving patients specific application tasks, such as finding their way back to the homepage (task 3 in Table 2), none of the patients managed to do so. Immediate evaluation indicated that the “home” button was not clear, and several patients suggested to make a prominent home button in the upper‐left corner of the website, which is consistent with the so‐called F‐shaped pattern on websites (Nielsen, 2006). A second search task yielded similar results (task 4 in Table 2): Eight of 9 patients were unable to find the correct web page and used either the wrong navigation area (on the right) or used the correct navigation area (on the left) but failed to select the correct menu item.

Overall, we concluded that the navigational structure of the website needed improvement, that font colours and sizes needed to be consistent throughout the website, and that the amount of text on one web page should be reduced and dosed. Interestingly, we found that older patients value textual hyperlinks to navigate throughout the website.

3.2. Phase 2: development of the redesigned prototype website

In the second phase, we developed the prototype of the redesigned website based on the insights from evaluating the content and design in phase 1 and additional literature on website design for older adults (e.g. Bolle et al., 2016; Nayak, Priest, Stuart‐Hamilton, & White, 2006; Pernice & Nielsen, 2002). This phase consisted of (a) developing website content, including website text and supporting illustrations and videos, and (b) developing the navigation structure and design of the website, as well as a tool for tailored information presentation. All steps undertaken in this process were done in close collaboration with GIOCA patients, healthcare providers, and policymakers (e.g. department of communication and department of patient relations).

3.2.1. Developing the website content (2a)

The textual content of the redesigned prototype website was guided by the outcome of the content evaluation in phase 1 of the development process. The content was extensively reviewed and approved by physicians and the department of patient relations. As the website information needed to be available in modalities other than text (see 2b below), we developed visual illustrations and videos to support the textual content. Regarding the illustrations, 15 older adults were asked to indicate their preferred style of illustrations: of these, 14 (93%) preferred realistically drawn illustrations to communicate medical information (rather than icons or abstract drawings). As cognitive illustrations (i.e. visuals aiming to improve comprehension and recall of information) increase satisfaction with online health websites and recall of health information (Bol et al., 2014; Bol, Smets et al., 2015), cognitive realistic illustrations were developed to support the textual content. All visuals were approved by various project members.

Furthermore, six informative videos were developed with a patient testimonial. Including patients’ experiences in a video contributes its realism, which is important when designing patient videos (Hillen, van Vliet, de Haes, & Smets, 2013). Although the video scripts were guided by the textual content of the website, we used a conversational style in which patients told their own experience regarding a specific information topic; this has been effective in increasing recall of health information and satisfaction with health websites (Bol, Van Weert, De Haes, Loos, & Smets, 2015; Bol, Smets et al., 2013). Video scripts were reviewed by all patients that appeared in the videos, and by various members of the multidisciplinary project team. Furthermore, an experienced film director reviewed the script and gave feedback to ensure script fluidity. Finally, the department of communication and department of patient relations reviewed the video scripts to ensure adherence to the hospital guidelines. Patients played a large role in the process of revising the film scripts, as it was important that the videos reflected what they considered important to convey in the videos. All involved stakeholders agreed on the video scripts before filming. Although most revisions were done before filming, patients suggested some additional changes to the script during filming. The videos reflect the various information themes that were identified in phase 1, and are, therefore, also congruent with the navigation structure of the website (see phase 2b). The final versions of the videos were approved by all involved stakeholders (see Appendix 1). The department of patient relations gave final approval for the entire website content.

3.2.2. Developing the design of the website (2b)

Regarding the design of the website, the structure was rearranged based on insights from the evaluation of, and Google Analytics data from the preexisting GIOCA website. Specifically, the evaluation study in phase 1 showed that several information topics that patients considered important were difficult to find. For example, patients could not find their way back to the homepage, suggesting that a clearly‐defined home button should be visible at all time. Regarding Google Analytics statistics, we identified which pages were most often consulted. For example, information on the fast‐track diagnostics procedure and general information on CRC were consulted often and were, therefore, integrated in the main navigation menu as distinct topics. Finally, in developing the structure and design of the website we followed established guidelines for websites targeted at older cancer patients (Bolle et al., 2016). For example, avoiding large amounts of textual information on one web page by using “pull out” menus for users who prefer more detailed information.

In this project we were specifically interested in presenting the website content in a tailored manner (i.e. via text, illustrations and/or video). Previous work by our group (Bol, 2015; Bol, Scholz et al., 2013) and others (Lustria et al., 2013; Smit, Linn, & van Weert, 2015) has indicated a need for tailored information provision via different modalities, particularly among older cancer patients. Therefore, an additional experimental study was conducted to investigate how online health information could be tailored to individual preferences and abilities regarding the modality of information presentation (n ≈ 563; Nguyen, Smets, Bol, Loos, & Van Weert, 2018; Nguyen et al., 2017). “Mode tailoring”, by enabling users to self‐select whether they would like to receive the information via text, illustrations and/or videos, was effective in increasing attention to and recall of online health information, and satisfaction with the attractiveness of health websites in older adults. Based on these insights, we integrated a horizontal built‐in navigation bar, enabling viewers to self‐select whether they would like to receive the information via textual, visual and/or audiovisual information (Nguyen et al., 2017, 2018; Soroka et al., 2006). Users could switch and change modalities at any time while viewing the information.

For this part of the development process, the contribution of a website designer specialised in digital interfaces for older adults was of utmost importance. With every version of the website design, a pool of six GIOCA patients was consulted to share their suggestions for improvement of the design. The healthcare providers from the GIOCA, the department of communication, and the department of patient relations considered it important to develop a website with the same “look and feel” as the preexisting website. Therefore, we used similar colours for the background and navigation areas as used in the preexisting website. The department of communication at the hospital gave final approval for the layout and design of the redesigned website (Figure 3).

Figure 3.

Figure 3

Open in a new tab

Redesigned prototype website

3.3. Phase 3: usability testing and final adjustments to the redesigned website

Subsequently, using a qualitative think‐aloud protocol similar to the study in phase 1, we tested the usability of the redesigned prototype website among 10 older CRC patients from the GIOCA recruited from PanelCom (M age = 69.60, SD age = 4.35, range 64–80 years, 70% male) [28]. The aim was to evaluate the comprehensibility of the website content and to assess the design structure of the website (navigation areas, information buttons, web pages). Although the procedure and interview protocol of the usability study were similar to the think‐aloud study in phase 1 (consisting of search tasks, application tasks, and evaluation tasks), a prediction task was added to assess the navigation of the website, for example “Please look up the menu item ‘I have complaints’. What information do you expect to find when you click on this menu item?” Patients were asked for their critical feedback on the website. Based on the results from the usability study, final adjustments were made to the final version of the website, which are described below (Table 2).

The first evaluation task showed that all patients had a positive first impression of the prototype websites. Almost all (n = 9 of 10) mentioned that the website looked clear and well‐organised. Patients appreciated the clear navigation menu of the website (n = 3) and the font size and colour (n = 7). Among the 10 patients, different preferences emerged regarding information delivery modes; some were satisfied with text and illustrations, while others preferred the videos (e.g. “The text is too formal, I can relate more to the video because it feels more personal with patients conveying the information”). Hence, regarding the mode tailoring tool, most patients found that it was good that the website provided this option (n = 6). The majority (n = 6) found it easy to use, and understood the purpose of the mode tailoring tool. Three patients explicitly mentioned that they liked the option to receive information in a dosed manner (e.g. “If I need to look something up quickly, I can select the text only. If I am ready for more information, I would select the video maybe on my second or third visit”).

With the open search task to look for information that they found to be personally relevant (task 2 in Table 2), all patients indicated they could find information that they considered important. They used either the left navigation menu or the textual hyperlinks on the homepage to look up this information. Six patients indicated that they could easily find this information, because the navigation menu was clear. With a specific search task (task 4 in Table 2), 8 of the 10 patients managed to find this information. However, many patients had difficulty finding the “close” button for the video pop‐up screen as it was located at the top of the video, while the play and pause button were at the bottom. Thus, patients were often looked at the bottom area to close the video. In the final version, all the technical errors or difficulties found in the website during the usability test were repaired and adjusted, such as relocating the close button of the video pop‐up screen.

With the application task to return to the homepage, eight of the patients managed to do so; two patients were unable to find the home button, due to a technical error in the website that made it impossible to scroll up to see this button. One patient wondered whether everyone would understand the English word “home”. The suggestions from patients guided us in making adjustments such as accompanying the “home” button with an icon of a house, to ensure that all patients would understand this. Regarding the expectation task about the menu item “I have complaints”, six patients expected the information to be about physical complaints, while three others expected information about complaints regarding communication within or between different hospitals. To ensure unambiguity, this menu item was adjusted to read “I have physical complaints”.

Overall, we concluded that the navigation structure of the website was usable for older patients, and that information modes in a tailored manner were well received. In general, when comparing the results from the think‐aloud study in phase 1 with this usability study, we can conclude that patients appreciated and were better able to use the redesigned mode‐tailored website. A more detailed comparison between the usability study outcomes in phase 1 and 3 is presented in Table 2. All results were discussed among the multidisciplinary project team before deciding upon the final adjustments. The final redesigned website is shown in Figure 4.

Figure 4.

Figure 4

Open in a new tab

Final redesigned website

4. DISCUSSION

This paper aimed to describe the process through which existing eHealth tools can be redesigned and optimised, using an existing hospital website as a case study. By describing the phases and methods for redesigning a health website for older cancer patients involving multiple stakeholders (i.e. patients, researchers, physicians, nurses, department heads, policymakers and website designers), this paper provides researchers and professionals with a practical roadmap on how to collaboratively redesign and optimise existing eHealth tools in an efficient and effective way. Below, we critically reflect on our redesign process and highlight practical matters that need to be considered to inform future research and practice in the eHealth context.

4.1. Strengths of applying a redesign approach

A particular strength of our redesign process is the evaluation of the content of an existing hospital website in phase 1, by comparing the content of audiotaped consultations with the existing website content. This is a unique approach, as the website content was guided by real‐life questions and topics that were being discussed rather than what information patients and providers believe to be important. Furthermore, such an approach ensures that the content of different information sources (e.g. website, nurses and physicians) are aligned. This is particularly important, as conflicting or inadequate information can have negative consequences for patients (Wald, Dube, & Anthony, 2007). It should be noted that aligning content over different information sources—mediated and/or interpersonal—is also important when developing new eHealth tools, and is thus not limited to redesign approaches.

Another benefit of applying a redesign approach is that it allows for a systematic comparison between the usability of the existing tool and the redesigned tool. In our case, we conducted the evaluation of the final redesigned website with the same research method as how the preexisting website was evaluated. This way, we were able to systematically map the adjustments that were made in improving the website, the consequences of these changes and whether these changes improved usability for older patients (see Table 2).

A final strength of a redesign approach is the availability of existing user data (e.g. Google Analytics) that can be used to improve the eHealth tool. In our case, these data gave insight into (a) which information patients consulted most often and was likely to be considered as most important by patients, and (b) which information patients were less likely to seek but is considered to be important by healthcare providers. These insights were then used to develop a navigational structure that placed these topics centrally, with the aim to direct patients to these particular information topics on the website. It is likely that the website would have been arranged differently if we did not rely on existing user data from the preexisting website to inform us, but merely followed existing guidelines. Studying use patterns of existing eHealth tools can provide further insight into patients’ informational needs, which are particularly valuable for the optimisation of eHealth tools.

4.2. Reflection, practical tips and guidelines

Developing or redesigning a health website in a multidisciplinary setting is undoubtedly a complex and challenging process. Besides integrating evidence from existing literature and guidelines into a website, such development generally involves collaboration with multiple stakeholders that might further complicate the development process. Moreover, during the development process, it might be necessary to conduct additional (unplanned) studies to decide on the exact website components. Overall, involving all parties in the process as co‐designers is desirable, as this is more likely to result in a feasible and usable end product (O'Brien et al., 2016; Oudshoorn, Rommes, & Stienstra, 2004).

Although redesigning and optimising existing eHealth tools can be a cost‐effective and efficient approach, this approach also has limitations. For example, although active patient involvement is important to ensure that patients can use and will use eHealth tools or websites to their full potential (Van Bruinessen et al., 2014; Van Gemert‐Pijnen et al., 2011), working with an existing product slightly limits this opportunity. Instead of allowing patients to be actively involved in designing the eHealth tool from scratch (e.g. deciding on its content, design and features), with a redesign approach existing materials are already available. For example, in our case, patients did not thoroughly evaluate the entire textual website content. Although they did evaluate parts of the website content during the usability study that aimed to assess the usability of the navigation and design of the existing website, no patients had read and assessed the entire textual website content. We did not ask patients to do this, because this involves considerable work and we did not want to burden the patients—regardless of developing an eHealth tool from scratch or redesigning an existing tool. The critical assessment was done by researchers and physicians to ensure the content was accurate, and by the department of patient relations to ensure the content met the hospital guidelines (e.g. formulation and sentence structure). Nevertheless, within our redesign process we strived to ensure that older cancer patients (the end‐users of the product) were involved throughout each phase of the process in the best possible way. This way, even though patients did not evaluate the entire website content, they did deliver valuable input for optimising the website. Future studies might aim to also involve patients in this process to ensure that all information is clear and comprehensible for the end user.

In phase 2, we collaboratively developed the prototype version of the mode‐tailored website for older CRC patients. All stakeholders were involved in both the development of the additional content (i.e. videos/illustrations) and design (i.e. website navigation structure and mode tailoring tool). Although a clear project planning was made beforehand to guide the development process, this phase took longer than initially expected. Specifically, during the project, an additional empirical study had to be conducted to investigate whether the mode‐tailoring tool was an effective way to tailor information presentation modality on health websites. In addition, as all steps required discussion with all stakeholders, meaning that the project team was also dependent on other parties such as patients, department heads and policymakers. Considering patients as end users and involving them in every step was time‐consuming but undeniably important, as they provided useful and novel insights into matters which researchers and health practitioners might have overseen. Physicians and nurses often lack the time to be deeply involved in the development process; nevertheless, their contribution and involvement is essential when aiming to redesign a product to be implemented in practice. Therefore, it is important to make a structured planning of the developmental process, especially when there is a limited time frame. Nevertheless, it is essential to allow some flexibility in the schedule, as some obstacles might be encountered. As for time management, multidisciplinary consensus meetings need to be planned ahead, and the topics that need to be decided on by the team need to be placed on a clear agenda beforehand. If the matter requires reviewing website content or video scripts, the other parties need to be informed in a timely manner. Thus, clear arrangements are required to ensure that the other parties understand what is expected from them and within which time frame.

In phase 3, we tested the usability of the prototype website and made the final adjustments. Although the redesign of the website was completed in this project, it is important to consider that websites and eHealth tools age rapidly and need to be updated on a frequent basis. For example, medical procedures may change, which requires an update of the website content. On the other hand, existing eHealth tools may not be supported by new devices and operating systems, which requires an update of the tool's technological system. In practice, this means that someone should be appointed to monitor this during and after the research project, to ensure that an effective website or eHealth tool does not go offline after the research project ends.

In our redesign project, we actively used existing guidelines for making decisions about the design structure of the website. Furthermore, it also guided our choice for participatory design over traditional top‐down approaches. Additionally, we closely worked with other researchers who developed design guidelines for older patients (Bolle et al., 2016). Yet, it is important to reflect on our own redesign process—therefore an overview of practical tips and guidelines is given in Table 3.

Table 3.

Practical tips and guidelines for redesigning eHealth tools

General guidelines
 Follow the three phases of the redesign process: evaluate the existing tool, optimise the tool into a prototype version and evaluate the prototype in the target audience;
 The end users of the technology should be central in each step of the redesign process;
 Consult existing guidelines on technology design for the intended target audience;
 Data from the existing technology, e.g. user data such as Google Analytics, can be used to guide decisions during the redesign process.
Collaboration
 Inform all stakeholders in the early stages of the redesign project;
 Stakeholders that facilitate the redesign process and implementation of the eHealth tool (e.g. healthcare providers, decision makers, policymakers) should be maximally involved;
 Stay updated on other eHealth projects to avoid working on similar projects and instead establish and consolidate cross‐disciplinary collaborations.
Planning
 Map all the steps that need to be undertaken in the redesign project beforehand and determine which stakeholders should be involved in which way for each step;
 Make a structured planning of the redesign project, but remain flexible for unanticipated delays or studies that are needed to gain a better insight for decision making.
Open in a new tab

4.3. Implications and future research directions

This website was redesigned with the aim to optimise information provision to older CRC patients who will be visiting a fast‐track diagnostics center. The next step is to formally evaluate the effectiveness of the website in a randomised controlled trial (RCT). The importance of formal evaluation has been noted by different intervention development guidelines, such as the Intervention Mapping protocol and the Medical Research Council framework (Bartholomew, Parcel, & Kok, 1998; Craig et al., 2008). In essence, a formal evaluation in the form of a RCT allows to assess the effectiveness of the website intervention. However, evaluation studies of eHealth interventions for older patients often lack a clear description of the development process and usability, which inhibits other researchers from determining the effective intervention components (Bolle, Van Weert, Smets, & Loos, 2015). By reporting detailed information on the redesign and user satisfaction of this website and its components, we hope to promote full transparency to further unravel the active ingredients of eHealth interventions for older patients.

Finally, we urge communication staff, practitioners and researchers to consider the various possibilities when taking on a project that involves the development of an eHealth tool: redesign an existing technology or develop a new tool from scratch? Both approaches have their benefits and limitations—as discussed here—and both should be considered in terms of cost‐effectiveness and efficiency.

ACKNOWLEDGEMENTS

This study was funded by the Dutch Cancer Society (KWF, grant number UVA 2014‐6700). KWF was not involved in the study design, data collection, data analysis and report writing. This project is registered at the Netherlands National Trial Register (NTR) under TC 5904. We would like to thank the following people for their contribution to collecting data, developing and reviewing the website materials, and/or facilitating the redesign process during this project: Monique Alblas, Felicitas van Alphen, Sifra Bolle, Dieke Folkeringa, Amanda Hol, Jean Klinkenbijl, Regina Rijpkema, Geke Romijn, Claire Soliman, Pieter Tanis, Hans Teunisse and Anthony van de Ven. Furthermore, we thank all patients and other parties that have been involved in the development of this website intervention. Without their contribution and cooperation this research project would not have been possible.

APPENDIX 1.

Videos

Video title Link
1 How does a day of fast‐track diagnostics at GIOCA look like? https://youtu.be/x97gBpxunYA
2 Practical information about the first visit to GIOCA https://youtu.be/TvxCUkWwxgY
3 Preparing for consultations with the physician and the nurse https://youtu.be/3ebJXROd8CI
4 Contact information and reasons (e.g. physical complaints) for consulting the hospital https://youtu.be/H_SaDSjoT6E
5 Tips about managing nutrition, weight and stool in daily life https://youtu.be/sPw4iWzwNBU
6 Tips about dealing with fatigue in daily life https://youtu.be/xOqsxb30HH4
Open in a new tab

Nguyen MH, Bol N, van Weert JCM, et al. Optimising eHealth tools for older patients: Collaborative redesign of a hospital website. Eur J Cancer Care. 2019;28:e12882 10.1111/ecc.12882

REFERENCES

  1. Albada, A. , Van Dulmen, S. , Lindhout, D. , Bensing, J. M. , & Ausems, M. G. (2012). A pre‐visit tailored website enhances counselees’ realistic expectations and knowledge and fulfils information needs for breast cancer genetic counselling. Familial Cancer, 11(1), 85–95. 10.1007/s10689-011-9479-1 [DOI] [PubMed] [Google Scholar]
  2. American Cancer Society (2013). Cancer facts and figures. Retrieved from https://www.cancer.org/research/cancer-facts-statistics/all-cancer-facts-figures/cancer-facts-figures-2013.html (Archived by WebCite® at http://www.webcitation.org/6pLksS1lY)
  3. American Cancer Society (2014). Colorectal cancer facts and figures 2014–2016. Retrieved from https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/colorectal-cancer-facts-and-figures/colorectal-cancer-facts-and-figures-2014-2016.pdf (Archived by WebCite® at http://www.webcitation.org/6pLmSl2vu)
  4. ATLAS.ti (2017). ATLAS.ti: The qualitative analysis and research software. Retrieved from http://atlasti.com/
  5. Bartholomew, L. K. , Parcel, G. S. , & Kok, G. (1998). Intervention mapping: a process for developing theory and evidence‐based health education programs. Health Education & Behavior, 25(5), 545–563. 10.1177/109019819802500502 [DOI] [PubMed] [Google Scholar]
  6. Becker, S. A. (2004). A study of web usability for older adults seeking online health resources. ACM Transactions on Computer‐Human Interaction, 11(4), 387–406. 10.1145/1035575.1035578 [DOI] [Google Scholar]
  7. Bol, N. (2015). How to present online information to older cancer patients. Amsterdam, the Netherlands: University of Amsterdam. ISBN 9789462039469. [Google Scholar]
  8. Bol, N. , Scholz, C. , Smets, E. M. A. , Loos, E. F. , de Haes, J. C. J. M. , & van Weert, J. C. M. (2013). Senior patients online: which functions should a good patient website offer? In Stephanidis C. & Antona M. (Eds.), Universal access in human‐computer interaction. User and context diversity. Proceedings HCII 2013, Part II, LNCS 8010 (pp. 32–41). Berlin Heidelberg: Springer‐Verlag. [Google Scholar]
  9. Bol, N. , Smets, E. M. A. , Eddes, E. H. , De Haes, H. C. J. M. , Loos, E. F. , & Van Weert, J. C. M. (2015). Illustrations enhance older colorectal cancer patients’ website satisfaction and recall of online cancer information. European Journal of Cancer Care, 24(2), 213–223. 10.1111/ecc.12283 [DOI] [PubMed] [Google Scholar]
  10. Bol, N. , Smets, E. M. A. , Rutgers, M. M. , Burgers, J. A. , De Haes, J. C. J. M. , Loos, E. F. , & Van Weert, J. C. M. (2013). Do videos improve website satisfaction and recall of online cancer‐related information in older lung cancer patients? Patient Education and Counseling, 92(3), 404–412. 10.1016/j.pec.2013.06.004 [DOI] [PubMed] [Google Scholar]
  11. Bol, N. , Van Weert, J. C. M. , De Haes, J. C. J. M. , Loos, E. F. , De Heer, S. , Sikkel, D. , & Smets, E. M. A. (2014). Using cognitive and affective illustrations to enhance older adults’ website satisfaction and recall of online cancer‐related information. Health Communication, 29(7), 678–688. 10.1080/10410236.2013.771560 [DOI] [PubMed] [Google Scholar]
  12. Bol, N. , Van Weert, J. C. M. , De Haes, J. C. J. M. , Loos, E. F. , & Smets, E. M. A. (2015). The effect of modality and narration style on recall of online health information: results from a web‐based experiment. Journal of Medical Internet Research, 17(4), e104 10.2196/jmir.4164 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bolle, S. , Romijn, G. , Smets, E. M. A. , Loos, E. F. , Kunneman, M. , & Van Weert, J. C. M. (2016). Older cancer patients’ user experiences with web‐based health information tools: A think‐aloud study. Journal of Medical Internet Research, 18(7), e208 10.2196/jmir.5618 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Bolle, S. , Van Weert, J. C. M. , Smets, E. M. A. , & Loos, E. F. (2015). Lack of development and usability descriptions in evaluation reports on online health information tools for older patients In Zhou J. & Salvendy G. (Eds.), Human aspects of IT for the aged population: Design for everyday life. Proceedings HCII 2015, Part II, LNCS 9194 (pp. 27–37). Cham, Switzerland: Springer International Publishing. [Google Scholar]
  15. Bronner, M. B. , Nguyen, M. H. , Smets, E. M. , van de Ven, A. W. , & van Weert, J. C. M. (2018). Anxiety during cancer diagnosis: Examining the influence of monitoring coping style and treatment plan. Psycho‐Oncology, 27(2), 661–667. 10.1002/pon.4560 [DOI] [PubMed] [Google Scholar]
  16. Cancer Research UK (2016). Bowel cancer incidence by age. Retrieved from http://www.cancerresearchuk.org/health-professional/cancer-statistics/statistics-by-cancer-type/bowel-cancer/incidence#heading-One (Archived by WebCite® at http://www.webcitation.org/6pLmaQ99w)
  17. Craig, P. , Dieppe, P. , Macintyre, S. , Michie, S. , Nazareth, I. , & Petticrew, M. (2008). Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ, 337, a1655 10.1136/bmj.a1655 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Crutzen, R. , Kohl, L. , & De Vries, N. (2012). Kennissynthese online preventie. Maastricht, the Netherlands: Universitaire Pers Maastricht. ISBN 9789461592248. [Google Scholar]
  19. Dutch Cancer Society (2011). Kanker in Nederland tot 2020: Trends en prognoses. Retrieved from https://www.kwf.nl/SiteCollectionDocuments/rapport-Kanker-in-Nederland-tot-2020.pdf (Archived by WebCite® at http://www.webcitation.org/6pLl8R7Zd)
  20. Eysenbach, G. (2005). The law of attrition. Journal of Medical Internet Research, 7(1), e11 10.2196/jmir.7.1.e11 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Fleisher, L. , Bass, S. , Ruzek, S. B. , & McKeown‐Conn, N. (2002). Relationships among Internet health information use, patient behavior and self‐efficacy in newly diagnosed cancer patients who contact the National Cancer Institute's NCI Atlantic Region Cancer Information Service (CIS). Proceedings of the American Medical Informatics Association Symposium, 260–264. [PMC free article] [PubMed]
  22. Frøkjær, E. , Hertzum, M. , & Hornbæk, K. (2000). Measuring usability: Are effectiveness, efficiency, and satisfaction really correlated? Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 2(1), 345–352. 10.1145/332040.332455 [DOI] [Google Scholar]
  23. Gerber, B. S. , & Eiser, A. R. (2001). The patient‐physician relationship in the Internet age: Future prospects and the research agenda. Journal of Medical Internet Research, 3(2), e15 10.2196/jmir.3.2.e15 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hillen, M. A. , van Vliet, L. M. , de Haes, H. C. , & Smets, E. M. (2013). Developing and administering scripted video vignettes for experimental research of patient–provider communication. Patient Education and Counseling, 91(3), 295–309. 10.1016/j.pec.2013.01.020 [DOI] [PubMed] [Google Scholar]
  25. Jansen, J. , Van Weert, J. , Van der Meulen, N. , Van Dulmen, S. , Heeren, T. , & Bensing, J. (2008). Recall in older cancer patients: Measuring memory for medical information. The Gerontologist, 48(2), 149–157. 10.1093/geront/48.2.149 [DOI] [PubMed] [Google Scholar]
  26. Kessels, R. P. (2003). Patients’ memory for medical information. Journal of the Royal Society of Medicine, 96(5), 219–222. 10.1258/jrsm.96.5.219 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kreps, G. L. , & Neuhauser, L. (2010). New directions in eHealth communication: Opportunities and challenges. Patient Education and Counseling, 78(3), 329–336. 10.1016/j.pec.2010.01.013 [DOI] [PubMed] [Google Scholar]
  28. Linn, A. J. , Van Weert, J. C. M. , Smit, E. G. , Perry, K. , & Van Dijk, L. (2013). 1+ 1= 3? The systematic development of a theoretical and evidence‐based tailored multimedia intervention to improve medication adherence. Patient Education and Counseling, 93(3), 381–388. 10.1016/j.pec.2013.03.009 [DOI] [PubMed] [Google Scholar]
  29. Loos, E. F. , & Romano Bergstrom, J. (2014). Older adults In Romano Bergstrom J. & Schall A. J. (Eds.), Eye tracking in user experience design (pp. 313–329). Amsterdam, the Netherlands: Elsevier; 10.1016/B978-0-12-408138-3.00012-1 [DOI] [Google Scholar]
  30. Lustria, M. L. A. , Noar, S. M. , Cortese, J. , Van Stee, S. K. , Glueckauf, R. L. , & Lee, J. (2013). A meta‐analysis of web‐delivered tailored health behavior change interventions. Journal of Health Communication, 18(9), 1039–1069. 10.1080/10810730.2013.768727 [DOI] [PubMed] [Google Scholar]
  31. Nayak, L. , Priest, L. , Stuart‐Hamilton, I. , & White, A. (2006). Website design attributes for retrieving health information by older adults: An application of architectural criteria. Universal Access in the Information Society, 5(2), 170–179. 10.1007/s10209-006-0029-9 [DOI] [Google Scholar]
  32. Netherlands Comprehensive Cancer Organisation IKNL (2017). Incidentiecijfers Nederlandse Kankerregistratie. Retrieved from http://www.cijfersoverkanker.nl/selecties/dataset_1/img58b432fd96c64?row=1&direction=up#table
  33. Nguyen, M. H. , Smets, E. M. , Bol, N. , Loos, E. F. , & Van Weert, J. C. M. (2018). How tailoring the mode of information presentation influences younger and older adults’ satisfaction with health websites. Journal of Health Communication, 23(2), 170–180. 10.1080/10810730.2017.1421729 [DOI] [PubMed] [Google Scholar]
  34. Nguyen, M. H. , Van Weert, J. C. M. , Bol, N. , Loos, E. F. , Tytgat, K. M. A. J. , van de Ven, A. W. H. , & Smets, E. M. A. (2017). Tailoring the mode of information presentation: Effects on younger and older adults’ attention and recall of online information. Human Communication Research, 43(1), 102–126. 10.1111/hcre.12097 [DOI] [Google Scholar]
  35. Nielsen, J. (2006). F‐Shaped Pattern For Reading Web Content (original study). Retrieved from https://www.nngroup.com/articles/f-shaped-pattern-reading-web-content-discovered/ (Archived by WebCite® at http://www.webcitation.org/6pLnHk3IM)
  36. O'Brien, N. , Heaven, B. , Teal, G. , Evans, E. H. , Cleland, C. , Moffatt, S. , … Moynihan, P. (2016). Integrating evidence from systematic reviews, qualitative research, and expert knowledge using co‐design techniques to develop a web‐based intervention for people in the retirement transition. Journal of Medical Internet Research, 18(8), e210 10.2196/jmir.5790 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Oudshoorn, N. , Rommes, E. , & Stienstra, M. (2004). Configuring the user as everybody: Gender and design cultures in information and communication technologies. Science, Technology & Human Values, 29(1), 30–63. 10.1177/0162243903259190 [DOI] [Google Scholar]
  38. PanelCom (2017). Welkom bij PanelCom. Retrieved from http://www.healthcommunication.nl/panelcom/. (Archived by WebCite® at http://www.webcitation.org/6qejW67TN)
  39. Pernice, K. , & Nielsen, J. (2002). Web usability for senior citizens. Design guidelines based on usability studies with people age 65 and older. Retrieved from http://www.nngroup.com/reports/senior-citizens-on-the-web (Archived by WebCite® at http://www.webcitation.org/6pLkffQSz)
  40. Pew Research Center (2014). Older adults and technology use. Retrieved from http://www.pewinternet.org/files/2014/04/PIP_Seniors-and-Tech-Use_040314.pdf (Archived by WebCite® at http://www.webcitation.org/ 6pLm2nQUW)
  41. Rideout, V. , Neuman, T. , Kitchman, M. , & Brodie, M. (2005). E‐health and the elderly: How seniors use the internet for health information. Retrieved from https://kaiserfamilyfoundation.files.wordpress.com/2013/01/e-health-and-the-elderly-how-seniors-use-the-internet-for-health-information-key-findings-from-a-national-survey-of-older-americans-survey-report.pdf
  42. Rogers, M. A. , Lemmen, K. , Kramer, R. , Mann, J. , & Chopra, V. (2017). Internet‐delivered health interventions that work: Systematic review of meta‐analyses and evaluation of website availability. Journal of Medical Internet Research, 19(3), e90 10.2196/jmir.7111 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Shapiro, D. E. , Boggs, S. R. , Melamed, B. G. , & Graham‐Pole, J. (1992). The effect of varied physician affect on recall, anxiety, and perceptions in women at risk for breast cancer: an analogue study. Health Psychology, 11(1), 61 10.1037//0278-6133.11.1.61 [DOI] [PubMed] [Google Scholar]
  44. Smit, E. S. , Linn, A. J. , & van Weert, J. C. (2015). Taking online computer‐tailoring forward: The potential of tailoring the message frame and delivery mode of online health behaviour change interventions. European Health Psychologist, 17(1), 25–31. [Google Scholar]
  45. Soroka, A. J. , Wright, P. , Belt, S. , Pham, D. T. , Dimov, S. S. , De Roure, D. , & Petrie, H. (2006). User choices for modalities of instructional information. In IEEE International Conference on Industrial Informatics (pp. 411–416). IEEE. 10.1109/indin.2006.275835 [DOI] [Google Scholar]
  46. Sparks, L. , & Turner, M. M. (2008). The impact of cognitive and emotive communication barriers on older adult message processing of cancer‐related health information: New directions for research In Sparks L., O'Hair H., & Kreps G. (Eds.), Cancer, communication and aging (pp. 17–47). New York, NY: Hampton Press Inc. [Google Scholar]
  47. Statistics Netherlands (2013). Internet ouderen fors toegenomen. Retrieved from http://www.cbs.nl/nl-NL/menu/themas/vrije-tijd-cultuur/publicaties/artikelen/archief/2013/2013-4005-wm.htm (Archived by WebCite® at http://www.webcitation.org/ http://www.webcitation.org/6pLlCouGd)
  48. Statistics Netherlands (2016). Acht procent van de Nederlanders nooit op internet. Retrieved from https://www.cbs.nl/nl-nl/nieuws/2016/22/acht-procent-van-de-nederlanders-nooit-op-internet (Archived by WebCite® at http://www.webcitation.org/6pLldmel2)
  49. Van Bruinessen, I. R. , van Weel‐Baumgarten, E. M. , Snippe, H. W. , Gouw, H. , Zijlstra, J. M. , & van Dulmen, S. (2014). Active patient participation in the development of an online intervention. JMIR Research Protocols, 3(4), e59 10.2196/resprot.3695 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Van Gemert‐Pijnen, J. E. , Nijland, N. , van Limburg, M. , Ossebaard, H. C. , Kelders, S. M. , Eysenbach, G. , & Seydel, E. R. (2011). A holistic framework to improve the uptake and impact of eHealth technologies. Journal of Medical Internet Research, 13(4), e111 10.2196/jmir.1672 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Van't Riet, J. , Crutzen, R. , & De Vries, H. (2010). Investigating predictors of visiting, using, and revisiting an online health‐communication program: A longitudinal study. Journal of Medical Internet Research, 12(3), e37 10.2196/jmir.1345 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Wald, H. S. , Dube, C. E. , & Anthony, D. C. (2007). Untangling the Web—The impact of Internet use on health care and the physician–patient relationship. Patient Education and Counseling, 68(3), 218–224. 10.1016/j.pec.2007.05.016 [DOI] [PubMed] [Google Scholar]

Articles from European Journal of Cancer Care are provided here courtesy of Wiley

RESOURCES