Skip to main content
PMC home page
Internet Interventions logoLink to Internet Interventions
. 2017 Jun 1;9:51–56. doi: 10.1016/j.invent.2017.05.002

ListeningTime; participatory development of a web-based preparatory communication tool for elderly cancer patients and their healthcare providers

Janneke Noordman a,⁎, Jeanine A Driesenaar a, Inge R van Bruinessen a, Sandra van Dulmen a,b,c
PMCID: PMC6096291  PMID: 30135837

Abstract

Objective

This paper outlines the participatory development process of a web-based preparatory communication tool for elderly cancer patients and their oncological healthcare providers (HCPs). This tool aims to support them to (better) prepare their encounters. An overarching aim of the project is to develop the tool in a participatory way to increase uptake and use.

Methods

Scrum, a participatory framework originated from software development, was applied to develop the tool. Using constant feedback loops, elderly (former) cancer patients, oncological HCPs and their representatives were, as end-users, involved.

Results

During six ‘sprints’, the communication tool ‘ListeningTime’ was developed with input from end-users. The use of scrum in developing an innovative tool was challenging in this context, because of time constraints of seriously-ill patients and busy HCPs and the co-creation involving non-profit scientific researchers and a for-profit development company.

Conclusions

The collaboration with end-users facilitated the development process of ListeningTime. Early involvement of end-users and flexibility in terms of planning and setup appear to be preconditions for creating a bottom-up inspired development procedure. Several challenges emerged from using scrum as participatory framework. Nevertheless, the ‘pressure cooking situation’, using scrum, resulted in a quick development process and a product ready for implementation.

Highlights

  • •

    Scrum, a participatory framework, was applied to develop a patient and provider supportive, web-based communication tool

  • •

    Constantly involving elderly cancer patients and oncological care providers in the development process is challenging.

  • •

    Software developers' requirements do not always coincide with scientific ones

1. Background

In oncology, both healthcare providers (HCPs) and patients are responsible for achieving effective communication during encounters. HCPs usually control the interaction, while patients are expected to participate actively. This is, however, not always reflected in daily oncology practice, especially in case of older patients. Elderly cancer patients ask fewer questions, find it hard to communicate their informational needs or preferences, and show less active behavior than younger patients (Posma et al., 2009, Jansen et al., 2010, Rademakers, 2014). Recently, 47% of elderly cancer patients (≥ 65 years) have indicated to experience barriers in communicating with their oncological care providers and expressed their need for supportive interventions (Noordman et al., 2017). Frequently reported barriers were: not wanting to be bothersome, remembering topics only afterwards, feeling nervous and having the perception that there is too little time (Noordman et al., 2017). These findings indicate the importance of not only training providers in communication, but also supporting elderly cancer patients. Modelling videos, demonstrating different communication strategies of simulated patient-provider encounters, can serve as a supportive intervention to overcome communication barriers. Modelling has proven to be effective in patient-targeted skill building interventions (Henselmans et al., 2013, Roter et al., 2012, Krouse, 2001). Also, previous studies show that listening back to audio recordings can enhance recall, improve decision making and the communication with family members and reduce anxiety (Watson and McKinstry, 2009, Hack et al., 2013). Combining these strategies can support elderly cancer patients in overcoming their communication barriers. Such supportive interventions can be especially helpful when delivered online, as the content and type of online interventions can be computer tailored to patients' needs and preferences, it is easily accessible, time-efficient and the cost of implementation is minimal once developed (Cegale, 2006, Noar et al., 2009). Compared to human-delivered interventions, intervention fidelity can also be more easily maintained in web-based interventions (Noar et al., 2009). However, many eHealth interventions report attrition (like drop-out, non-usage) and adoption problems (i.e. poor uptake after implementation) (van Gemert-Pijnen et al., 2011, Eysenbach, 2005). By actively involving patients and providers in developing a web-based intervention, the use and uptake of the intervention is expected to increase. Actively involving patients with cancer in designing and evaluating a web-based supportive communication tool appears to be feasible and appreciated (van Bruinessen et al., 2014). The process of involving patients and providers in developing a web-based intervention is not often described, yet can be helpful for researchers and developers in achieving the aim of a user-centered tool. For example, Green et al. (2016) showed that applying a participatory framework to involve service users in developing an aid for physical healthcare results in quality improvement. In addition, Winterling et al. (2016) found that the collaboration between researchers and young cancer patients contributed substantially to the development of a self-help web-based intervention. Another study reported that service users and healthcare professionals perceived their involvement as having a positive impact on mental health services (Omeni et al., 2014).

There are several participatory frameworks available to involve end-users in the development of web-based interventions, e.g. intervention mapping, CeHRes framework and 4Pi National Involvement Standards (van Gemert-Pijnen et al., 2011, Green et al., 2016, Bartholomew et al., 1998, de Beurs et al., 2017). For the present project, a ‘scrum’ framework (Schwaber and Sutherland, 2011) was used to guide the participatory development process. Scrum is an iterative approach that allows input from end-users on meaningful, small steps in the development process of an online intervention. The present paper outlines the participatory development process, by using a scrum framework, of a web-based preparatory communication tool for elderly cancer patients and their oncological healthcare providers.

2. Material and methods

2.1. Aim

The project aspired to develop ‘ListeningTime’, a web-based preparatory communication tool for elderly cancer patients (≥ 65 years), in order to help them (better) prepare their encounters with oncological HCPs. The tool was also designed to support HCPs in preparing their encounters with elderly patients. An overarching aim of the project was to develop ListeningTime in a participatory way to increase uptake and use. In this paper, we describe this development process; how we applied a participatory framework and the lessons learned during this process.

2.2. Participants and recruitment

Elderly cancer patients (≥ 65 years), their oncological healthcare providers, and representatives of the patient organization NFK (Nederlandse Federatie van Kankerpatiënten organisaties) and the ‘Quality institute for oncological and palliative research and practice’ were involved in the development of ListeningTime.

(Former) cancer patients (with all types of cancer, ≥ 65 years, and sufficient mastering of the Dutch language) were recruited via the patient organization NFK. Oncological healthcare providers (e.g. oncologists, oncology nurses) were recruited via the NFK, the ‘Quality institute for oncological and palliative research and practice’ and the network of the authors.

3. Calculation

3.1. Participatory framework: scrum

For the content and technical development of ListeningTime we used a ‘scrum’ framework. Scrum is defined as “A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value” (Schwaber and Sutherland, 2011). Scrum is a popular ‘agile working’ framework in software development (Fowler and Highsmith, 2001). It is an iterative, dynamic and flexible way of working, guided by input of end-users.

Scrum is characterized by short sprints (1–4 weeks, with regular meetings). Every sprint contains some form of analysis, design, implementation, evaluation and planning for the next steps. This enables the scrum team to deliver a first version of the product in the earliest stage of the development process. Moreover, it creates constant feedback loops involving all end-users; i.e. end-users reflect upon every version of the product.

We applied scrum to the development of ListeningTime in the following way: a sprint was defined as two weeks and six sprints were planned by the software company, in agreement with the researchers. The researchers were the initiators of ListeningTime (the product owners and the development team of the content) and contacted an external software company to develop the technical part of the online intervention (the software development team). The scrum team consisted of two researchers (JN and JD) and three employees of the external software company (scrum master, software engineer and designer).

During the two weeks of a sprint, members of the scrum team worked on the product components within their own expertise. As many as possible end-users were involved during each sprint; i.e. (representatives of) elderly (former) cancer patients and oncological HCPs. At the end of the two weeks, a meeting with the scrum team was planned, to present the product components and to discuss with the team members and the end-users. Next, appointments for the next sprint(s) were made. The aim was to deliver a working web-based preparatory communication tool at the end of the sixth sprint. Moreover, through participatory development we aimed to increase the uptake and use of ListeningTime.

ListeningTime was based on a similar intervention designed for patients with malignant lymphoma (van Bruinessen et al., 2014). The previously developed intervention contained tailored simulated video-fragments of patient-provider encounters, among others. For ListeningTime we also used tailored simulated video fragments of patient-provider encounters, but the content and tailoring of the video fragments was based on subjects indicated as relevant by elderly cancer patients and their HCPs.

No specific aims per sprint were defined beforehand, as the aims supposed to be defined during the process based on the previous sprint outcomes. In the result section a detailed description of every sprint as executed is presented, and how the involvement of end-users influenced the development of the tool.

4. Results

Between September and December 2015, four two-week and two three-week sprints with six meetings of ± 1 h between the scrum team and end-users were executed to develop ListeningTime. In addition, meetings between end-users and researchers took place (see details below). The tool was launched in March 2016. See Box 1 for a short description of the final version of ListeningTime.

Box 1. ‘ListeningTime’.

ListeningTime is a website for elderly patients with cancer (≥ 65 years), to help them (better) prepare their encounters with (oncological) healthcare providers (HCPs). The website is designed to help HCPs prepare encounters in a similar way. The website contains two video diaries, with each 12 short video fragments, of simulated patient-HCP encounters. The video fragments demonstrate different communication strategies. Patients are asked to watch a selection of personal relevant video fragments, based on an algorithm. HCPs are asked to watch one entire diary. The website also contains an audio-facility. Patients and HCPs can listen back to their audio-recorded encounter through the facility. Patients and HCPs can access the website anywhere, at any time, with a personal login.

Alt-text: Box 1

4.1. Input for sprints and intervention

Before the first sprint, the authors performed a needs assessment (i.e. interviews) among 14 elderly (ex-)cancer patients (Noordman et al., 2017) and eight oncological HCPs (i.e. 2 oncologists and 6 nurses), to map their communication barriers and needs for supportive interventions. At the end of this needs assessment, one video fragment (about dealing with emotions while being sick) of the previously developed intervention was shown as example (van Bruinessen et al., 2014). The results of the need assessment are described elsewhere (Noordman et al., 2017). The needs assessment provided input for the first sprint. In addition, the researchers used insights from the previous developed similar intervention (van Bruinessen et al., 2014) and other studies for the design of the tool (Henselmans et al., 2013, Roter et al., 2012, Krouse, 2001, Bol et al., 2013). The simulated patients (and their companions) in the video fragments were aged 65 years or older. Also, for the design of the website, guidelines for targeting elderly patients online were followed (e.g. avoiding large amounts of text by using ‘pull out’ menus for more detailed information and larger font) (Bolle et al., 2016).

4.2. First sprint

The first sprint started with a ‘brown paper session’, to map user requirements of end-users. The scrum team mapped user-requirements based on results of the needs assessment.

On a brown paper, sticky notes were taped with intervention requirements of elderly cancer patients (pink notes), oncological HCPs (orange notes) and requirements for the researchers (yellow notes). These requirements were divided in four sub-categories: ‘registration’ (login, information about the intervention, informed consent), ‘before the encounter’ (e.g. algorithm/tailoring, watching simulated videos), ‘the encounter’ (e.g. audio recording encounters, making use of the intervention) and ‘after the encounter’ (audio-facility to listen back to recorded encounter). When requirements of end-users overlapped (e.g. login, informed consent), the sticky notes were placed upon each other.

Next, a ‘product backlog’ (i.e. shared excel document) was created by the team in which requirements of the intervention were defined (what, when, for whom) and prioritized per requirement (must, should, could, won't have). Requirements of end-users that overlapped were prioritized as must. The requirements of patients were defined as most important, followed by the needs of the HCPs and finally the priorities of the researchers. However, most of the time we could include all requirements. For example, the patients required a ‘tailored’ intervention, while the HCPs did not mention this as requirement. As scrum team we decided therefore to tailor the content for patients and provide general content for HCPs.

At the end of this sprint, researchers invited end-users to participate during the next sprints.

4.3. Second sprint

During the second sprint meeting, the product backlog was discussed with the scrum team; i.e. are the requirements and prioritizing still correct or changed. For example, implementation of the tool in practice (in case of a positive evaluation) was marked as a could by the software developers, but defined by the researchers as a must. By including an implementation plan from the start (as a must) we aimed to avoid adoption problems of the intervention. Also, the design of the tool was briefly discussed. For example, as many elderly people make use of tablets instead of PCs, the tool should be also designed for use on tablets. The first features of ListeningTime were built by the software company, but were not ready for demonstration to the scrum team or end-users. Therefore, participation of end-users during this sprint was considered not useful.

4.4. Third sprint

During the third sprint, software developers (n = 3) shared a first concept of ListeningTime, including the design, with the researchers (n = 2). Furthermore, a meeting took place between a researcher (n = 1) and the representatives of the patient and the provider organizations (n = 2). These representatives provided feedback on the concept of ListeningTime (content and lay-out). They suggested for example to offer the audio-facility (to listen back to their encounters) not only to patients but also to HCPs. Furthermore, they agreed upon the importance of tailoring the content of the intervention for patients (i.e. video fragments), but suggested that HCPs should be able to watch all the video fragments as they see different patients. Their feedback was forwarded to the software company. Moreover, implementation arrangements were made for ListeningTime to become publicly available after the study, in case of a positive evaluation.

4.5. Fourth sprint

The fourth sprint was marked by the development of video diaries. Two video diaries were created, consisting of 12 ‘modelling’ fragments (0.34–2.59 min per fragment). In the fragments different communication strategies of simulated patient-provider encounters were demonstrated. One diary displays the story of a male patient with prostate cancer (and spouse), the other diary the story of a female patient with malignant lymphoma (and spouse), visiting their oncologist. Two cameras captured the perspective of the patient and the provider, respectively, which was translated in similar diaries for patients and providers from a different viewpoint. The simulated patients (and their companions) in the video were aged 65 years or older.

The scripts for the diaries were based on personal stories expressed during the needs assessment (Noordman et al., 2017) and previous research (van Bruinessen et al., 2014). Before recording, the scripts were reviewed by end-users: six (former) patients, one representative of a patient- and one of a provider organization and two researchers/communication experts. The content of the scripts was changed to some extent (e.g. more appropriate wording) and some subjects were included (e.g. fear of death). Next, minor alterations were made in the scripts after a rehearsal session with the simulated patients (n = 4) and providers (n = 2). Selected communication topics were, e.g., dealing with emotions, asking questions, need for support, and sharing information. Specific topics of importance for elderly patients were also included in the fragments, e.g. quality of life, fear of death and intimate issues. One fragment could include a communication skill (e.g. asking questions) and communication topic(s) (e.g. question about intimate issues). Each video ends with a simulation question (e.g. What do you do when medical information is not clear?/What do you do, so that the medical information is understandable for patients?). An algorithm was constructed to select four personal relevant video-fragments of one diary for a patient. The algorithm was based on patients' answers to questions about their confidence in communication with their oncological healthcare provider (using the PEPPI (Zandbelt et al., 2006)), the importance of discussing several subjects (e.g. quality of life, intimate issues) and their sex (male/female). The first two fragments contain an introductory narrative of the simulated patient (and the companion). To prepare the consultation, patients were asked to watch these six fragments, with the possibility to watch the entire diary (or diaries). Providers were asked to watch all video fragments of one diary, with the possibility to watch both diaries (see Table 1). Although the video fragments are part of one diary, they are developed to be watched individually (i.e. a personal relevant set of four fragments is selected through the algorithm). Furthermore, fragments from the two diaries can be mixed and there is no specific correct order in watching them, as this depends on the preference of end-users.

Table 1.

Overview of the content and techniques of ‘ListeningTime’.

Section Content Technique
Homepage Introduction of the website; Register/login for patients and providers –
About ListeningTime Aim of ListeningTime; Background; Videos; Audio-recordings; Registration –
Patients Information for patients: evaluation of ListeningTime; what does participation mean?; data gathering and privacy; medical ethics committee; what if I don't want to participate?;questions? –
Healthcare providers Information for healthcare providers: what does participation mean?; data gathering and privacy; medical ethics committee; what if I don't want to participate?; questions? –
Contact Contact information: who are we?; participating hospitals; cooperating (patient and provider) organizations; financial support –
Your personal page/after login Informed consent (for research; optional: audio-recording of encounter) –
Pre-questionnaire for patients: background characteristics, COOP/WONCA(Van Weel, 1993, Van Weel et al., 2012),
PEPPI(Zandbelt et al., 2006), importance to discuss subjects.
Pre-questionnaire for providers: background characteristics, adapted PEPPI for providers.		 Algorithm: Based on patients' confidence in communication with their oncological healthcare provider (PEPPI), the importance of discussing several subjects (e.g. quality of life, intimate issues) and their sex (male/female) four video-fragments were selected of one diary. No tailoring/algorithm was created for providers.
Video diaries Video-modelling: Four personal relevant video fragments of one diary were selected for patients. The first two fragments were always the introduction of the simulated patient's (and companion's) story. To prepare the consultation patients were asked to watch the six fragments, and had the option to watch the entire diary (or both diaries). Providers were asked to watch all video fragments of one diary (with the option to watch the other diary). Each video ends with a simulation question (e.g. What do you do when (medical) information is not clear?/What do you do, so that the (medical) information is understandable for patients?)
Audio-facility Audio-facility: An audio file can be uploaded to the patient's account (by the HCP). Patients and providers have the possibility to listen back to their encounter via an audio player. The use of the audio-facility is optional.
Post-questionnaire for patients: evaluation of website, video-fragments and use of (listening back to) audio-recording
Post-questionnaire for providers: evaluation of website, video-fragments and use of (listening back to) audio-recordings		 –
Open in a new tab

4.6. Fifth sprint

During the fifth sprint, researchers (n = 2) and the software company (n = 2) inquired input from an oncological nurse (n = 1), resulting in several small adaptations in content and lay-out. Logistic and practical issues (e.g. automatic emails, connection of audio-recordings to patient and provider) were also discussed for the evaluation and implementation in of the tool. During sprint 1 to 5, researchers delivered all the content for the website, plain texts, the questionnaires, the video diaries, simulation questions and the algorithm for the video diaries to the software developers (see Table 1).

4.7. Final sprint

During the final (6th) sprint meeting, the software developers presented the (almost) final version of ListeningTime. The researchers invited (former) patients (n = 5), ICT experts (n = 2) and a representative of the patient organization (n = 1) to perform usability tests on this final version. A heuristic evaluation (expert-based) and a think-aloud procedure (user-based) were set up with these participants, which was considered to be enough to detect over 80% of the usability problems (Nielsen, 1994). They were asked to navigate through the website ListeningTime and to answer several questions; e.g. What is your impression of the website? Could you easily login? Do you miss something? Based on these usability tests several, mainly textual, adaptations were made. We did not receive fundamental feedback from end-users about the content of the videos or the selection of the videos through the algorithm. Before the website became online available (as planned in January 2016) researchers tested the website once more (in different browsers) and found out that some aspects did not work properly, e.g. algorithm was not always correct and .wav files could not be uploaded as audio-file. As of March 2016, ListeningTime has been operational. Fig. 1 presents a screenshot of the homepage.

Fig. 1.

Fig. 1

Open in a new tab

Screenshot of the homepage of the website.

4.8. Next step

Patients with cancer (≥ 65 years) will be approached, through the patient panel ‘kanker.nl’, to evaluate ListeningTime through an online questionnaire. In addition, ListeningTime will be evaluated in three Dutch hospitals by elderly patients with cancer and their oncological healthcare providers. Results of these evaluations will be used to finally adapt ListeningTime. In case of positive evaluation, the tool will become publicly available through the website of the Dutch Cancer Society (‘kanker.nl’).

5. Discussion

5.1. Lessons learned & study limitations

At the end of the six sprints ‘ListeningTime’, a web-based preparatory communication tool for elderly cancer patients and their HCPs, was developed with input from end-users within the planned timeframe. However, the input from end-users was not possible during every sprint and the total sample size of involved end-users was small. This could influence the uptake and use of the intervention.

After a few sprints, it appeared that sprints of two weeks were not feasible and sprint duration was adapted to three weeks. Constantly involving elderly cancer patients and their HCPs is challenging. Not only because of time constraints (for all parties), but also the burden for patients to regularly participate when being seriously ill and/or elderly. Some patients participated during several sprints and therefore became ‘co-researchers’ or ‘research partners’ in the development process, having a decisional role (Symposium Panta Rhei, TU Delft). Most patients provided input during one sprint (or during the needs assessment) and had a more advising role at a particular stage in the development process. Most HCPs participated once, only one HCP participated twice. It would be interesting to explore the influence of these different degrees of participating in future studies. Another question one needs to ask, when involving end-users using scrum, is if these end-users (i.e. patients and providers) are able to reflect upon a tool ‘in development’. Especially early in the process, ‘ListeningTime’ consisted of bits of (technical) information and the lay-out was not very presentable. This made it difficult to see how a ‘final’ version of ListeningTime could become. In addition, end-users may not know what kind of intervention they exactly prefer or need until it is right in front of them. During sprint four the video diaries were created, end-users reflected upon the scripts of these videos and we had a rehearsal session to adapt the scripts. For future studies it would be interesting to ask end-users to also reflect on the recorded videos, next to the scripts, and then record for a second time. This was not possible for this project due to time constraints and costs.

Although researchers and software developers both had the aim to develop a user-centered tool, the requirements of the software developers did not always coincide with scientific ones. The short sprints were useful for the software developers, but a challenge for the researchers as they had to develop and deliver all the content materials and recruit and involve end-users at the same time. In addition, for scientific research it is important to include a representative sample, as far as conceivable, of patients and providers to develop the tool. Because of the short sprints this was not always possible. A limitation of this study is that we included mainly male (former) patients with prostate cancer, and also the more empowered patients are probably overrepresented. In addition, it was very difficult to include oncological care providers, especially oncologists, because of their busy schedule. Time constraints were also the reason that end-users did not meet with the software developers but only with the researchers, except for one HCP during the fifth sprint.

Involved patients, HCPs and their representatives were positive about (the way of) involvement in developing the tool. Attracting involved end-users may be a precondition in developing a successful intervention (van Bruinessen et al., 2014). However, it is also important to develop an intervention that reaches out to the entire population of end-users. Similar to a previous study (van Bruinessen et al., 2014), we found that early involvement of end-users and flexibility in terms of planning and setup seem to be preconditions for creating a bottom-up inspired development procedure with (seriously ill) patients, and also (oncological) HCPs.

Altogether, scrum may not be the best framework for developing an innovative tool like ListeningTime. This was also debated in a recent symposium, where they concluded that scrum and working agile ‘hinders or obstructs true innovation’, because of too much focus on quick development and learning by doing (Arnstein, 1969). The recently published IDEAS framework (Mummah et al., 2016), may be an interesting alternative for co-creation between non-profit scientific research and for-profit (development) companies, involving end-users. According to the authors: “the IDEAS framework strives to provide sufficient detail without being overly prescriptive so that it may be useful and readily applied by both investigators and industry partners in the development of their own mHealth, eHealth, and other digital health behavior change interventions” (Mummah et al., 2016). Nevertheless, the ‘pressure cooking situation’, by using scrum, resulted in a quick development process and a ‘final’ product. In contrast to traditional (research) methods where more time investment is needed for perhaps a similar result.

6. Conclusions

For the development of the web-based preparatory communication tool ‘ListeningTime’ we used a scrum framework to involve end-users. Elderly (former) cancer patients, oncological HCPs and their representatives participated in the development of the tool in several ways. This collaboration with end-users influenced the development process. Several challenges emerge from using scrum as participatory framework.

ListeningTime is developed for elderly cancer patients and their oncological HCPs to help them (better) prepare the consultation and overcome communication barriers. The evaluation and implementation of the tool should make clear whether or not this is accomplished in practice.

Conflicts of interest

The authors have no conflict of interest to declare.

Ethics

This study was carried out according to Dutch privacy legislation. According to Dutch legislation, approval by a medical ethics committee was not required. Participation was voluntary and participants gave their informed consent at the start of their participation.

Acknowledgement

The study was funded by the Dutch Cancer Society (KWF Kankerbestrijding: NIVEL 2014-7271). We would like to thank all (former) patients, oncological healthcare providers, the Nederlandse Federatie van Kankerpatiënten organisaties (NFK) and the ‘Quality institute for oncological and palliative research and practice’ (IKNL: Integraal Kankercentrum Nederland) for their involvement in the development of ListeningTime.

References

  1. Arnstein S.R. A ladder of citizen participation. J. Am. Inst. Plann. 1969;35:216–224. [Google Scholar]
  2. Bartholomew L.K., Parcel G.S., Kok G. Intervention mapping: a process for developing theory- and evidence-based health education programs. Health Educ. Behav. 1998;25:545–563. doi: 10.1177/109019819802500502. [DOI] [PubMed] [Google Scholar]
  3. Bol N., Smets E.M., Rutgers M.M., Burgers J.A., de Haes J.C.J.M., Loos E.F., van Weert J.C.M. Do videos improve website satisfaction and recall of online cancer-related information in older lung cancer patients? Patient Educ. Couns. 2013;92:404–412. doi: 10.1016/j.pec.2013.06.004. [DOI] [PubMed] [Google Scholar]
  4. Bolle S., Romijn G., Smets E.M.A., Loos E.F., Kunneman M., van Weert J.C.M. Older cancer patients' user experiences with web-based health information tools: A think-aloud study. J. Med. Internet Res. 2016;18 doi: 10.2196/jmir.5618. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Cegale D.J. Emerging trends and future directions in patient communication skills training. Health Commun. 2006;20:123–129. doi: 10.1207/s15327027hc2002_3. [DOI] [PubMed] [Google Scholar]
  6. de Beurs D., van Bruinessen I., Noordman J., Friele R., van Dulmen S. Active involvement of end users when developing Web-based mental health interventions. Front. Psychiatry. 2017;8(72) doi: 10.3389/fpsyt.2017.00072. (May 03) [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Eysenbach G. The law of attrition. J. Med. Internet Res. 2005;7 doi: 10.2196/jmir.7.1.e11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Fowler M., Highsmith J. The agile manifesto. Softw. Dev. 2001;9:28–35. [Google Scholar]
  9. Green S.A., Evans L., Matthews R., Jayacodi S., Trite J., Manickam A., Evered R., Green J., Williams J., Beveridge E., Parker C., Tiplady B. Service user engagement in quality improvement: applying the national involvement standards. J. Ment. Health Train. Educ. Pract. 2016;11:279–285. [Google Scholar]
  10. Hack T.F., Ruether J.D., Weir L.M., Grenier D., Degner L.F. Promoting consultation recording practice in oncology: identification of critical implementation factors and determination of patient benefit. Psychooncology. 2013;22:1273–1282. doi: 10.1002/pon.3135. [DOI] [PubMed] [Google Scholar]
  11. Henselmans I., de Haes H.C., Smets E. Enhancing patient participation in oncology consultations: best evidence synthesis of patient-targeted interventions. Psychooncology. 2013;22:961–977. doi: 10.1002/pon.3099. [DOI] [PubMed] [Google Scholar]
  12. Jansen J., van Weert J.C.M., de Groot J., van Dulmen S., Heeren T.J., Bensing J.M. Emotional and informational patient cues: the impact of nurses' responses on recall. Patient Educ. Couns. 2010;79:218–224. doi: 10.1016/j.pec.2009.10.010. [DOI] [PubMed] [Google Scholar]
  13. Krouse H.J. Video modelling to educate patients. J. Adv. Nurs. 2001;33:748–757. doi: 10.1046/j.1365-2648.2001.01716.x. [DOI] [PubMed] [Google Scholar]
  14. Mummah S.A., Robinson T.N., King A.C., Gardner C.D., Sutton S. IDEAS (Integrate, Design, Assess, and Share): a framework and toolkit of strategies for the development of more effective digital interventions to change health behavior. J. Med. Internet Res. 2016;18 doi: 10.2196/jmir.5927. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Nielsen J. Estimating the number of subjects needed for a thinking aloud test. Int. J. Hum. Comput. Stud. 1994;4:385–397. [Google Scholar]
  16. Noar S.M., Black H.G., Pierce L.B. Efficacy of computer technology-based HIV prevention interventions: a meta-analysis. AIDS. 2009;23:107–115. doi: 10.1097/QAD.0b013e32831c5500. [DOI] [PubMed] [Google Scholar]
  17. Noordman, J., Driesenaar, J.A., Henselmans, I., Verboom, J., Heijmans, M., van Dulmen, S., 2017. Patient participation during oncological encounters: barriers and need for supportive interventions experienced by elderly cancer patients. (under review) [DOI] [PubMed]
  18. Omeni E., Barnes M., MacDonald D., Crawford M., Rose D. Service user involvement: impact and participation: a survey of service user and staff perspectives. BMC Health Serv. Res. 2014;14(1):491. doi: 10.1186/s12913-014-0491-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Posma E.R., van Weert J.C.M., Jansen J., Bensing J.M. Older cancer patients' information and support needs surrounding treatment: an evaluation through the eyes of patients, relatives and professionals. BMC Nurs. 2009;8:1. doi: 10.1186/1472-6955-8-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Rademakers J. [Knowledge Synthesis. Health Skills: Not for Everyone]. NIVEL; Utrecht: 2014. Kennissynthese. Gezondheidsvaardigheden: niet voor iedereen vanzelfsprekend. [Google Scholar]
  21. Roter D.L., Wexler R., Naragon P., Forrest B., Dees J., Almodovar A., Wood J. The impact of patient and physician computer mediated communication skill training on reported communication and patient satisfaction. Patient Educ. Couns. 2012;88:406–413. doi: 10.1016/j.pec.2012.06.020. [DOI] [PubMed] [Google Scholar]
  22. Schwaber K., Sutherland J. Scrum Alliance; 2011. The Scrum Guide. [Google Scholar]
  23. Symposium Panta Rhei, TU Delft http://www.emerce.nl/opinie/teveel-focus-op-agilescrum-staat-echte-innovatie-in-de-weg (Accessed November 17, 2016)
  24. van Bruinessen I.R., van Weel-Baumgarten E.M., Snippe H.W., Gouw H., Zijlstra J.M., van Dulmen S. Active patient participation in the development of an online intervention. J. Med. Internet Res. 2014;3 doi: 10.2196/resprot.3695. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. van Gemert-Pijnen J.E., Nijland N., van Limburg M., Ossebaard H.C., Kelders S.M., Eysenbach G., Seydel E.R. A holistic framework to improve the uptake and impact of eHealth technologies. J. Med. Internet Res. 2011;13 doi: 10.2196/jmir.1672. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Van Weel C. Functional status in primary care: COOP/WONCA charts. Disabil. Rehabil. 1993;15:96–101. doi: 10.3109/09638289309165878. [DOI] [PubMed] [Google Scholar]
  27. Van Weel C., König-Zahn C., Touw-Otten F., van Duijn N.P., Meyboom-de Jong B. Noordelijk Centrum voor GEzondheidsvraagstukken (NCG)/Northern Centre of Health Care Research (NCH); Groningen, the Netherlands: 2012. Measuring Functional Status with the COOP/WONCA Charts. A Manual. [Google Scholar]
  28. Watson P.W., McKinstry B. A systematic review of interventions to improve recall of medical advice in healthcare consultations. J. R. Soc. Med. 2009;102:235–243. doi: 10.1258/jrsm.2009.090013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Winterling J., Wiklander M., Obol C.M., Lampic C., Eriksson L.E., Pelters B., Wettergren L. Development of a self-help web-based intervention targeting young cancer patients with sexual problems and fertility distress in collaboration with patient research partners. JMIR Res. Protoc. 2016;5(2) doi: 10.2196/resprot.5499. (12) [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Zandbelt L.C., Smets E.M., Oort F.J., Godfried M.H., de Haes H.C. Determinants of physicians' patient-centred behaviour in the medical specialist encounter. Soc. Sci. Med. 2006;63:899–910. doi: 10.1016/j.socscimed.2006.01.024. [DOI] [PubMed] [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES