The Usefulness of the ASSUSTENT Application and ASSIST Brochure in Cancer Patients Using Sunitinib

Abstract

Purpose of Research

The ASSUSTENT application and the ASSIST brochure have been developed to support medication intake and symptom monitoring. This study aimed to evaluate patient experiences and the factors that are a barrier to or facilitate the use of these tools. Additionally, the effect of their use on Health-Related Quality of Life (HRQoL) and satisfaction with information about medication was also assessed.

Methods

An exploratory study with a mixed method design was performed. Patients starting or already using sunitinib were asked to use the application or the brochure for 6 months. They completed questionnaires about their experiences with the intervention, that is, the Satisfaction with Information about Medication scale (SIMS) and the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC QLQ-C30) at baseline (T0), three months (T1), and 6 months (T2) following inclusion. Patients were also asked to participate in a semi-structured interview at T2. The main study endpoint was the feasibility of the use of the application and the brochure.

Results

Of the 22 (65%) patients who signed the informed consent, 19 (86%) completed T0, 15 (68%) T1, and 12 (54%) T2. Twelve agreed to be interviewed. Both the application and brochure were considered user friendly and useful to manage symptoms and prepare for consultations. Patients were generally satisfied with the information about medication. The mean global HRQoL increased from 69 (T0) to 84 (T2).

Conclusion

As supplements to usual care, both the application and the brochure met the needs of cancer patients using sunitinib. Their use led to an increased self-efficacy in managing symptoms. The availability of the brochure adds to patient-centered care and equal access to care, and increases self-efficacy.

Introduction

Sunitinib is an oral anticancer agent (OACA) acting as a tyrosine kinase inhibitor preventing tumor growth and progression of metastases. ¹ It is used in the treatment of metastatic renal carcinoma, gastrointestinal stromal tumor (GIST) after imatinib failure, and pancreatic neuro-endocrine tumors (patients having a contraindication with immunotherapy).^2,3 Sunitinib is given once daily according to a patient’s response to treatment, with a 14 day period off-treatment after four weeks off-treatment, or continuously. ²

As sunitinib is an OACA that has a complex dosing regimen, causes serious side effects, and is self-administered at home, administration errors and both intentional and unintentional non-adherence may occur.^4,5 During treatment, patients experience both sunitinib-induced side effects and disease-related symptoms. Side effects include fatigue, nausea and vomiting, hand-foot syndrome, diarrhea, insomnia, skin complaints, loss of appetite, dizziness, and arthralgia. ² Usually they are mild to moderate and well managed with supportive measures. ⁶ Their occurrence, however, may decrease Health-Related Quality of Life (HRQoL) and lead to non-adherence, which in turn is likely to affect treatment outcomes.^7-9

Patient reported outcomes (PROs) are ‘direct reports from patients about the status of their health condition without amendment or interpretation by others’, while patient reported outcome measures (PROMs) ‘are the tools used to measure PROs’. ¹⁰ PROMs have shown to improve the symptom management of cancer patients. ⁷ When PROs were shared directly with health care professionals (HCPs), the use of PROMs also improved HRQoL and overall survival.^11-13 In a recent study it was observed that PROs were also effective in improving HRQoL when patients were activated to share them with their HCPs (reactive approach). ¹³

Mobile health applications are increasingly used as a convenient means to provide treatment information, promote medication adherence, monitor side effects and collect PROs.^7,10,14-19 Although communicating with HCPs and collecting PROs by means of an application seems simple and easy, the widespread use of applications is still hampered by several factors. They include difficulties of patients in using the application and a high rate of dropouts, inadequacies of the IT infrastructures supporting PRO collection and the processing of PROs into Electronic Health Records (EHR). Other difficulties comprise organizational issues preventing the full incorporation of electronic PROMs and the use of PROs in daily clinical practice.^7,10,17-19 Moreover, some patients cannot or for various reasons object to using applications. Therefore, it remains necessary to be able to collect PROs with printed media as well.^7,10,20-23 Although numerous applications are presently available, very limited research has been conducted into their real world use, patient preferences, and integration into routine care with regard to the features offered.^7,18,24

The ASSUSTENT application and ASSIST brochure (brochure) have been developed to support patients using sunitinib by: 1) facilitating medication adherence through the registration of medication intake (both), 2) assisting in side effect management by enabling patients to track side effects experienced (both), 3) providing general information about the medication and potential side effects (brochure) as well as specific details about the side effects experienced (application), and 4) allowing patients to generate reports of their medication intake and experienced side effects (application).

The present study aimed to investigate patient experiences and the factors that are a barrier to or facilitate the use of these tools as supplements to usual care in cancer patients initiating treatment or already using sunitinib. The effects on HRQoL and patient satisfaction with the information provided about the medication and its use were also assessed.

Patients and Methods

Design

An exploratory intervention study with a mixed-methods design, using both quantitative (questionnaires) and qualitative data (semi-structured interviews) on the feasibility of the use of the ASSUSTENT application and/or ASSIST brochure was conducted. ²⁵ The effects on HRQoL and patients’ satisfaction with medication information in patients starting or already on sunitinib were assessed by means of a questionnaire. To assess the experiences of patients, the questionnaire at T2 included questions about the usability, the utility, and the overall usefulness. In addition, semi-structured interviews were performed at T2 (Figure 1).

Figure 1.

Timeline of the Study.

Interventions: The ASSUSTENT Application and Brochure

The application and brochure were developed by Pfizer Inc. and were specifically designed to support cancer patients using sunitinib. Both interventions collected PROs. The application (available for iOS and Android) and the brochure are available free of costs from the companys' website, Apple App store, and Google Play store. All patient data was stored locally on the patients' device. Patients were asked to use the application or the brochure for a 6-month period.

ASSUSTENT application key functionalities included (Figure 2): (optional) reminders (medication intake and HCP consultations); medication use (medication schedule and intake [yes or no], symptom and general well-being reporting (6-point scales) and generating overall reports (medication intake, experienced symptoms and general well-being by producing schematic overviews over time); providing information (tips and advice about symptom relief); possibility to make comments. All information can be downloaded in a PDF for the patients’ use.

Figure 2.

Screenshots of the ASSUSTENT Application.

Patients not willing or unable to use a digital tool, could opt for using the ASSIST brochure instead of the application. Key functionalities included: medication use (medication schedule and intake [yes or no], symptom and general well-being reporting (severity on a 4-point Likert scale and on a 3-point Likert scale, respectively), providing general information (about most common symptoms relief), extra notes (hospital schedule, and space to take notes and or questions).

Sample

Patients with cancer aged 18 years or older, who were starting or on treatment with sunitinib, were asked by phone or during a consultation by either their outpatient pharmacist or their oncologist whether they were willing to participate in the study. A researcher sent the information of the study, the informed consent form and a return envelope by post to the patient. Patients were asked to complete and return the informed consent form. Before inclusion, after returning the inform consent form, patients who were able and willing to use a smartphone were asked to use the ASSUSTENT application. Others were asked to use the ASSIST brochure. The aim was to include 50 patients. Patients were recruited from July 2020 to April 2022 from six hospitals in the Netherlands.

Measurements and Data Collection

Quantitative data were collected digitally by means of Survalyzer or manually by post, according to patients’ preferences.

Patient Reported Outcomes, Quantitative Data

Health related quality of life

The validated EORTC QLQ-C30 questionnaire was used to evaluate HRQoL.^26,27 The 30-item questionnaire incorporates a global health status, five functional scales (physical, role, emotional, cognitive, and social), three symptom scales (fatigue, nausea/vomiting, and pain), five single items for additional symptoms (dyspnoea, insomnia, appetite loss, constipation, and diarrhea) and one question about the financial impact of the disease. Each item was scored on a 4-point Likert scale (1 = not at all to 4 = very much). Scale scores were linearly transformed into a scale of 0 to 100 according to the original scoring manual. Higher functional, summary score and global health status/HRQoL scores represent a better HRQoL and higher symptom scores indicate greater symptom severity.^26,28 Clinical relevance for global QoL was determined according to the mean difference guideline suggested by Cocks et al. ²⁹

Reliability of the EORTC QLQ-C30 questionnaire was satisfactory with a Cronbach’s alpha coefficient >0.70. Validity was also considered sufficient, because all interscale correlations were statistically significant. The correlation was moderate, indicating that the scales were assessing distinct components of the HRQoL construct. In addition, results differed by clinical status and interventions resulted in changes into the expected direction. ²⁶

Satisfaction With information about Medicines

The SIMS questionnaire has been validated to measure patients' satisfaction with information about the medication used. ³⁰ The SIMS has a total score ranging from 0-17 and can be subdivided into two subscales; “action and usage (AU)” (score 0-9) and “potential problems from medication (PP)” (score 0-8). Items 1-9 (AU) address how the medication is used and how it works, while items 10-17 (PP) cover side effects, interactions and problems. Responses “about right” and “not needed” indicate satisfaction, while “too much”, “too little” and “not received” indicate dissatisfaction.^30,31 Higher scores indicate greater satisfaction with the information received.

The SIMS was well accepted by patients in a variety of clinical settings and showed satisfactory internal consistency and test-retest reliability. The Cronbach’s alpha coefficient was 0.88 in oncology patients. ³⁰

Patient Reported Experiences, Quantitative and Qualitative Data

The questionnaire at T2 also contained one question on the innovation domain and six on the user domain, which could be answered on a 5-point Likert scale (ranging from “totally disagree” to “totally agree”). An additional question on the user domain could be answered on a 11-point scale (ranging from “very good” to “very bad”).

The interview topic list (see Appendix 1) and the questionnaire (see Appendix 2) were created by the research team and were based on the Measurement Instrument for Determinants of Innovations (MIDI) developed by Fleuren et al. ³² The MIDI is categorized into four domains: the innovation, the user, the organization and the socio-political context. In this study, only topics from the innovation and user domain were assessed and additional study-specific topics and questions were added.

Semi structured interviews were conducted by two researchers (VT and EH), guided by a topic list for both the application and brochure. All interviews were held online, adhering to the COVID-19 guidelines at the time of the study and lasted between about 17 to 51 mins. The focus was on patient experiences and the identification of facilitators and barriers related to the accessibility, features and daily use of the application or brochure.

Data Analysis

Quantitative Data

Descriptive statistics were used for patient and disease characteristics. Data were presented as frequencies for categorical variables, means and standard deviation (SD) for normally distributed continuous variables, and medians and interquartile range (IQR) for skewed continuous variables. The EORTC-QLQ C30 subdomains were scored according to the manual of the EORTC and compared with the thresholds for clinical importance (TCIs). ³³ The mean of the summary score and global health score at T0 and T1 (3 months) and T0 and T2 (6 months) were compared using a paired sample t-test.

Variables for SIMS were dichotomized into satisfied vs unsatisfied before analysis. Differences between the T0, T1 and or T2 were assessed using a Chi-Square test for categorical data and an independent t-test for continuous, normally distributed data.

For all analyses, a two-tailed significance level of 0.05 was used and P-values below this level were considered statistically significant. All quantitative data were analyzed using SPSS version 28.0 for Windows.

Qualitative Data

The interviews were recorded and transcribed verbatim. Transcripts were analyzed thematically by three researchers identifying facilitators and barriers according to the Measurement Instrument for Determinants of Innovations (MIDI). Inconsistencies were discussed with a fourth researcher. Study results have been reported by displaying the facilitators and barriers identified in the innovation and user domains of the MIDI, as these domains were applicable to the present study. ³²

Results

Patient Demographics

Thirty-four patients were approached of whom 22 (65%) signed the informed consent. Nineteen (86%), (14 used the application; 5 used the brochure) completed T0, 15 (44%) (10 used the application; 5 used the brochure) completed T1, and 12 (35%) (8 used the application; 4 used the brochure) completed T2 (Figure 3). Reasons for dropouts from study included: lack of time, not interested, discontinued treatment, loss to follow-up and not being satisfied with the application and its use. Patients who participated had a mean age of 65 years (SD = 14.4). Most patients were male (79%), were living with a partner (90%), had a low-middle education level (74%) and were not in a paid work situation (68%). About half of the patients (52%) had renal cell cancer followed by GIST (32%) colon (11%) and pancreatic neuro-endocrine tumors cancer (5%).

Figure 3.

Patient Inclusion: Application Users vs Brochure Users; App = Application.

Patient Reported Outcomes

SIMS

Fifteen patients completed the SIMS questionnaire at T1, and 12 at T2. Of these patients, 83.9% and 89.2%, respectively, were satisfied with the information about their medication. Satisfaction with action and usage (AU), decreased slightly with 3.4% between T1 and T2 (86.7% and 83.3%), but satisfaction with information on potential problems increased with 5% between T1 and T2 (86.7% and 91.7%). These differences were not statistically significant.

HRQoL

The mean summary score increased from 82 (SD = 13.5) to 86 (SD = 7.8) (T0-T2) while the mean global health score increased from 69 (SD = 21.7) to 84 (SD = 12.5) (T0-T2) (scale 0-100). This 15-point difference in the global health score is considered of medium clinical relevance according to Cocks et al. ²⁹ Statistical significance was found for the increase in global health score between T0-T2, indicating an improvement in HRQoL after 6 months (T0-T2, P < .05). The mean scores of the functional scales were all above the threshold for clinical importance (TCI). Of the symptom scales, those for nausea and diarrhea exceeded the TCI (Figure 4).

Figure 4.

(a) Summary Score and Global Health Score for HRQoL. (b) Functional Scales including TCIs. Threshold refers to the Thresholds for Clinical Importance. ³³ (c) Symptom Scales including Thresholds for Clinical Importance. Threshold refers to the Thresholds for Clinical Importance. ³³

Subgroup Insights From Interviews

Patient Experiences With the ASSUSTENT Application and Brochure

Twelve (63%) patients were interviewed (mean age 63 [SD = 13.6]), nine used the application and three the brochure. All quotes are presented in Table 1. Facilitators and barriers divided over the innovation and user domains are presented in Table 2 for the application and in Table 3 for the brochure.

Table 1.

Interview Quotes.

Number	Quote
1	“I think it is a user-friendly application” (120005)
2	“And then of course the application is fantastic, because it even makes graphs, … so then you can see, how the development (of the side effects) is. (170001)”
3	“It actually takes only a few minutes or so, and because I already have that thing (medication) regularly. It is actually 3 clicks and you are done” (120006)
4	“If you select all the side effects together, you have a lot of (overlapping) lines with many colors
	So you cannot see a specific part of them. Only if you have 4 or 5 of the side effects, then it is clear enough” (200001)
5	“You cannot change it, like I did last time. It works for 4 weeks and then 2 weeks rest and again if I wanted to change something, it was not possible, I tried. So I deleted it (the application) and I installed it again” (160002)
6	”In the beginning I had some problems with recording the data, I thought it was fixed and then it turned out to be wrong. Then it got better and better and I think it runs very smoothly” (180004)
7	“With me it is always the same (side effects). And if I’ve read it one or two times I know it […] so for me, if I’ve seen it once or twice or three times it could be turned off” (200001)
8	“Sometimes I had my spouse do that because some of the questions were a bit unclear to me” (200001)
9	“Yes of course, it is very useful, particularly about the side effects. You can also get tips on how to handle them” (160002)
10	“That is exactly how I feel about it, because you basically just look back and see how things went and where the most common side effect is and which most affected you over the past period. You can see that easily when you take it to the clinic” (200001)
11	“Sometimes I have almost no problems. But in the second and third week, the number of side effects increased as it (medicine) increased in the blood, so you will experience that side effect. Yes, and you actually see that every time” (120006)
12	“I have issues with my jaw, that means it is harder for me to eat. Now I have to make an appointment with the dentist and I check (the application and the overview) when, […]. When it is long-term you see precisely, it will probably be less then so it will be okay to schedule the appointment then” (180004)
13	“I find it useful as a kind of diary when I go to the oncologist[…] I can see the report about what I did and how it was determined during that period” (170001)
14	“If you already noted them (the side effects) down, you can let the oncologist see. That is why we were able to go from 50 mg to 37.5 mg, and I feel very good now with less side effects” (150001)
15	“Which is very important for me to have a reminder to know that I really have 4 weeks or 3 weeks from now” (120006)
16	“Yes, I will continue using it. As long as I will be using sunitinib, I would like to continue using it. Yes, to keep an overview of side effects” (120005)
17	“That was only a little effort to fill it in. It is a bit of matter of starting to fill in the dates and so on, and then just keeping track” (180001)
18	“Perhaps you could also consider addition of the blood pressure and saturation” (180002)
19	“I think it was more something to give me some information about the use and benefit of sunitinib
	Yes, I find it useful, like a reference book” (120003)
20	“I always take it with me when I go to the consultation and we could go through it, and the thing that stands out is, that I had high blood pressure and I have not felt it. So we found an explanation for it” (120003)
21	“When we go to the doctor I know exactly when my parent experienced side effects” (180001)
22	“I had problems with sunitinib in the beginning. With taking it 4 weeks then 2 weeks stop” (120003)
23	“I think the booklet is absolutely fine. I did that myself on a paper and with that booklet it will be more handy for me” (120003)

Table 2.

Overview of the Barriers and Facilitators Experienced by the Patients Using the ASSUSTENT Application.

Facilitator	Barrier
Innovation domain
User manual provided useful instructions on how to use the application	Unclear how to change the date so that the report is on the correct date
How to use the application was clear	The application worked slightly different using different (version) operating systems (i.e., android and apple, not specified in user manual)
Symptoms had to be confirmed prior to saving them in the application	The tool did not provide the option to be customized, with respect to
Colors of the smiley’s helped to select the most appropriate grade of the experienced side effect	- Changing the personal profile and or medicine regime
The application was easy to use	- Stop seeing the automatically triggered self-management advice after a few times
It did not take too much time to use the tool	- Including medication dosage on report
	- Adding heart rate and or blood pressure
	- Adding side effects in ‘stop period’
	Use of the application was perceived as complex
	When the report presented graphs for multiple side effects, it layered them on top of each other. This approach was perceived as unclear
	Graphs in the report were too small to properly display on a mobile
User domain
The report provided insights into past and/or current symptoms	The application was not perceived as useful in case of stable and unchanging symptoms
The tool helped to prepare for (or provided input during) a consultation	HCP showed no interest in tool when discussing it during a consultation
The application provides a moment of self-reflection/awareness	Being confronted with the disease daily was not appreciated
The self-management advice following completion of the symptom list was useful	Medication intake reminder was not necessary
HCP were positive or neutral about patients using the tool	No possibility to be monitored by HCPs (with alerts)/or to share the experienced side effects with HCPs automatically
The report provided an overview of the general health condition by summarizing the symptoms	Minor technical issues of the tool in the beginning due to lack of knowledge of the patient
The report provided an overview of patient’s medicine intake and facilitated medication adherence
The tool was helpful for patients in recognizing symptom patterns during the medication course
The tool helped patients to feel more in control about side effects
The provided information about side effects of sunitinib within the tool was appreciated
Keeping track of side effects while using the tool possibly prompted a dose reduction
Possibility to add experienced symptoms, that were not specified in the tool
Help using the application from family members was possible
Using the application has become part of the daily routine

HCP, healthcare provider.

Table 3.

Overview of the Barriers and Facilitators Experienced by the Patients Using the ASSIST Brochure.

Facilitator	Barrier
Innovation domain
The brochure is easy to use	No designated place to note heart rate and or blood pressure
The brochure is clear
User domain
The brochure provides insights into (past and/or current) symptoms	No possibility to be monitored by HCPs/or to share the information with HCPs digitally
The brochure helps to prepare for (or provided input during) a consultation	A method that provided medicine intake reminders was preferred
The brochure helps to notice the symptom pattern during medication course (on-treatment and off-treatment)	The brochure did not provide an overview of the course of symptoms
The brochure provides an overview of the general health condition by summarizing the symptoms provides overview of general condition
HCP was positive about the patient using the brochure
Information about possible side effects of sunitinib in the tool was perceived as more clear than package insert
Help using the brochure from family members was possible
Caregivers could easily have an overview of experienced symptoms
The brochure provided a moment of self-reflection/awareness
Keeping track of side effects while using the tool possibly prompted a dose reduction

HCP, healthcare providers.

Patient Experience: ASSUSTENT Application

Innovation Domain

Most patients participating in the interviews considered the application to be very nice and clear (quote 1,2). As a result of the clear structure of the features, the application was easy to use according to several patients. This was confirmed by the patients who completed the questionnaire: 60% agreed that the application was easy to use (Figure 5). Using the appliction took little time and also fitted well in daily routines (quote 3). Several patients appreciated the manual as a clear and practical support for learning to use the application. One patient reported not needing the manual because they were technical.

Figure 5.

Evaluation of the ASSUSTENT Application.

On the other hand, when multiple side effects were recorded, the overview of side effects was found to be confusing due to overlapping colors (quote 4). Many patients, however, had not noticed that clicking on one particular side effect in the overview produced a display of its development over time. Several patients reported that updating the application profile with regard to dosing and/or stopping frequency/tips and/or tricks frequency was not very easy (quote 5). Difficulties with saving data or generating reports during the initial phase of use were also reported (quote 6). Some patients reported that after a certain period of time the tips and tricks became redundant (quote 7).

User Domain

Patients who completed the questionnaire rated the use of the application at 6.2 at T1 and at 6.0 at T2 on a scale of 0-10. Some patients appreciated the support of family members in using the application (quote 8). Most patients appreciated the regular information provided about the medication as well as the tips and tricks (quote 9). The quantitative data showed similar results (Figure 2).

Most patients indicated that the application supported their treatment well (Figure 2). In the interviews several patients said that the application facilitated self-reflection and provided an overview of general well-being and of symptoms (quote 10). One patient mentioned that in the initial phase of medication use, the application is useful for tracking symptoms to recognize their potential relationship with the medication (quote 11). One patient noted that by keeping track of side effects, they had discovered patterns in the occurrence of side effects which could be used to plan activities accordingly (quote 12).

Several patients used the application to prepare for consultations with the oncologist, others reported using it during a consultation (quote 13). Some patients even reported that their oncologist was positive after having viewed the data collected in the application and the reports produced. In one case tracking side effects resulted in a dose reduction, which subsequently led to fewer side effects (quote 14).

Some patients reported a missed opportunity to be directly monitored by the oncologist. A link to the electronic health record (EHR) might solve this shortcoming of the application.

Several patients indicated that the application supported them to adhere to the complex dosing regimen (periods on- and off-treatment) (quote 15). Some said to have used the application reminder function for this purpose, but other patients continued to use an alarm clock, as they had done prior to using the application.

Study Specific

More than half of the patients indicated that they would continue to use the application after study completion (quote 16). All patients would recommend the application to other patients. Several noted that patients could especially benefit from using the application at the start of sunitinib treatment, when the need for information on medication is most urgent.

Patient Experience: ASSIST Brochure

Innovation Domain

Patients using the brochure reported that the lay out was very clear and therefore easy to use and that it required little effort to record symptoms (quote 17). Patients also reported to miss the option to record values like blood pressure and saturation (quote 18).

User Domain

Patients were positive about the information on medication, the possibility to keep track of experienced symptoms and the tips and tricks to mitigate side effects (quote 19).

Some patients reported that their oncologist was also positive about the brochure. One patient reported to have taken the brochure to the consultation with the oncologist. This resulted in a discussion of side effects followed by a dose reduction (quote 20).

One patient appreciated the possibility for easy family/care giver involvement. The brochure helped the caregiver to keep track of how the patient felt (quote 21).

Some patients missed a reminder function for medication intake and appointments for HCP consultations (quote 22). Patients also reported missing the option of a graphic overview of recorded symptoms as well as their severity, and HRQoL status while using the brochure.

Study Specific

Some patients expressed the wish to continue using the brochure after study completion, and one patient specifically mentioned to recommend the brochure to future patients (quote 23).

ASSUSTENT Application vs ASSISST Brochure

Although the small sample size limited the possibility of statistical testing, some differences revealed some trends. Patients who used the brochure reported slightly higher satisfaction with the medication information at both T1 and T2. By T2, all the brochure users (100%) were satisfied with the information, compared to 87.5% of application users. HRQoL improved in both groups, with the application group showing a slightly greater increase in global health scores (from 68 to 85) compared to the brochure group (from 71 to 83). While both tools were beneficial, application users appreciated features like reminders and symptom tracking visualizations, whereas brochure users highlighted the simplicity and its usefulness for involving HCPs.

Discussion

In this exploratory intervention study, lung cancer patients using sunitinib considered both the application and the brochure to be useful tools for managing side effects and, to a lesser extent, medication adherence. A modest increase in the HRQoL was also observed. These findings align with studies on symptom monitoring using electronic PROMs^{7,13,16,34-36} and of various applications monitoring adherence and symptoms in cancer patients using OACA.^21,37,38

Patients considered both the application and brochure to be clear, easy to use, and quick to complete. The most valued features of the application were the overview of specific symptoms and the ‘tips and tricks’ function for managing them.^{20,21,31,37-39} While the medication reminder was less valued, as most patients had already found effective ways to maintain adherence, the daily reminder to record side effects provided a useful overview of symptom progression. This overview was particularly useful in preparing for routine HCP visits. The tools may also serve as valuable facilitators of shared decision making by allowing patients not only to track and understand their symptoms over time but to provide them with a tool that empowers them to actively participate in conversation and discussions with their HCP. It also helped patients to discover symptom patterns, to adhere to a specific dosage schedule for sunitinib and/or co-medication, or to schedule certain activities. All patients would recommend other patients to use the application and the brochure, especially in the initial phase of treatment, to manage side effects. Thus, in line with evaluations of similar tools, the application and brochure have been shown to be very useful.^{7,20,37,39,40}

Patients rated both the application and the brochure easy to use and experienced no difficulties in familiarizing themselves with the various features. However, they suggested improvements in changing the initial profile information. Several patients indicated that the graphs were unclear when showing multiple side effects simultaneously, due to the overlapping colors. This complexity might have deterred patients from interpreting the graphs. ⁴¹ In this respect, it has been suggested that simple line graphs are preferred over those showing all data at once. ⁴² Therefore, better instructions for using this function must be provided or the application should be adapted to display separate graphs for the symptoms recorded.

Some patients found the medication reminder feature of the application less useful, likely due to their established routines prior to the study. This feature might have been also less beneficial for those already adhering to their regimen, which is consistent with the finding that many patients on OACA do not appreciate reminders.^38,43,44 Redundant features indeed often contribute to the suboptimal use of PROMs.^7,10,22,24 The reminder function, however, could be disabled if its use was deemed unnecessary. On the other hand, the reminder function has been referenced as especially useful for patients initiating sunitinib treatment or those who have difficulty in adhering to the cyclic dosing schedule. This might concern patients with mental distress or patients using several other medications who are more likely to be non-adherent.^40,44,45

Although patients were generally satisfied with the application, it was not linked to the EHR. Some patients therefore regretted that HCPs could not actively monitor side effects, as has been done in several large studies.^12,13,34-36 EHR linkage could provide extra reassurance and safety, particularly in cases of progressive diseases or severe side effects, ensuring a timely call to action.^39,40 In addition, linkage with EHR might facilitate patient-physician relationships as discussions during consultations could be better tailored as HCPs also have real-time insight into the patients’ data. The application also lacked an algorithm to evaluate the severity of side effects and could not alert patients to contact their HCP if this was needed. On the other hand, patients could rate the severity of side effects and access information to manage them. Moreover, HCPs could be contacted by email, e.g. in the case that the self-rated severity of a side effect led to concern. It is also important to note that if an application without EHR linkage and/or an alert function meets the patients’ needs in daily practice, its users may feel sufficiently informed and supported to decide for themselves whether to report a side effect symptom or not.^{13,20-22,37,39,40}

We observed an increase in Global health/HRQoL among patients; however, due to the exploratory nature of the study and the absence of a control group, we cannot definitively attribute this improvement solely to the use of the application or brochure. However, several studies have shown that using PROMs to support cancer patients in managing side effects indeed improved HRQoL, especially if symptoms were monitored by HCPs or if patients were alerted to contact HCPs.^7,12,13,35 Providing adequate information about side effects and their management is crucial for maintaining HRQoL in cancer patients.^7,12,20,31 The application and brochure are likely to have contributed to an improved HRQoL by offering practical information on the management of side effects (‘tips and tricks’ function) and helping patients to understand the evolution of symptoms over time. The overview of side effects also allowed users to plan and prepare for routine consultations with HCPs, and thereby also contributed to HRQoL improvement. The SIMS scores support this relationship since during the study, patients’ satisfaction with information about the medication increased, especially when it came to information about possible problems such as side effects.

The small number of patients participating in the study is a clear limitation. Initially, the intention was to include at least 50 patients. In 2019 RCC treatment guidelines were updated resulting in the replacement of sunitinib by immunotherapy as first-line treatment. ⁴⁶ This change led to a rapid decline in the number of patients treated with sunitinib. Furthermore, very few patients could be included during the COVID-19 outbreaks in 2020. By extending the inclusion period by about two years, 22 patients could ultimately be included. Because of this small sample size and level of drop out at T2, especially the quantitative results of the study need to be interpreted with care. In addition, selection bias may have occurred and information from non-responders is not available. This introduces a limitation for the generalizability of the results. The study should therefore be replicated in a larger sample. Furthermore, the dropout from the study was considerable. This may relate to disease progression and treatment changes or a decreased need to use the application or brochure. In this respect, attrition rates reported in other studies vary widely, from none to over 75%. ⁴⁷ Little research, however, is available on specific factors behind attrition. ⁴⁸ Patient reported reasons for attrition include technical problems, time constraints, feeling fine and failing to meet patients’ needs. ¹⁶ Nevertheless, we believe that due to the mixed methods design of the study the results adequately reflect patients’ views on the usefulness of the application and the brochure. Another strength was the use of the validated EORTC QLQ-C30 questionnaire with the global quality of life score and the summary score as outcomes to obtain a more detailed overview of the effects brought about by using the application and complementing the questionnaire data with information obtained from interviews. Patients could also choose between support via a digital or a paper system, which allowed for the inclusion of digitally illiterate patients and patients not willing to use digital tools. Nevertheless, as the study focused on patient views and preferences, the qualitative data, those of the interviews in particular, appear to provide a reasonable impression of the usability of these supportive tools for cancer patients in general. Finally, the design of this exploratory study does not allow to make a statement about the cost-effectiveness of using the application and the brochure, the more so because these tools are not commercially available but offered free of charge to sunitinib users by the manufacturer of the drug. A more quantitative assessment of the benefits of using the application, such as improved medication adherence and a more efficient health care utilization, also requires a different study design with a larger number of participants.

Conclusion

The results of this exploratory study show that patients treated with sunitinib experienced both the application and the brochure as helpful tools. The usefulness of these tools appears mainly related to their ease of use and the support they provide in managing side effects. In their current form the application and brochure seem particularly suitable for patients initiating sunitinib, as they may help them to adapt to the (cyclic) dosing schedule and manage distressing side effects. The availability of the brochure adds to patient-centered care and equal access to care and increases self-efficacy. Patient satisfaction with the application could be improved if users were given the option to disable the ‘tips and tricks’ function, the usability of the side effects overview were improved and if relevant application data were directly transferred to the EHR. Future research should focus both on the efficacy of these tools in a randomized clinical trial and the use of these tools for patients with other types of cancer.

ORCID iDs

Vashti N. M. F. Tromp https://orcid.org/0000-0001-6086-0668

Mirjam Crul https://orcid.org/0000-0002-4929-2388

Nicole E. Billingy https://orcid.org/0000-0002-3590-4961

Corina J. G. van den Hurk https://orcid.org/0000-0002-7802-1034

Jacqueline G. Hugtenburg https://orcid.org/0000-0001-6098-2438

Ethical Statement

Ethical Approval

The Medical Ethics Committee (METC) of the Amsterdam UMC, location VUMC, confirmed that the Medical Research Involving Human Subjects Act (WMO) did not apply to the study and that their approval was not required (registration number 2019.330). The study was conducted according to the principles of the Declaration of Helsinki (version 64, October 2013).

Informed Consent

Patients were informed about the voluntary nature of the study, and written informed consent was obtained from all the patients prior to data collection.

Data Availability Statement

The authors confirm that the data associated with the publication is available and can be made accessible.*