Abstract
Objectives
To design an electronic questionnaire for symptom monitoring and to evaluate the feasibility, usability and acceptability when applied to patients with metastatic lung cancer.
Setting
Single-centre feasibility study.
Participants
Patients with stage IV lung cancer in antineoplastic treatment.
Interventions
This study describes the first three phases of a complex intervention design: phase 1, development of the intervention; phase 2, feasibility testing and phase 3, evaluation of the intervention. In phase 1, items were selected for the questionnaire and adjusted following patient interviews. In phase 2, patients completed the electronic questionnaire weekly during a 3-week feasibility test. In case of symptom deterioration, a nurse was notified with the aim to contact the patient. In phase 3, patients evaluated phase 2 by paper questionnaires, and interviews were conducted with the participating nurses.
Primary outcome measures
The study outcomes: phase 1, usability and relevance; phase 2, recruitment rate, compliance and threshold functionality and phase 3, usability, acceptability and relevance.
Results
In phase 1, a questionnaire was designed and reviewed by patients (n=8). The interviews revealed high usability and relevance of the intervention.
For phases 2 and 3, 20 of 29 approached patients (69%) responded to the questionnaire on a weekly basis. Two patients did not complete any questionnaires (compliance 90%). The remaining 18 patients completed 65 of a total of 72 possible questionnaires (7 missed, 93% completed). Reported symptoms led to a phone call from a nurse in 30% of the responses.
The patients reported high usability and acceptability of questionnaire and software. The substance of the telephonic conversations was relevant, and the study set-up was logistically acceptable.
Conclusions
An electronic questionnaire designed for symptom monitoring revealed high usability, acceptability and relevance in the target population. In conclusion, the study set-up was considered feasible for a randomised controlled trial.
Trial registration number
Keywords: oncology, telemedicine, interstitial lung disease
Strengths and limitations of this study.
Electronic symptom monitoring of patients with lung cancer was tested for feasibility before conducting a randomised controlled trial.
A mixed-methods design was used to refine the electronic questionnaire.
Feasibility was tested by a three-phase complex intervention approach.
Patients were involved in evaluation and adjustment of the intervention.
Limitations include a short study period and a limited number of participating patients.
Introduction
Lung cancer is one of the most common cancers and the leading cause of cancer-related mortality globally.1 In Denmark, its annual incidence is approximately 4600 and more than 3700 persons die from the disease every year; thus, lung cancer accounts for 24% of all cancer-related deaths and 7% of the total mortality rate.2 3
Patients diagnosed with metastatic lung cancer frequently suffer from multiple and severe symptoms adversely affecting their health-related quality of life and causing psychological distress.4 5 These symptoms may impair their overall health condition, potentially reducing antineoplastic treatment efficacy.6 However, studies showed that symptom management during early palliative care may reduce the symptom burden and increase survival in patients with metastatic lung cancer.7 However, symptom deterioration between scheduled outpatient visits may go unnoticed which could delay a timely management. Furthermore, clinicians are not always consistent in their assessment of symptoms and often estimate them to be less severe than do patients themselves.8–10 These discrepancies may be remedied by the use of systematic communication tools.11 Patient-reported outcomes (PROs) used in clinical practice for symptom monitoring have been shown to improve patient satisfaction, patient–caregiver care and communication, and to lead to earlier symptom management.12 13 Currently, software solutions exist where patients can report symptoms from home to the department via the internet.14 15 In such set-ups, weekly electronic PRO (ePRO) monitoring has been found to improve overall survival and health-related quality of life in patients with lung cancer16 17 and in a broad cancer population during chemotherapy.18 Both studies used a threshold mechanism to notify clinicians in case of concerning symptoms. These results may be attributed to a combination of early detection of progressive disease, enhanced management of adverse events and improved palliative care.
Successful implementation of ePROs into clinical practice is a complex task involving several stakeholders. This task must be adapted to local logistic set-ups and it requires a clinically relevant ePRO system.19 20 However, no consensus has been established on which specific ePRO questionnaires should be used for patients with lung cancer or when and how clinicians should be notified of symptom deterioration in patients with lung cancer.16 17
The aims of this study were to design an electronic questionnaire for symptom monitoring and to evaluate its feasibility, usability and acceptability in patients with metastatic lung cancer.
Methods
Study population
The study was conducted in May–July 2018 at the Department of Oncology at Hospital Unit West Jutland, Herning, Denmark. Outpatients diagnosed with stage IV lung cancer with an available internet connection at home were eligible. Patients treated for lung cancer with a curative intent are not treated in our department, therefore patients with lower stages of disease were excluded. Patients were required to read and speak Danish, and they were receiving first-line or second-line medical antineoplastic treatment at the time of enrolment.
The electronic PRO software
The AmbuFlex system is a generic Danish PRO software system integrated into the electronic medical records at Hospital West Jutland.21 22 AmbuFlex has been used for follow-up on cancer and other chronic diseases since 2014, and it is used both in clinical practice and for research. Patients fill in health-related questionnaires via a home page and clinicians can access their responses in real-time on-screen. Mirroring longitudinal symptom development, consecutive answers are presented visually with colour bars, numbers and text. An automated threshold mechanism can be activated to identify patients needing clinical attention based on individual responses and symptom severity.23–25 The acceptability and usability of AmbuFlex in the clinical setting is deemed high by both nurses and physician.22 25 26
Study design
This feasibility study covers the first three phases of a complex intervention designed according to the Medical Research Council’s (MRC) guidelines27: phase 1, development of the intervention; phase 2, feasibility testing and phase 3, evaluation. After each phase, we adjusted the system before entering the next phase.27 28 ePRO monitoring in the clinic was implemented in accordance with the guidelines recommended by the International Society for Quality of Life Research.29 30 The purpose was to design a symptom monitoring system added to standard of care. Thus, the number of scheduled CT scans was not reduced for any patient in the current study or subsequent randomised controlled trial (RCT).
Definitions
The following definitions were used:
Usability: design factors affecting users’ experience of operating the questionnaire software and navigating it for the intended purpose.
Acceptability by patients: factors affecting users’ willingness to participate in weekly symptom self-reporting.
Acceptability by clinicians: factors affecting users’ willingness to use the system.
Relevance: a subjective perception of whether the questionnaire addressed issues deemed relevant both for patients to report to the hospital and for clinicians to be notified of.
The EORTC Item Library
The European Organisation for Research and Treatment of Cancer (EORTC) Item Library is an online database of hundreds of individual items from previously validated and translated EORTC questionnaires which allows for single-item combinations in construction of item lists for clinical and research purposes.31
Phase 1: development of the intervention
The development phase comprised an initial review of studies reporting improved survival by ePRO-based symptom monitoring.16–18 32 The authors who are also clinicians initially selected specific items for the study and integrated them into the AmbuFlex software. Patient interviews were finally conducted to appraise the need for further adjustment before phase 2.
Adjustment of the electronic questionnaire
To assess the usability and clinical relevance of the electronic questionnaire, we conducted semistructured interviews with individual patients. We kept recruiting interviewees until data saturation was reached. The interviewees were provided with a portable computer and a written instruction describing the login procedure. They were encouraged to fill in the questionnaire and comment on the procedure, while the investigator observed the process and conducted the interview.33 The interviews were supported by an interview guide to explore and address potential issues regarding usability and relevance perceived by patients or observer. The interviews were recorded, transcribed and analysed using thematic text analysis.34
To the best of our knowledge, there is no consensus on the use of threshold definitions for symptom monitoring. The authors then defined individual symptom thresholds through consensus discussion. During phase 2, the department was notified when symptoms reached these thresholds.
Phase 2: feasibility test
The second phase was a 3-week prospective feasibility test of the AmbuFlex PRO software for which a sample size of minimum 15 patients was considered sufficient based on general pilot and feasibility study recommendations.35 36 To compensate for potential compliance problems, we enrolled 20 patients.
The patients were provided with written instructions and asked to answer the ePRO questionnaire from a home internet connection a total of four times with 1-week intervals.
Two clinical oncology nurses with prior experience with the AmbuFlex system were trained in the study procedures. They were provided with a guide describing the threshold functionality of the software and how to review and manage symptom charts. The AmbuFlex software was programmed to automatically include a given patient on a notification list whenever the response exceeded the predefined symptom severity threshold. As a part of their daily work routine, the nurses were instructed to access the notification list, review responses and contact patients (figure 1). If a written comment in the comments field had triggered notification, the nurse could choose not to contact the patient if it was not deemed necessary. If a phone conversation indicated progressive disease, the planned CT scan should be brought forward and performed as soon as possible (usually done within a week); otherwise, the patient’s symptoms were treated according to best supportive care practice. The nurse recorded time spent on all study-related procedures and the number of phone calls on a daily basis.
Figure 1.
The logistic set-up. Symptoms are reported weekly via the internet. Patients who have reported symptoms that require attention are placed on a notification list. The symptom chart is reviewed daily by a nurse who contacts the patients. ePRO, electronic patient-reported outcome.
The recruitment rate was defined by the number of enrolled patient/approached patients. Compliance was the proportion of enrolled patients responding to at least one questionnaire. The threshold algorithm was evaluated as the fraction of responses leading to notification and a subsequent phone call from the nurse.
Phase 3: evaluation of the intervention and study set-up
All patients participating in phase 3 filled in a paper questionnaire by the end of the study period, evaluating the electronic questionnaire and the software as a unified entity. Evaluation themes covered usability, acceptability and relevance. Later, we conducted semistructured interviews with the involved nurses to evaluate the logistic set-up and the clinical relevance of the chosen thresholds. The authors finally evaluated the results at a consensus meeting, agreeing on adjustments before initiation of the RCT.
Patient and public involvement
Patients participating in this study were involved in the design of the ePRO intervention for the subsequent RCT. The intervention was evaluated and adjusted based on questionnaires and interviews with the patients.
Results
Phase 1: development of the intervention
Initial item selection and threshold definitions for the electronic questionnaire
Our literature review identified two previous studies, both suggesting improved overall survival following weekly internet-based monitoring of patients with cancer.16–18 Studies of patients with lung cancer have suggested other tools for symptom monitoring but reported no improved clinical intervention outcomes.37 38 The first study16 17 included patients with stages II–IV lung cancer and focused solely on symptoms relevant to patients with lung cancer, and the second study18 focused on adverse events caused by antineoplastic treatment among patients receiving active treatment of whom 25% were diagnosed with lung cancer.
Before patient interviews in phase 1, we selected 12 symptoms for the ePRO questionnaire based on previous studies.16 18 32 39 Eleven of these symptoms were identical with the symptoms reported by Denis et al.16 Self-rated overall health was, due to known prognostic properties, included in the questionnaire instead of depression.40 41 The 12 symptoms selected for the initial version of the questionnaire are shown in box 1.
Box 1. The initial symptoms selected for the electronic questionnaire.
Symptoms graded by severity*
Overall health.
Dyspnoea.
Pain.
Fatigue.
Appetite loss.
Coughing.
Haemoptysis.
Alarm symptoms
Fever.
Voice.
Facial swelling.
Sense of a growing tumour.
Weight.
Other
Comments field.
*Items selected from the EORTC Item Library,(https://www.eortc.be/itemlibrary/).
Seven of the 12 selected symptoms were available as EORTC items and all graded by a Likert scale.31 For all EORTC items, the recall period was ‘the past week’. Three supplementary symptoms (facial swelling, hoarse voice and sense of a growing tumour) were considered alarm symptoms needing specific attention and were not scored. The wording of the initial versions of these items was produced by four study group members (RBF, CTM, NHH and HS). Current weight and temperature were entered in additional boxes. The items were intended to be used as a screening tool to identify patients with deterioration of specific symptoms requiring clinicians’ attention.
Semistructured interviews
Semistructured interviews were conducted in phase 1 with eight patients while they were filling in the questionnaire in the AmbuFlex system. They were interviewed about the ePRO software and the questionnaire design. Thematic text analysis was used, identifying the following five subthemes: usability, acceptability, inaccurate phrasing, insufficient number of items and lack of response options.
The ePRO system
The usability of the software was high. Patients had very few issues with login and response procedures. Two of them mentioned that they had difficulties using a computer, but both could complete the questionnaire by following the instructions. One patient stated that ‘it’s quick to answer’, and the other stated that ‘it was easy to log in’. However, the interviewer observed that a few clarifications of the login instructions were provided to the patients.
The acceptability of the AmbuFlex software was high. Patients wanted to participate even if it took some effort. In the words of one patient; ‘using a computer is difficult, but if it would help, I would do it’. No patients expressed reluctance using the electronic questionnaire.
Design of the questionnaire
All patients were pleased with the short length of the questionnaire. The majority of the symptoms were selected from the EORTC item bank, and no misconceptions were perceived in relation to these questions.
Several patients found two of the alarm questions referring to the time frame ‘since last time’ to be confusing since these questionnaire had not been answered earlier. The wording of the time frame was then changed to ‘during the past week’. There was one misunderstanding concerning the alarm symptoms. One patient felt that she was unable to evaluate whether she had a sense of a growing tumour and consequently could not answer this question. The issue was solved by adding the response option ‘I don’t know’ to the questionnaire.
Another patient wanted to be able ‘to describe the psychological burden of lung cancer’. We acknowledge that this issue is very important to many patients. However, due to the complexity of this theme and given the purpose of this study, we decided to confine remarks on psychological issues to the comments field. The patients made no further suggestions concerning other relevant symptoms.
Threshold definitions
We then defined the initial symptom severity thresholds for each item by consensus decision. For symptoms graded by severity (symptoms 2–6, box 1), the threshold was cut between ‘none’/‘a little’ and ‘quite a bit’/‘very much’. For self-assessed health, a score ≤3 was the threshold for notifying the department. Clinicians were notified only when a symptom had become worse since the previous week. The authors decided that any presence of haemoptysis should trigger notification.
Conditional branching was used for three of the alarm symptoms. If facial swelling or hoarse voice was reported, the patient was prompted to report whether the symptom had worsened during the past week. If a sensation of fever was reported, the patient was prompted to measure and enter the temperature. The thresholds used were ≥38.2°C for temperature and ≥3 kg for weight loss compared with baseline. Finally, a supplementary comments field was added to enable the patients to report other symptoms.
Phase 2: feasibility test
In phase 2, we approached 29 patients in the outpatient clinic, five of whom were ineligible because they had no internet connection at home. Four patients declined participation, feeling that they could not comply with the intervention. The recruitment rate was 69% (20/29).
The baseline patient characteristics are presented in table 1. The median age was 70.5 years (range 54–86 years). Most of the enrolled patients had prior experience with the internet, although one patient categorised herself as a very inexperienced internet user, and she was, nevertheless, able to complete all the four questionnaires in the test period.
Table 1.
Baseline characteristics
| n | % | |
| Age, years; median (range) | 70.5 (54–86) | |
| Sex | ||
| Male | 13 | 65 |
| Female | 7 | 35 |
| Treatment line | ||
| 1st | 13 | 65 |
| 2nd | 7 | 35 |
| Civil status | ||
| Married | 17 | 85 |
| Widow | 2 | 10 |
| Divorced | 0 | 0 |
| Single | 1 | 5 |
| Highest completed education | ||
| Primary school | 8 | 40 |
| High school | 0 | 0 |
| Professional education | 7 | 35 |
| Short higher education | 3 | 15 |
| Medium higher education | 2 | 10 |
| Long higher education | 0 | 0 |
| Internet experience | ||
| Very experienced | 4 | 20 |
| Experienced | 5 | 25 |
| Neither | 3 | 15 |
| Inexperienced | 7 | 35 |
| Very inexperienced | 1 | 5 |
Two patients completed none of the questionnaires and were excluded from analysis. Among participating patients, weekly questionnaires were completed 72/80 (93%) times (table 2). The threshold was exceeded by 55% (37/67), leading to further action by a clinical nurse, and in 30% (20/67) action consisted in a phone call. One programming error regarding the severity of dyspnoea unintentionally led to five false notifications. The time spent responding to alarm notifications, including phone calls, was managed by the nurse in a median of 6 min (range 0.2–30 min) per day.
Table 2.
Compliance and notifications (n=18*)
| Week | 1 | 2 | 3 | 4 | Total |
| Possible completions | 18 | 18 | 18 | 18 | 72 |
| Questionnaires completed | 18 | 18 | 17 | 14 | 67 |
| Completion rate, % | 100 | 100 | 94 | 78 | 93 |
| Notification thresholds exceeded | 15 | 5 | 5 | 7 | 32 |
| Additional notifications sent due to erroneous algorithm programming | 0 | 3 | 2 | 0 | 5 |
| Notification thresholds exceeded/completed questionnaire, % | 83 | 44 | 41 | 50 | 55 |
| Phone calls made | 4 | 7 | 6 | 3 | 20 |
| Phone calls made/per completed questionnaire, % | 22 | 39 | 35 | 21 | 30 |
A phone call was handled in a median time of 11 min.
The nurse spent a median of 6 min (min 0,2; max 30) per day on study-related procedures.
*2/20 enrolled patients did not participate in the pilot study.
Phase 3: evaluation of the intervention
Patient questionnaires
The intervention and the study set-up were evaluated in a questionnaire completed by all patients participating in phase 2 (n=18) (table 3).
Table 3.
Evaluation questionnaire (n=18), % (n)
| Usability | |||||
| To which extent do you agree or disagree with the following statements? | Strongly disagree/disagree | Neither | Agree/strongly agree | Doesn’t know | N/A |
| It is easy to log in to answer the questionnaire | 6% (1) | 0% (0) | 94% (17) | 0% (0) | – |
| I find it easy to read the questionnaire | 0% (0) | 0% (0) | 100% (18) | 0% (0) | – |
| The questions in the questionnaire are easy to understand | 0% (0) | 11% (2) | 89% (16) | 0% (0) | – |
| <5 min | 5–10 min | 10–15 min | >15 min | ||
| How long have you approximately spent answering the questionnaire each week? | 39% (7) | 39% (7) | 22% (4) | 0% (0) | – |
| No | Occasionally | Yes, every time | |||
| Did you need any help to fill in the questionnaire? | 89% (16) | 11% (2) | 0% (0) | – | |
| No | Occasionally | Yes, every time | |||
| Have you experienced technical problems? | 100% (18) | 0% (0) | 0% (0) | – | |
| Acceptability | |||||
| To which extent do you agree or disagree with the following statements? | Strongly disagree/disagree | Neither | Agree/strongly agree | Doesn’t know | N/A |
| I am confident that the outpatient clinic will contact me when they have seen my answers, if needed | 0% (0) | 6% (1) | 94% (17) | 0% (0) | – |
| I always call the outpatient clinic if I have problems that I need to discuss with a doctor or nurse | 0% (0) | 22% (4) | 67% (12) | 11% (2) | – |
| I get more worried about my cancer when I fill in the questionnaire | 39% (7) | 28% (5) | 17% (3) | 11% (2) | 6% (1) |
| It is difficult to remember to answer the questionnaire every week | 61% (11) | 11% (2) | 22% (4) | 6% (1) | – |
| No | Yes | ||||
| Were you generally satisfied with the questionnaire used in the study period? | 0% (0) | 89% (16) | 11% (2) | ||
| Relevance | |||||
| To which extent do you agree or disagree with the following statements? | Strongly disagree/disagree | Neither | Agree/strongly agree | Doesn’t know | N/A |
| The questionnaire makes me more aware of symptoms that may be due to my illness | 6% (1) | 33% (6) | 56% (10) | 6% (1) | – |
| The questionnaire helps me to remember problems that I would like to discuss with the doctor/nurse | 6% (1) | 50% (9) | 39% (7) | 6% (1) | – |
| Not at all/to a lesser extent | To some extent/highly | Doesn’t know | |||
| To which extent do you find the questions relevant to you? | 17% (3) | 78% (14) | 6% (1) | – | |
| No | Yes | ||||
| Did you miss any topics? | 94% (17) | 6% (1) | – | ||
| No | Yes | ||||
| Did you find any topics irrelevant? | 94% (17) | 6% (1) | – | ||
N/A, not applicable.
Usability
The patients found it easy to log in and to read and answer the questions. The estimated time spent to complete the weekly questionnaire was less than 10 min for 78% of the patients and less than 15 min for the remaining patients. Two needed help from a relative to complete the questionnaires, but nobody experienced any technical problems. One patient reported that ‘the questionnaire is easy to complete, and it is good to be aware of possible side effects’.
Acceptability
Seventeen (94%) patients felt confident that they would be contacted by the clinic if needed and 16 (89%, 2 non-respondents) were satisfied with the questionnaire. Three (17%, 1 non-respondent) patients felt that they got more worried about their cancer when answering the questionnaire. By contrast, 61% disagreed and two patients stated the opposite view in the comments field. The first patient was ‘satisfied with the additional sense of security’, and the other specified ‘it is reassuring to know that one is being watched if complications occur’.
Relevance
Seven (39%) patients felt better prepared for the dialogue with the doctor and 10 (56%) felt more aware of disease-related symptoms. Fourteen (78%) patients found the questions relevant. One patient expressed a need to report more responses that are detailed, and the other would like to be able to report the functional level and the psychological burden. It was possible to report only a broad picture of the situation in the comments field, but the evaluation made it evident that the EORTC item ‘self-rated quality of life’ could usefully be added to the questionnaire. This item allows patients to report their own assessment of quality of life and, in combination with self-rated overall health, enables the calculation of a longitudinal EORTC global quality of life score.
Nurse interviews
Two clinical nurses involved in the management of the notification list were interviewed about clinical perspectives. Both experienced that the phone consultations were very relevant for the patients, and there were only few examples of unnecessary contacts. They felt that the daily task of monitoring patients was acceptable and meaningful. However, one nurse was concerned that the workload could grow and become a problem if many patients were enrolled in the RCT without additional resource allocation.
The clinical relevance of the threshold limits was also explored. One of the nurses thought that the individual symptoms were of different clinical importance and stated that ‘loss of appetite … and also fatigue … often notifies. And the question is how much we actually use it. We only really do something if the fatigue is disabling … or if the loss of appetite is prolonged’. The other nurse agreed and suggested to change the threshold definitions for these two symptoms to a higher degree of severity.
The threshold definitions for fatigue and appetite loss were therefore raised to a higher severity grade. The final design of the electronic questionnaire and threshold definitions can be found in the online supplementary appendix.
bmjopen-2019-035673supp001.pdf (73.3KB, pdf)
Discussion
In this study, we designed and tested the feasibility of an electronic questionnaire for weekly internet-based symptom monitoring in patients with metastatic lung cancer. We found that the use of an electronic questionnaire based on an EORTC item score for symptom monitoring in lung cancer is feasible for both patients and healthcare professionals. The results pave the way for testing the set-up in an RCT.
The electronic questionnaire had a high usability in both phases 1 and 3, but the instructions for patients had to be simplified. No technical issues arose, although one symptom-specific programming error was identified and corrected. The high usability was consistent with other studies having used the AmbuFlex software to collect PRO data.24 25
The feasibility test demonstrated a need for the software to be supplemented with a functionality ensuring early identification of non-responders as 2 of 20 feasibility testing participants never started filling in a weekly questionnaire. Moreover, the ePRO questionnaire completion rate dropped towards the end of the test period. This could be due to misunderstandings since patients were asked to fill in the questionnaire the day before the next planned treatment, which coincided with the conclusion of the study period. Some patients had already answered other ePRO questionnaires as a part of routine care in the department. They were thus supposed to answer two questionnaires on the same day, which could explain why some forgot to answer the last questionnaire. To ensure compliance in the upcoming RCT, notifications of non-responding patients will be sent to the nurses as a part of the daily routine. Nurses may then contact non-compliant patients, offering them the guidance they need. By introducing a fixed daily work routine where nurses checked notification lists, we ensured proper response whenever score thresholds were passed. Conclusions about the attrition rate cannot be made due to the short study period.
Patients’ questionnaire responses made nurses call patients a total of 20 times (30%) during the 3-week test period. The algorithm was programmed to notify the clinicians only when symptoms grew worse compared with the previous week. The questionnaire responses given in week 1 triggered more notifications than subsequent responses because the system was programmed to always notify clinicians when a symptom threshold was exceeded and no previous response was available for comparison in the first week. The nurses were instructed to contact patients only if patients’ answers were concerning; however, initially, the nurses acted proactively and made more phone calls than they were trained to make. The interviews with the nurses revealed that they acted with a high sense of responsibility but also had some uncertainty about the procedures. Had the instructions been followed strictly, only half as many (viz. 10) phone calls would have been made in the test period. This underscores the need for clear and concise instructions for staff managing the notifications. Accordingly, the training plans for the nurses were updated with relevant clarifications prior to the RCT.
The amount of time spent on managing notifications and contacting patients was a serious concern raised by the nurses and department managers as well as by collaborators in the subsequent multicentre RCT. However, once it was clarified how much time was actually spent on the daily procedures, the initial concerns among all stakeholders were substantially reduced.
Previous studies have tested other electronic systems for patients with lung cancer. Maguire et al found that mobile technology used for monitoring radiotherapy-related toxicity was feasible and had high acceptability in patients with lung cancer.37 An RCT with 253 patients with lung cancer showed that weekly tele monitoring was feasible and acceptable.38 However, the study that used a phone-based interactive voice response technology failed to improve satisfaction or clinical outcomes. This could be due to the fact that follow-up lasted only 12 weeks and that patients were recruited along different treatment lines. The internet-based ePRO systems may offer higher usability and acceptance among both patients and clinicians than a voice response technology as it could ease interpretation of the reported symptoms.
The mechanisms underlying the effect of intensified ePRO-based monitoring are complex. Denis et al found high compliance in a pilot study of web-based symptom monitoring of patients with lung cancer.39 42 This study also showed a potential for detection of early relapse. The six symptoms included in the pilot was later expanded to 12 symptoms used in the previously mentioned RCT where the ePRO intervention improved overall survival.16 17 The authors suggested that early relapse detection was the main reason for the effect. Other potential mechanisms proposed by Basch et al were early responsiveness to symptom management, supportive care and drug dose modifications improving treatment tolerance.18 Additionally, studies have found that early palliative care could improve both health-related quality of life and survival in lung cancer.7 43
The strength of this study was its multidimensional approach conforming with the MRC guidelines for complex interventions.27 All enrolled patients were real-life patients receiving outpatient treatment, some of whom had limited computer skills and moderate educational attainment. It was important to test the system in a setting where patients used their own internet device so that any technical issues could be addressed before launching the subsequent RCT.
The short study period with a relatively low number of participating patients was a limitation to this study. Since the AmbuFlex PRO system has already been widely tested, we may conclude that use of the AmbuFlex software is feasible in this study set-up.25
Conclusions
A study set-up for a national RCT using weekly symptom monitoring based on EORTC items is feasible.
The following trial, ProWide (Patient-Reported Outcomes used for Weekly Internet-based Detection of progressive disease in lung cancer, Clinicaltrials.gov NTC03608410), is a two-arm, open-labelled, multicentre RCT aiming to determine the effect of ePRO-based symptom monitoring added to standard care. This study will include 492 patients diagnosed with lung cancer in Denmark. The power calculation is based on an anticipated effect on overall survival of half the size of the 1-year overall survival in the study by Denis et al and a compliance rate of 90%.16 The study is open and recruitment is ongoing.
Supplementary Material
Acknowledgments
The authors thank all the participating patients and the dedicated clinical nurses Vibeke Holst May and Anne Marie Hill from the Department of Oncology, Hospital Unit West Jutland, Herning. They also thank Gry Assam Taarnhøj for valuable feedback on the manuscript.
Footnotes
Contributors: RBF: Designed the draft of the electronic questionnaire. Enrolled all patients, conducted the interviews, monitored the 3-week test period, evaluated and analysed the data and wrote the initial draft of the paper. NHH: Programmed the software. Major input regarding the logistic planning. Did all the software and threshold programming. Large contribution on the design of the item list for the electronic questionnaire. CTM: Had a large say in the design of the interview guide, the analysis of the interview data and the design of both the electronic questionnaire and the evaluation questionnaire. HP: Major influence on the interpretation of data and the structure of the paper. HS: Idea of the project. Primary supervisor. Wrote the initial study plan and made major contributions to the manuscript. Large impact on the design of the electronic questionnaire.
Funding: The study was funded by a grant from The Danish Cancer Society (Grant No. R184-A11805) and by the Department of Oncology, Hospital Unit West Jutland Hospital, Central Denmark Region.
Competing interests: None declared.
Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Patient consent for publication: Not required.
Ethics approval: The study was approved by the Danish Data Protection Agency (2017-41-5251). According to Danish law, no approval was required by the Danish Research Ethics Committee (enquiry 266 received on 7 December 2017). All participants received oral information and signed a written consent prior to enrolment.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data availability statement: All data relevant to the study are included in the article or uploaded as supplementary information. No additional data available.
References
- 1.Ferlay J, Ervik M, Lam F, et al. Global cancer Observatory: cancer today. Lyon, France: IARC, International Agency for Research on Cancer, 2018. [Google Scholar]
- 2.Engholm G, Ferlay J, Christensen N. NORDCAN: cancer incidence, mortality, prevalence and survival in the Nordic countries. Version 8.2 Association of the Nordic Cancer Registries. Danish Cancer Society, 2016. [Google Scholar]
- 3.Statistics Denmark Statistics Denmark - Mortality Denmark 2012-2016. Available: https://www.statistikbanken.dk/statbank5a/selectvarval/saveselections.asp [Accessed 13 Aug 2019].
- 4.Lutz S, Norrell R, Bertucio C, et al. Symptom frequency and severity in patients with metastatic or locally recurrent lung cancer: a prospective study using the lung cancer symptom scale in a community hospital. J Palliat Med 2001;4:157–65. 10.1089/109662101750290191 [DOI] [PubMed] [Google Scholar]
- 5.Iyer S, Taylor-Stokes G, Roughley A. Symptom burden and quality of life in advanced non-small cell lung cancer patients in France and Germany. Lung Cancer 2013;81:288–93. 10.1016/j.lungcan.2013.03.008 [DOI] [PubMed] [Google Scholar]
- 6.Simmons CP, Koinis F, Fallon MT, et al. Prognosis in advanced lung cancer—a prospective study examining key clinicopathological factors. Lung Cancer 2015;88:304–9. 10.1016/j.lungcan.2015.03.020 [DOI] [PubMed] [Google Scholar]
- 7.Ambroggi M, Biasini C, Toscani I, et al. Can early palliative care with anticancer treatment improve overall survival and patient-related outcomes in advanced lung cancer patients? A review of the literature. Support Care Cancer 2018;26:2945–53. 10.1007/s00520-018-4184-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Atkinson TM, Dueck AC, Satele DV, et al. Clinician vs patient reporting of baseline and Postbaseline symptoms for adverse event assessment in cancer clinical trials. JAMA Oncol 2019:jamaoncol.2019.5566. 10.1001/jamaoncol.2019.5566 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Pakhomov SV, Jacobsen SJ, Chute CG, et al. Agreement between patient-reported symptoms and their documentation in the medical record. Am J Manag Care 2008;14:530–9. [PMC free article] [PubMed] [Google Scholar]
- 10.Laugsand EA, Sprangers MAG, Bjordal K, et al. Health care providers underestimate symptom intensities of cancer patients: a multicenter European study. Health Qual Life Outcomes 2010;8:104. 10.1186/1477-7525-8-104 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Homsi J, Walsh D, Rivera N, et al. Symptom evaluation in palliative medicine: patient report vs systematic assessment. Support Care Cancer 2006;14:444–53. 10.1007/s00520-005-0009-2 [DOI] [PubMed] [Google Scholar]
- 12.Kotronoulas G, Kearney N, Maguire R, et al. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol 2014;32:1480–501. 10.1200/JCO.2013.53.5948 [DOI] [PubMed] [Google Scholar]
- 13.Porter I, Gonçalves-Bradley D, Ricci-Cabello I, et al. Framework and guidance for implementing patient-reported outcomes in clinical practice: evidence, challenges and opportunities. J Comp Eff Res 2016;5:507–19. 10.2217/cer-2015-0014 [DOI] [PubMed] [Google Scholar]
- 14.Jensen RE, Snyder CF, Abernethy AP, et al. Review of electronic patient-reported outcomes systems used in cancer clinical care. J Oncol Pract 2014;10:e215–22. 10.1200/JOP.2013.001067 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Bennett AV, Jensen RE, Basch E. Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin 2012;62:336–47. 10.3322/caac.21150 [DOI] [PubMed] [Google Scholar]
- 16.Denis F, Lethrosne C, Pourel N, et al. Randomized trial comparing a web-mediated follow-up with routine surveillance in lung cancer patients. J Natl Cancer Inst 2017;109:1–8. 10.1093/jnci/djx029 [DOI] [PubMed] [Google Scholar]
- 17.Denis F, Basch E, Septans AL, et al. Two-Year survival comparing web-based symptom monitoring vs routine surveillance following treatment for lung cancer. JAMA 2019;321:jama.2018.18085:306. 10.1001/jama.2018.18085 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Basch E, Deal AM, Dueck AC, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA 2017;318:197. 10.1001/jama.2017.7156 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Snyder CF, Aaronson NK, Choucair AK, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res 2012;21:1305–14. 10.1007/s11136-011-0054-x [DOI] [PubMed] [Google Scholar]
- 20.Wintner LM, Sztankay M, Aaronson N, et al. The use of EORTC measures in daily clinical practice—a synopsis of a newly developed manual. Eur J Cancer 2016;68:73–81. 10.1016/j.ejca.2016.08.024 [DOI] [PubMed] [Google Scholar]
- 21.Hjollund NHI, Larsen LP, Biering K, et al. Use of patient-reported outcome (pro) measures at group and patient levels: experiences from the generic integrated pro system, WestChronic. Interact J Med Res 2014;3:e5. 10.2196/ijmr.2885 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Schougaard LMV, Larsen LP, Jessen A, et al. AmbuFlex: tele-patient-reported outcomes (telePRO) as the basis for follow-up in chronic and malignant diseases. Qual Life Res 2016;25:525–34. 10.1007/s11136-015-1207-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Grove BE, Schougaard LM, Hjollund NH, et al. Self-Rated health, quality of life and appetite as predictors of initiation of dialysis and mortality in patients with chronic kidney disease stages 4-5: a prospective cohort study. BMC Res Notes 2018;11:371. 10.1186/s13104-018-3472-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Schougaard LMV, Mejdahl CT, Petersen KH, et al. Effect of patient-initiated versus fixed-interval telePRO-based outpatient follow-up: study protocol for a pragmatic randomised controlled study. BMC Health Serv Res 2017;17:83. 10.1186/s12913-017-2015-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Baeksted C, Pappot H, Nissen A, et al. Feasibility and acceptability of electronic symptom surveillance with clinician feedback using the patient-reported outcomes version of common terminology criteria for adverse events (PRO-CTCAE) in Danish prostate cancer patients. J Patient Rep Outcomes 2017;1:1. 10.1186/s41687-017-0005-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Dørfinger L, Grønbæk C, Rahbek T. Clinical use of patient reported outcome measures (PROM)—an evaluation report, 2016. https://www.cancer.dk/dyn/resources/File/file/5/5305/1455711944/klinisk-anvendelse-af-patient-reported-outcome-measures-prom-en-evalueringsrapport.pdf [Google Scholar]
- 27.Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: the new medical Research Council guidance. Int J Nurs Stud 2013;50:587–92. 10.1016/j.ijnurstu.2012.09.010 [DOI] [PubMed] [Google Scholar]
- 28.Richards DA, Hallberg IR. Complex Interventions in Health—an overview of research methods. 1st edn Routledge, 2015. [Google Scholar]
- 29.Aaronson N, Elliott T, Greenhalgh J, et al. User’s Guide to Implementing Patient-Reported Outcomes Assessment in Clinical Practice Produced on behalf of the International Society for Quality of Life Research, 2015: 1–47. [Google Scholar]
- 30.Chan EKH, Edwards TC, Haywood K, et al. Implementing patient-reported outcome measures in clinical practice: a companion guide to the ISOQOL user's guide. Qual Life Res 2019;28:621–7. 10.1007/s11136-018-2048-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.EORTC The EORTC quality of life Group. EORTC Item Library. Available: https://www.eortc.be/itemlibrary/ [Accessed 16 Apr 2019].
- 32.Basch E, Deal AM, Kris MG, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 2016;34:557–65. 10.1200/JCO.2015.63.0830 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Collins D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res 2003;12:229–38. 10.1023/A:1023254226592 [DOI] [PubMed] [Google Scholar]
- 34.Malterud K. Systematic text condensation: a strategy for qualitative analysis. Scand J Public Health 2012;40:795–805. 10.1177/1403494812465030 [DOI] [PubMed] [Google Scholar]
- 35.Stallard N. Optimal sample sizes for phase II clinical trials and pilot studies. Stat Med 2012;31:1031–42. 10.1002/sim.4357 [DOI] [PubMed] [Google Scholar]
- 36.Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat 2005;4:287–91. 10.1002/pst.185 [DOI] [Google Scholar]
- 37.Maguire R, Ream E, Richardson A, et al. Development of a novel remote patient monitoring system: the advanced symptom management system for radiotherapy to improve the symptom experience of patients with lung cancer receiving radiotherapy. Cancer Nurs 2015;38:E37–47. 10.1097/NCC.0000000000000150 [DOI] [PubMed] [Google Scholar]
- 38.Yount SE, Rothrock N, Bass M, et al. A randomized trial of Weekly symptom telemonitoring in advanced lung cancer. J Pain Symptom Manage 2014;47:973–89. 10.1016/j.jpainsymman.2013.07.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Denis F, Viger L, Charron A, et al. Detecting lung cancer relapse using self-evaluation forms Weekly filled at home: the sentinel follow-up. Support Care Cancer 2014;22:79–85. 10.1007/s00520-013-1954-9 [DOI] [PubMed] [Google Scholar]
- 40.DeSalvo KB, Bloser N, Reynolds K, et al. Mortality prediction with a single General self-rated health question. A meta-analysis. J Gen Intern Med 2006;21:267–75. 10.1111/j.1525-1497.2005.00291.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Shadbolt B, Barresi J, Craft P. Self-Rated health as a predictor of survival among patients with advanced cancer. J Clin Oncol 2002;20:2514–9. 10.1200/JCO.2002.08.060 [DOI] [PubMed] [Google Scholar]
- 42.Denis F, Viger L, Charron A, et al. Detection of lung cancer relapse using self-reported symptoms transmitted via an internet web-application: pilot study of the sentinel follow-up. Support Care Cancer 2014;22:1467–73. 10.1007/s00520-013-2111-1 [DOI] [PubMed] [Google Scholar]
- 43.Temel JS, Greer JA, Muzikansky A, et al. Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med 2010;363:733–42. 10.1056/NEJMoa1000678 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2019-035673supp001.pdf (73.3KB, pdf)

