Development and results of a patient-reported treatment experience questionnaire on a 1.5 T MR-Linac

Highlights

• •Questionnaires are important tools in assessing the tolerability of new technology.
• •Patient reported experience of treatment on the MR-Linac is consistently high.
• •The MR-Linac questionnaire has been validated as relevant and appropriate for use.

Abstract

Introduction

With the implementation of new radiotherapy technology, it is imperative that patient experience is investigated alongside efficacy and outcomes. This paper presents the development of a specifically designed validated questionnaire and a first report of international multi-institutional preliminary patient experience of MRI-guided adaptive radiotherapy (MRgART) on the 1.5 T MR-Linac (MRL).

Methods

A patient experience questionnaire was developed and validated before being distributed to the Elekta MRL Consortium, to gather first patient-reported experience from participating centres worldwide. The final version of the questionnaire contains 18 questions covering a range of themes and was scored on a Likert scale of 0–3. Responses were post-processed so that a score of 0 represents a negative response and 3 represents the most favourable response. These results were analysed for patient-reported experience of treatment on the MRL. Results were also analysed for internal consistency of the questionnaire using Chronbach’s Alpha and the questionnaire contents were validated for relevance using content validity indexes (CVI).

Results

170 responses were received from five centres, representing patients with a wide range of tumour treatment sites from four different countries. MRgART was well tolerated with an 84% favourable response across all questions and respondents. When analysed by theme, all reported the highest percentage of results in the favourable categories (2 and 3). Internal consistency in the questionnaire was high (Cronbach’s α = 0.8) and the item-level CVI for each question was 0.78 or above and the Scale-level CVI was 0.93, representing relevant content.

Conclusion

The developed questionnaire has been validated as relevant and appropriate for use in reporting experience of patients undergoing treatment on the MRL. The overall patient-reported experience and satisfaction from multiple centres within the Elekta MRL Consortium was consistently high. These results can reinforce user confidence in continuing to expand and develop MRL use in adaptive radiotherapy.

Introduction

Accuracy of conventional radiotherapy delivery relies on reproducible patient positioning. Many studies have investigated duplicability of patient position, the effectiveness of immobilisation devices and patient preparation, all of which aims to improve reproducibility of shape and position of internal organs [1], [2], [3], [4], [5]. Patient experience in radiotherapy has been less intensively investigated and is more often focused on techniques which may cause claustrophobia, for example masks used for head and neck treatments [6], [7]. With the recognised importance of evaluating and including patient experience in health care [8], [9], it is essential that patient experience is investigated alongside efficacy and outcomes when introducing a new technology.

The implementation of systems combining magnetic resonance imaging (MRI) with radiotherapy on a linear accelerator (MR-Linac) provides a unique situation to investigate patient experience. Not only is the treatment time longer than conventional radiotherapy (30–45 mins) but each system has specific factors which may affect patient experience. Specifically with MRI, noise and restricted space are the main factors reported to cause distress. Patients’ responses to MRI exams have ranged from acceptable [10] to distressing and provoking anxiety [11], [12]. Questionnaires used in these studies were specifically designed for MRI with a focus on anxiety. Although apprehension of the MRI scan can be a predictor for anxiety, in cancer patients the main reason for concern was the possible outcome of the scan [13]. The needs and experience of patients undergoing daily radiotherapy are likely to be different to those seen in patients undergoing a single diagnostic scan. Whilst general reports of patient experience on linacs are few, a validated questionnaire [14] was developed in Sweden to capture patient experience during the radiotherapy pathway. This study reported a significant variation in experience depending on tumour site treated [14]. Twenty-four questions, covering psychological stress, physical discomfort and coping, were answered by 825 respondents during the radiotherapy course. ‘Situational unease’ scores, as a measure of discomfort caused by the treatment setting, were significantly greater for head and neck cancer patients than for other tumour sites. There was no reported difference regarding the time point during the treatment period when the survey was completed [14]. The questionnaire is a useful tool to measure patients’ comfort and experiences of conventional radiotherapy, however translation and validation in other languages is required.

Early reports of MRI-guided adaptive radiotherapy (MRgART) delivered using an MR-Linac (MRL) have included locoregional control rates and feasibility. Typically, these have not focused specifically on patient experience [15], [16], [17]. Three single centres have independently reported feasibility and patient acceptability of MRgART using questionnaires developed in-house [18], [19], [20]. Treatments were delivered on a 0.35 T MRIdian (ViewRay®, Ohio, USA) Linac or Cobalt system. Gated breath-hold was used in ~50% of patients and respondent numbers were 34, 150 and 90 respectively [18], [19], [20]. Patient acceptability varied greatly between studies (29% versus 65%) [18], [19] and details of questionnaire design and validity were not included [18], [19], [20]. Experience with non-gated treatments on a 1.5 T MRL has not been reported. Acoustic noise inside the bore is a known complaint from patients undergoing MRI examinations [12] and the difference in Tesla from 0.35 T to 1.5 T will increase the noise levels experienced by the patients.

This paper is a first report of international multi-institutional preliminary patient experience of adaptive radiotherapy using MRI-guidance on a 1.5 T MRL and presents the development and validation of a specifically designed questionnaire.

Methods and materials

To capture patient experience of this fully integrated MRI-guided treatment system, the 1.5 T Elekta Unity MR-Linac (Elekta AB, Stockholm, Sweden), a pilot questionnaire was created by the author (HMCN) from two previously validated questionnaires. One measured radiotherapy experience [14] and the other MRI experience [21]. The pilot questionnaire was designed to be completed immediately after treatment. Since many patients would be treated under clinical trials and asked to fill out repeated patient-reported outcome measures (PROMS), the time taken to complete was considered carefully and the length of the questionnaire was kept to 18 questions on a single page. The following themes were included from both validated questionnaires: situational discomfort; physical discomfort; situational coping; informational needs; environmental coping; and psychological coping (Table 1), with the addition of two questions from the MRI questionnaire [21] ‘I forced myself to manage the situation’ and ‘I understand the procedure’. Respondents were asked to complete a Likert scale with scored responses of ‘Not at all’ (score 0), ‘Slightly’ (score 1), ‘Moderately’ (score 2) and ‘Very’ (score 3).

Table 1.

Themes attributed to each question.

Question	Theme
I needed more detailed information before my treatment	Informational needs
I found the treatment position comfortable	Situational coping
I found the treatment bed comfortable	Situational coping
I found it easy to stay still and maintain the treatment position	Physical discomfort
I wanted to come out of the machine during my treatment	Situational discomfort
I felt calm during my treatment	Situational coping
I found the noise in the room easy to tolerate	Environmental coping
I found the lighting in the room easy to tolerate	Environmental coping
I found the time taken for the treatment easy to tolerate	Situational coping
I felt dizzy during my treatment	Physical discomfort
I felt dizzy immediately after my treatment	Physical discomfort
I felt hot during my treatment	Physical discomfort
I felt tingling sensations during my treatment	Physical discomfort
I experienced a metallic taste during my treatment	Physical discomfort
I needed more communication from staff during my treatment	Informational needs
I forced myself to manage the situation	Situational discomfort
I found listening to the music helpful whilst having my treatment	Environmental coping
I understood the procedure	Psychological coping

The pilot questionnaire (in English) was circulated to patient and public involvement (PPI) groups and health professionals in the UK and Netherlands for comments and responses on clarity of questions and length of time taken to complete. PPI feedback suggested adding a space for comments and altering some wording. For example, ‘endure’ was changed to ‘tolerate’ and ‘claustrophobia’ to ‘I wanted to come out of the machine during my treatment’. The start of some questions were altered from ‘Did you feel’ to ‘I felt’. Also questions about information given and time in treatment room were added as suggested by the health professionals.

The second version of the questionnaire (Appendix A) was circulated to ten oncology expert health professionals and content validity was evaluated using the content validity index (CVI) method described by Ahlander et al. [21]. The health professionals included a mixture of radiation oncologists and therapeutic radiographers/RTTs. Experts were asked to rate the relevance of each question using a 4-point scale; 1 = not relevant. 2 = somewhat relevant. 3 = quite relevant. 4 = highly relevant. When an expert deemed a question to be relevant (3 or 4), it was given a score of ‘1′ and when it was not relevant (1 or 2), a score of ‘0′. Item-level CVI (I-CVI) was calculated by summing the number of experts rating the item as relevant (‘1′) divided by the number of experts. With seven or more experts in the group, I-CVI is recommended to be at least 0.78 [19]. CVI on scale-level (SCVI) was calculated as the average proportion of items rated as relevant and should be at least 0.90 [22].

The final version of the questionnaire was posted on the MRL Consortium web pages and participating centres were asked to send back patient experience from their initial cohort of patients. Patients included were the first patients treated on the MRL at each centre. Details of consent to participation and timing of questionnaire completion can be seen in table 2. Participating centres from non-English-speaking countries translated the questionnaire according to a previously established method for best practice [23]. Questionnaire responses were anonymised for inclusion in this study, but staff were not blinded at the time of questionnaire completion, meaning that negative responses or complaints could be addressed with the patient at the time of completion if appropriate.

Table 2.

Breakdown of data by clinical site.

Site	Country	Number of completed questionnaire responses	Number of patient responders	Number of questionnaires not completed	Questionnaire schedule	Location of questionnaire completion	Site response rate (%)	Consent process for questionnaires	Comments
1	UK	35	10	3	First 3 and final fractions (4 per patient)	Immediately after treatment	92	Included within a feasibility trial	Two patients received only three treatments and therefore were not given a fourth questionnaire
2	UK	12	6	0	First and final fractions (2 per patient)	Immediately after treatment	100	Included within a feasibility trial
3	Germany	103	99	10	Weekly, least favourable response included (1 per patient)	Immediately after treatment	91	Included within a feasibility trial	Four patients received two separate treatment courses and therefore contributed two responses each
4	Denmark	10	10	0	Final fraction	Immediately after treatment	100	Included within a feasibility trial
5	USA	10	10	0	Once during the treatment course	At home	100	IRB waiver acquired retrospectively

All responses were analysed individually. Scores were post-processed for all negatively phrased questions such as, ‘I needed more detailed information before my treatment’, so that high scores (2,3) were attributed to a favourable response and low scores (0,1) to a less favourable response across all questions. This is in-line with the previous study where high scores represented a positive result [14]. The total score across all responses was calculated for each question as (0 × number of responses scoring 0) + (1 × number of responses scoring 1) + (2 × number of responses scoring 2) + (3 × number of responses scoring 3). The free text comments section of the questionnaire was analysed by tallying the frequency of similar comments.

Data collated from the questionnaire were analysed using Cronbach’s alpha (α), a statistical analysis of internal validity, testing the reliability of the questionnaire. Cronbach’s alpha produces results between 0 and 1, with a high scoring alpha representing good internal consistency within the questionnaire. If all questions in the questionnaire measured the same outcome, in this case tolerability of the MRL procedure, a high alpha correlation will be produced [24]. Reported levels of acceptability range from 0.7 to 0.95, but it is important to note that an alpha greater than 0.9 suggests repetition in the test and that some items are redundant [25]. Cronbach’s alpha was calculated as follows [26)].

$$\alpha =\left\{\frac{k}{\left(k-1\right)}\right\}\ast \left(1-\left(\left(\sum s_1^2\right)/s_t^2\right)\right)$$

Where, k = number of questions

S_i = SD of i^th question

S_t = sum of score

Results

Five departments out of fourteen consortium members agreed to participate and returned questionnaires from their initial cohorts of patients. Three were from English-speaking countries, two from the UK, one from the USA and the remaining two from mainland Europe; Denmark and Germany.

A total of 170 completed questionnaires were received from patients undergoing radiotherapy to a range of tumour sites, with most treatments in the pelvic region (35%). Other treatment sites included: abdominal (pancreas and liver) (26%), thoracic (partial breast) (10%), head and neck (10%) and other sites, including oligometastatic disease (19%). All treatments were MRgART delivered on a 1.5 T MRL. The time-point and location of questionnaire completion varied between centres according to site protocol. Table 2 demonstrates patient numbers and differences in collection methods for each site that provided data to this study. The Questionnaire response rate was 92% across all participating centres. Response rates for each question within the questionnaire ranged from 96% to 100%. The question with the lowest response rate was “I found listening to the music helpful whilst having my treatment”.

Patient acceptability of treatment on the MRL was high with more than 80% of patients reporting it was moderately or very easy to maintain or stay in treatment position (Fig. 1). Analysis across all questions resulted in an overall 84% favourable response rate (scores 2 and 3) and 16% unfavourable (scores 0 and 1). “I needed more detailed information before my treatment” and “I felt hot during my treatment” both reported the least favourable distribution of scores, while “I experienced a metallic taste during my treatment” and “I wanted to come out of the machine during my treatment” reported the most favourable distribution of scores (Fig. 2).

Fig. 1.

Percentage of responses on the Likert scale by question.

Fig. 2.

Percentage of response scores by question (after post processing of negatively phrased questions).

The mean (standard deviation) score per question across the cohort was 393 (30.8) out of the maximum possible total score of 510 if all responses for that question had scored most favourably. Total scores per question ranged from 453 (“I experienced a metallic taste during my treatment”), a favourable response meaning patients did not experience a metallic taste, to 351 (“I forced myself to manage the situation”) meaning that some patients did force themselves to manage the situation.

Questionnaire responses were analysed by theme with all themes demonstrating the highest percentages in the two most favourable categories (scores 2 and 3), reinforcing the positive overall outcome (Fig. 3). The physical discomfort theme had the highest percentage of results (58%) scoring 3 (most favourable), while the informational needs theme had the highest percentage of results scoring 0 (7%) and 1 (18%), both non favourable results. Although situational coping was the only theme that did not report the majority of responses in the most favourable score (3), it was the theme with the highest result in score 2 and so a favourable outcome overall.

Fig. 3.

Percentage of response scores by theme.

In the comments section a range of responses were recorded. The most common comment was describing specific areas of discomfort (4), followed by feeling cold (3). Other topics mentioned in the comments were; dry mouth after treatment, metallic taste after treatment, tingling after treatment, treatment time too long and a request for softer lighting. One patient reported communication issues with staff as “microphone crackled”. Some patients noted in the comments that they had requested to have no music during their sessions.

A high Cronbach’s alpha (α = 0.8) was calculated according to Mondal and Mondal [26], representing reliability and good internal consistency within the questionnaire. The I-CVI for each question was 0.78 or above and the SCVI was 0.93, validating the questionnaire and its contents as relevant and appropriate.

Discussion

Patient responses to the questionnaire were collated from the five participating centres. The range of treatment sites was broad and representative of the varied range of diagnoses treated on the MRL [15], [16], [17]. Response rates for each question within returned questionnaires were high, with all questions recording frequent responses. This suggests that all items of the questionnaire were appropriate and easy to answer. The lowest response rate to the question “I found listening to the music helpful whilst having my treatment” was because not all patients wanted to listen to music. Therefore, this question could not be answered as it was not applicable to that individual. This provides scope for improvement of the questionnaire to allow for patients to respond “not applicable”, instead of leaving it unanswered, when a question is not relevant to their experience. This would improve the response rate for individual questions. Although the questionnaire was administered at different times during the treatment course, this has been shown to have no effect on the responses [14], hence all results were collated. Overall response rates for each centre were high, with low numbers of un-returned questionnaires (Table 2). This re-enforces the successful development of the questionnaire, which was designed to be quick and easy to complete and could be done immediately after treatment or at home.

The questionnaire scored within the recommended ranges for both I-CVI and SCVI, validating the content of the questionnaire as relevant. Response data analysed using Cronbach’s alpha returned a good result, signifying that the questionnaire is internally consistent and without unnecessary repetition. The questionnaire can be used effectively to assess experiences of patients undergoing MRgART and the number of centres requesting to use it is increasing. The requests have been received via meetings with, and visits to, centres using the MRL, indicating that posting on the web page was a less effective method of distribution. The applications of this questionnaire will be expanded in the future to include tumour site-specific versions and non-MRI treatment technology, to continue to provide healthcare professionals with an appropriate measure of patient experience in a range of radiotherapy settings and specific treatment sites and fractionation schedules.

The international multi-centre results represent a favourable outcome where the majority of patient experiences were positive (85%). Across the range of questions in the study, scores were consistently high with a small range and SD, suggesting little deviation between scores for all questions. The lowest scoring question, “I forced myself to manage the situation”, is not an unexpected result as previous studies have reported anxiety in up to 35% of patients undergoing MRI procedures [27], which the patient may feel the need to manage in order to continue the session. However, since 75% of responses to this question were favourable (scores 2 and 3), it represents good levels of patient satisfaction, even in this lowest scoring portion of the questionnaire. While individual questions cannot be compared with previously published single centre studies, overall positive results are consistent with greater than 95% of patients reporting MRgRT as well tolerated [18], [19], [20].

The number of responses from each centre may be a limiting factor in this study, as more than 50% were provided by a single centre, potentially skewing the data. On comparison, it was found that the centre with the highest number of responses (n = 103) showed less favourable overall results (78% positive) compared to the other centres combined (n = 67) (93% positive). This is caused by the centre’s selection process, where the least favourable response per patient was returned for inclusion. Therefore, this study may represent a more negative overall outcome than would be presented if all centres contributed equal numbers of questionnaire responses.

Results analysed by theme showed positive outcomes across all themes. Situational coping scored the highest percentage of score 2 and included questions such as “I found the treatment position comfortable”, “I found the treatment bed comfortable”, “I felt calm during my treatment” and “I found the time taken for the treatment easy to tolerate”. While this is a favourable result, the smaller number of top scoring results compared to the other themes, suggests that patient experience within this theme was good but not excellent. This presents scope for further improvement.

It was interesting to note that noise, which has been reported as an issue in MRI examinations [12], [13], was reported as tolerable, even though patients were attending multiple times and images were acquired constantly or frequently during sessions. Reports of noise disturbance has ranged from 3% to 60% in previous MRgRT reports [18], [19], [20] and may be due to the number and type of images acquired. Tetar reported disturbing noise as their most frequently reported complaint (60%) which differs greatly from the findings of this study, but also reports additional noise caused by retraction of the cobalt source on top of that caused by MRI image acquisition [19], which would not be relevant for other non-cobalt MRgRT systems.

The Informational needs theme recorded the highest percentage of unfavourable results, highlighting this as an area that could be improved to enhance the patient experience. It has been proven by previous studies [28] that patients’ informational needs change over time. Within the informational needs theme in this research, the question “I needed more detailed information before my treatment” scored poorly when compared to other questions within the same theme, for example “I understood the procedure”, suggesting the need for improvement of patient preparation and information giving prior to treatment. The scores for “I needed more detailed information before my treatment” varied between centres, most likely as methods for providing information to patients, such as oral or written, was not standardised in this study and would have been completed according to the centre’s own protocols. Tetar also recommended improvements to patient information giving at the pre-treatment setting, including the option to experience the MR bore [19].

Feedback from a PPI group related to the MRL at the lead centre (RMH) reported that the questionnaire was clear and quick to complete. It was suggested that if the questionnaire was given at fraction one, it could be reviewed with the patient prior to fraction two. Actions can then be implemented to resolve any issues that have arisen, hence improving the patient experience from thereon. This was illustrated by the use of the free text comments section by patients, which provided an opportunity for them to be more specific about their experiences. Feeling cold during treatment was an interesting result as the questionnaire asks specifically about feeling hot, a potential side effect of MR imaging. Feeling cold pertains to the environment and was the most common negative feedback previously reported with 24%, 29% and 61% of patients in single centre studies involving 34, 150 and 90 patients respectively [18], [19], [20]. The immediate written feedback from the questionnaire enables staff to further improve patient experience by adapting the environment. For example, at the leading centre any patient that reported feeling cold during treatment was offered a blanket at subsequent treatment sessions.

We have developed an internationally validated questionnaire which shows that treatment sessions are well tolerated by patients, despite the extended treatment times required for MRgART on a 1.5 T MRL [15], [29]. This is an encouraging outcome of positive patient experiences during treatment using this advanced treatment system and can provide confidence to users in continuing to expand and develop the use of the MRL in adaptive radiotherapy. Through the development of this questionnaire from previously validated radiotherapy and MRI specific questionnaires [14], [21] the questions included remain relevant to standard radiotherapy techniques. Therefore, the validated MRL questionnaire provides the opportunity to collect patient experience data from the MRL and other treatment techniques such as traditional C-arm linacs, for direct comparison.

Conclusion

The MRL questionnaire has been validated as relevant and appropriate for use in patient reported experience when undergoing treatment on the MRL. The overall patient-reported experience and satisfaction from multiple centres within the Elekta MRL Consortium, was consistently high across all questions and themes in this questionnaire. MRgART on the MRL is a viable treatment option and is well tolerated by patients. Use of the MRL can be expanded in the future to include additional indications and workflow methods, with the knowledge that patient acceptability will not be compromised. This questionnaire can also be used as a foundation to develop further methods of assessing and eventually improving patient experiences during the radiotherapy pathway.

Funding

This report is independent research supported by the National Institute for Health Research and Health Education England (HEE/ NIHR ICA Programme Senior Clinical Lectureship, Dr Helen McNair, ICA-SCL-2018-04-ST2-002). We also acknowledge NHS funding to the NIHR Biomedical Research Centre at The Royal Marsden and The Institute of Cancer Research.

The Institute of Cancer Research is supported by Cancer Research UK Programme Grants (C33589/A19727) and the CRUK ART-NET Network Accelerator Award (A21993); MRC Grant MR/M009068/.

The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health and Social Care.

ICR/RMH is a member of the Elekta MR Linac Research Consortium

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following are the Supplementary data to this article: