Skip to main content
Neuro-Oncology Practice logoLink to Neuro-Oncology Practice
. 2018 Oct 17;6(3):226–236. doi: 10.1093/nop/npy040

Quality improvement of neuro-oncology services: integrating the routine collection of patient-reported, health-related quality-of-life measures

Rocío Fernández-Méndez 1,, Rebecca J Rastall 1,, William A Sage 1,, Ingela Oberg 2, Gemma Bullen 2, Amy Louise Charge 2, Anna Crofton 3, Thomas Santarius 2,1, Colin Watts 2,1,, Stephen J Price 2,1, Andrew Brodbelt 3, Alexis J Joannides 2,1
PMCID: PMC6656295  PMID: 31385996

Abstract

Background

Brain cancer has a strong impact on health-related quality of life (HRQoL), and its evaluation in clinical practice can improve the quality of care provided. The aim of this project was to integrate routine collection of HRQoL information from patients with brain tumor or metastasis in 2 specialized United Kingdom tertiary centers, and to evaluate the implementation process.

Methods

Since October 2016, routine collection of electronic self-reported HRQoL information has been progressively embedded in the participating centers using standard questionnaires. During the first year, the project was implemented, and the process evaluated, through regular cycles of process evaluation followed by an action plan, monitoring of questionnaire completion rates, and assessment of patient views.

Results

Main challenges encountered included reluctance to change usual practice and limited resources. Key measures for success included strong leadership of senior staff, involvement of stakeholders in project design and evaluation, and continuous strategic support to professionals. Final project workflow included 6 process steps, 1 decision step, and 4 outputs. Questionnaires were mostly self-completed (75.1%), and completion took 6-9 minutes. Most patients agreed that the questionnaire items were easy to understand (97.0%), important for them (93.0%), and helped them think what they wanted to discuss in their clinical consultation (75.4%).

Conclusions

Integrating HRQoL information as a routine part of clinical assessments has the potential to enhance individually tailored patient care in our institutions. Challenges involved in innovations of this nature can be overcome through a systematic approach involving strong leadership, wide stakeholder engagement, and strategic planning.

Keywords: brain tumors, process evaluation, quality improvement, quality of life


Primary brain tumors and brain metastases have a strong impact on health-related quality of life (HRQoL). This is a consequence not only of the disease process directly, but also secondary to treatment effects, including surgery and adjuvant therapy, as well as to the psychological effects that an incurable disease has on the patient and his or her family and friends.1,2 These involve symptoms like seizures, headaches, and fatigue; physical deficits such as hemiplegia and dysphasia; cognitive deficits such as memory loss and confusion; difficulties in regulating affect and behavior; and psychological distress including depression and anxiety.3,4 Therefore, care for these patients should be focused not only on increasing survival, but also on maintaining or improving HRQoL and function for the individual’s remaining life span.5,6

Accordingly, patient-reported outcome measures (PROMs) of HRQoL are the most commonly used outcome measures in the evaluation of the quality and impact of treatments in these patients. PROMs are defined as patient-centered outcome measures of health status as perceived by themselves.7 Instruments measuring HRQoL are defined as multidomain PROMs that evaluate patients’ self-assessed ability to function in 4 main areas of day-to-day life named physical, psychological, emotional, and social domains,2 as well as key symptoms induced by the disease or its treatment.7 Furthermore, the routine evaluation of PROMs has been found to increase overall satisfaction with care provided for patients with cancer, including satisfaction with communication about emotional concerns, symptom control, and the supportive care provided.8

Previous evidence suggests that the measurement of HRQoL in neuro-oncology outpatient clinics is feasible and generally welcome by patients and informal carers, and it has the potential to improve patient-centered care.9–11 Among the benefits highlighted are better detection of symptoms by professionals, higher attention to symptoms over time among patients and carers, and better management of issues identified.9,10 Consequently, it is essential to integrate ways of measuring the well-being and level of functioning of patients with brain cancer in clinical practice, along their entire care pathway. Despite this, widespread implementation of instruments measuring HRQoL in clinical practice remains a challenge because of a combination of human, practical, logistic, and technical factors.12,13 There is scarce evidence about the optimal methods to overcome these challenges and successfully integrate such PROMs as a core element of neuro-oncology care. With this background, we aimed to embed the routine collection of electronic self-reported HRQoL information for patients with brain tumors in 2 specialized United Kingdom (UK) tertiary centers, and evaluate this implementation process. We report the main elements and approach of the service improvement project, and present key results of the implementation process evaluation.

Methods

The Service Improvement Project

Since October 2016, the routine collection of HRQoL information has been gradually embedded in the neuro-oncology services of the 2 participating tertiary centers, using the European Organization for Research and Treatment of Cancer (EORTC) QLQ-C30 and QLQ-BN20 questionnaires. The EORTC generic QoL questionnaire for patients with cancer (QLQ-C30) is the most commonly used instrument to evaluate the disease and treatment impact on patients with cancer.8 It provides a multidomain PROM that is relevant to most cancer patients, and is composed of 30 items grouped in 5 functional scales, 3 symptom scales, a global health status or HRQoL scale, and 6 single items.14 The EORTC brain tumor module (QLQ-BN20) is a 20-item questionnaire specifically designed and validated for patients with brain cancer, including brain cancer-specific deficits, disorders and symptoms, future uncertainty, and treatment side effects.15 The QLQ-C30 and QLQ-BN20 are widely used questionnaires in HRQoL studies on the impact of brain tumors and their treatment, including high-16 and low-grade gliomas,1 meningiomas,17 and brain metastases.18

The questionnaires must be self-completed by patients in clinic prior to their consultation using a tablet device, and submitted electronically to a secure, web-based health informatics platform, the Outcome Registry Intervention and Operation Network (ORION). ORION is a national health informatics platform that already supports several electronic care management and disease registry services within the UK. All information submitted on the platform is saved within secure servers in real time, making data readily accessible online, instantly and securely to authorized members of the care team, including across multiple health care institutions.

The selected time points of the questionnaire completion include: (a) at the first clinic appointment, where probable diagnosis and treatment options are to be discussed with the patient; and (b) at subsequent appointments after biopsy and/or tumor resection surgeries, the first usually being within 1 to 4 weeks after the surgical intervention.

Implementation Evaluation

During the first year, the project implementation was continuously evaluated following the principles and methods of action research.19,20 Action research aims to reveal issues or problems, and solutions for these, in specific situations and localized settings, to promote or facilitate change. This is performed using iterative cycles of situation analysis, planning, action (implementation of change and monitoring), and evaluation, during which insiders and outsiders participate.19,20 Insiders are the research participants, or the individuals who will ultimately implement or benefit from the change, and they are actively involved in determining priorities to different degrees. The outsiders are the researchers, or the team managing the initial implementation process, who hold the main responsibility over the project directions. In our project, the insiders included the clinical and administrative teams at each institution, patients, and the software development team; while the outsiders were a support team specifically created for project implementation.

The support team trained and continuously assisted the clinical team and included health care assistants (HCAs), clinical nurse specialists (CNSs), neurosurgery consultants, and administrative staff. They were trained both in the questionnaire administration, typically carried out by HCAs, and the access and usage of the questionnaire results, carried out mostly by neuro-oncology surgery physicians and CNSs. Support was normally provided in situ, during the neuro-oncology surgery clinics, and involved initial training sessions, continuous assistance, and weekly discussions about practical issues, barriers, and possible solutions identified by the teams. Additional meetings among the project support, software, and clinical teams were organized regularly to discuss the implementation process and meeting logs were systematically kept. Following team discussions, the support team, in conjunction with relevant members of the clinical or software teams, would design an action plan to solve the issues, overcome barriers, and make the most of facilitators. For this, an initial compilation of the identified issues, barriers, and proposed solutions was conducted by 1 person from the support team and shared with key clinical representatives and the software team. Based on their feedback, the final action plan was updated by the support team representative. The action plan included what should be performed, when, how, and by whom, and was monitored and discussed in a subsequent meeting.

Questionnaire completion was monitored at random days to evaluate recruitment rate, defined as the percentage of patients attending the clinic who submitted the questionnaire, reasons for nonrecruitment, and the time needed by patients to complete the entire questionnaire.

Patient views

During the last 3 (Center 2) to 7 (Center 1) months of this study period, 4 4-level Likert scale questions about patients’ views on the relevance, ease, and length of the information collected were included at the end of the EORTC questionnaires as a mandatory item. These questions were based on previous literature highlighting the usefulness of collecting HRQoL PROMs from patients with cancer.21–26 Results were analyzed descriptively using counts and percentages. Statistical significance of differences in answers by center were tested to account for potential center effect, using chi-squared or Fisher exact tests, as required. Significance level was set at 0.05. Outliers were explored in depth to determine whether there may have been a data error or reasons why these may have happened.

Project workflow

The final set of necessary steps, outputs, and people involved in the project were summarized schematically using open-loop flowcharts. Steps included processes (or actions) and decisions. Processes were represented with a rectangle, decisions with a diamond, and outputs or data with a parallelogram, following the International Organization for Standardization 5807 standard.27

Graphical representation

Pie charts and graph bars were used to represent the frequency distribution of categorical variables. The number of questionnaires submitted per month was represented using time-line graphs. Relevant graphs were presented separated by center to account for potential center effect.

This was a nonresearch, service improvement project, and thus it did not require the ethical approval of a research ethics committee. The project was registered as a service improvement program with local audit departments.

Results

Implementation Barriers and Measures for Success

The main challenges and barriers identified during the project implementation are summarized in Table 1, and further details about key practical issues encountered and their subsequent action plan can be found in Supplementary Table 1. Challenges involved practical issues and factors related to attitudes and beliefs of clinicians, patients, and carers. Practical issues included technical, workload, and time-related challenges. Attitudes and beliefs mainly concerned potential and initial misunderstandings about the project, and reluctance to change usual practice (Table 1).

Table 1.

Main Barriers and Challenges Encountered During the First Year of the Project Implementation

Main Barriers and Challenges
Reluctance to change usual practice related to:
Initial views of the project as a research activity, rather than a component of the service
Unawareness of the relevance and usefulness of health-related quality of life (HRQoL) information
Insufficient time for some of the tasks
Difficulties with the usage of information technologies
Difficulties with real-time routine use/interpretation of HRQoL raw scores during a patient’s consultation
Insufficient resources (time and devices) to approach all patients during clinic
Higher infection transmission risk with shared tablets
Concerns with tablets being taken away by patients, carers or professionals
Technical issues involving the software platform or the WiFi connection
Occasional inability to log-in into the software (expired accounts, new staff, etc.)
New staff unawareness, or current staff occasional forgetfulness, of the HRQoL administration or usage methods
Risk of patients/carers seeing the project as research, not related to their care pathway

A list of key measures to address identified practical issues and other implementation challenges, and make the implementation successful, is included in Table 2. Among the key measures implemented, the strong leadership and support from senior staff in the care team were crucial for the project embracement by the rest of the team. The involvement of the multidisciplinary care team in all phases of the implementation was essential too, not only to gather firsthand feedback, identify issues, and design an action plan to solve them (an example is included in Supplementary Table 1), but also to ensure a realistic division of roles and responsibilities (Table 2). The continuous training of and support to, as well as regular gathering of feedback from the staff, were key for the continuous improvement of the project. This required the availability of a project support team within the clinic during the first year of the project, permanent availability of a software support team, contactable via telephone or email, and regular meetings among the implementation support, care, and software teams (Table 2). Finally, locally tailored patient information sheets and project implementation guidelines, which consisted of simple but comprehensive instructions supported by infographics (Table 2), were instrumental to engage patients and to decrease the number of issues encountered by staff.

Table 2.

Key Measures for Success During the First Year of the Project

Key Measures for Success
Clear, strong leadership and support from senior staff
Involvement of stakeholders in the project design and implementation evaluation, including the multidisciplinary neuro-oncology team, patients and their carers, and the implementation support and software development teams
Division of roles and responsibilities agreed by the implementation support, clinical, and software development teams
Continuous evaluation of the implementation process through:
Regular meetings with the professionals to analyze the situation and develop an action plan, followed by its implementation and evaluation in subsequent meetings
Monitoring of data collection recruitment rate and reasons for decreases in this rate
Collection of feedback from patients on the relevance, ease, and length of the questionnaires
Initial training of, continuous support to, and regular feedback collection from staff on the preparation and administration of questionnaires
Availability of a project support team in the clinic, including a main coordinator, for the first 12 months of implementation, allowing the solving of any practical issues on the go
Availability of a support team, contactable via telephone or email, to address software-related issues
Availability of at least 2 internet-connected, traceable, fully charged tablet devices for each clinic, with a screen protector that can be wiped with disinfectant
Tailoring of software to the needs of clinicians and patients
Design, distribution, and continuous refinement based on feedback, of locally tailored guidelines for professionals describing different aspects/procedures of the implementation:
Guidelines for the preparation of the patient list (and unique identifiers) before clinic
Guidelines for the introduction and administration of health-related quality of life (HRQoL) to patients and carers during clinic
Design and distribution of locally tailored, hospital-approved welcome sheet for patients describing HRQoL implementation, sent by mail in advance, or handed-in by reception staff on the appointment day

In Center 1 there was a progressive departure of the support team from the neuro-oncology clinic until the team stopped supporting data collection in October 2017. As a consequence, during the last quarter of 2017 there was a decrease in the number of questionnaires submitted, and thus in the capture rate of patients. A team meeting to reinforce the relevance of the project for the clinical team and patients, and to rearrange the roles in the implementation, was determinant to bring the number of submissions back to normal in Center 1.

Implementation Flowchart

The final workflow that resulted from the continuous refinement process, including all steps, outputs, and people involved, is depicted in Fig. 1. This included 6 process steps, 1 decision step, and 4 outputs, organized into 3 main parts named preparation, questionnaire administration, and data utilization. People involved included patients and carers, HCAs, CNSs, neurosurgery consultants, and administrative staff. Details about the final system established for each of the 3 phases are described below.

Fig. 1 .

Fig. 1

Implementation Flowchart of the Routine Collection of Health-Related Quality of Life Information in Neuro-Oncology Surgery Services. CNS indicates clinical nurse specialist; COT, clinical outcomes team; HCA, health care assistant; ORION, Outcome Registry Intervention and Operation Network; PA, personal assistant; QoL, health-related quality of life; QoL-Q, QoL questionnaire.

Preparation

The preparation phase takes place before a neuro-oncology surgery clinic, and involves loading an HRQoL questionnaire for each patient of a clinic list into his or her ORION record. This is usually performed by the personal assistant of CNSs (Center 1) or by the clinical outcomes team (Center 2). A list of unique identifiers for each of the questionnaires loaded is automatically created by ORION in printable format, and will be subsequently used in clinic by relevant staff (Fig. 1).

Questionnaire administration

Questionnaire administration takes place when patients attend a neuro-oncology surgery clinic while waiting to be seen. A member of the clinical team, usually an HCA, loads the questionnaires onto a tablet device using the previously created questionnaire unique identifiers for each patient questionnaire. Alternatively, prepared questionnaires can also be accessed through an ORION account with the relevant access permissions. Subsequently, the HCA identifies, invites and, if required, helps patients to complete the questionnaire. If patients require help, the team emphasizes and puts all efforts to reflect the most accurate answer given by patients. Carers are instructed to not answer the questions on their own, but only help patients reminding them about recent events that could help them give the most accurate answer. Patients with cognitive impairment that would not allow them answer the questionnaire, as judged by the clinical team, are not approached.

Data utilization

The data utilization phase refers both to the clinical and research usage of the data. Summary scores and graphs are accessible by professionals online through ORION immediately after the patient submits the questionnaire, allowing them to use these and provide feedback while they see the patient (Fig. 1). In addition, ORION has an anonymized export option by which each hospital can extract its HRQoL information, including raw data, for audit and research purposes.

Questionnaire Administration Monitoring

Patient characteristics

During the study period, 466 (45.8%) and 552 (54.2%) patients from Center 1 and Center 2 respectively submitted at least 1 questionnaire. Median (interquartile range [IQR]) age at the time of first questionnaire submission was 57 (44-67) years, and 52.3% were females (Fig. 2). Age and sex distributions of patients attending neuro-oncology surgery clinics are similar to the national figures of patients registered in the Office for National Statistics (ONS) Cancer registration statistics for England in 201628 (Fig. 2), with the exception of an underrepresentation of individuals over 80 years, who are less likely to be suitable surgical candidates.

Fig. 2.

Fig. 2

Age group and sex distributions of A, patients who completed the questionnaires between October 2016 and April 2018, and B, patients with brain cancer in England, including International Classification of Diseases, 10th Revision codes C70, C71, C79.3, D32, D33, D42, and D43 (Source: Office for National Statistics. Cancer Registration Statistics, England—dataset for 2016).

Questionnaire administration

Questionnaire administration monitoring showed that completing the questionnaire took a median (IQR) of 7 (6-9) minutes. Questionnaires were mostly self-completed (75.1%), with a smaller proportion completed with the help of a carer (19.2%) or a health professional (5.7%) (Fig. 3). Patients who needed some help did not differ in their sex distribution but were significantly older than those completing the questionnaire alone, including 44.4% patients 65 years or older in the group requiring some help, compared to 27.7% in the more independent group (Fig. 3).

Fig. 3 .

Fig. 3

Features of Questionnaires Submitted Between October 2016 and April 2018. A, Number of monthly questionnaires submitted at each center. B, Distribution (percentage) of submissions by type of help given to patients during questionnaire administration. C1, Distribution (percentage) of age groups in patients who required some help to complete the questionnaire. C2, Distribution (percentage) of age groups in patients who did not require any help to complete the questionnaire.

Between October 2016 and April 2018, the median (IQR) number of questionnaires completed monthly was 34 (23-40) and 36 (29-45) in Center 1 and Center 2, respectively (Fig. 3), which represented a 75% to 85% capture rate. Several technical and human factors contributed to the capture rate not being 100%. Technical issues, such as ORION access timing out, or difficulties with accessing and using the platform by the clinical team, were reduced over time both with software upgrades and the professionals getting used to the technology. Problems due to HCAs having insufficient time to approach a patient to complete the questionnaire were also reduced progressively, as professionals got used to the newly introduced element. On the other hand, at times there were still situations when missing patients could not be avoided, such as particularly busy clinics or other tasks of the team taking priority, such as HCAs having to assist a clinical procedure or chaperone a medical examination.

Outliers in the number of monthly submissions included 7 questionnaires submitted in December 2017 in Center 1, and 105 questionnaires submitted in November 2017 in Center 2 (Fig. 3). The reasons for the decrease in monthly submissions in Center 1 have been described earlier. In Center 2, an additional 5 clinics were carried out in November 2017 as part of the National Health Service (NHS) Waiting List Initiative, which explains the peak.

Patient Views

During the 7 (Center 1) and 4 (Center 2) months during which patients were asked about their views on the QoL-Qs, 128 (Center 1) and 71 (Center 2) patients completed the questionnaire. The clear majority of patients agreed (completely or partly) that the questionnaire is easy to understand (97.0%) (Fig. 4) and its length is right for them, neither too short nor too long (86.4%) (Fig. 4).

Fig. 4.

Fig. 4

Distribution (Percentage) of Patient Views About the Questionnaire, by Center. A, Importance, usefulness, and ease of the questionnaire. B, Length of the questionnaire.

Most patients also agreed that the items in the questionnaire were important for them (93.0%) and helped them think of what they wanted to discuss during their clinic appointment (75.4%) (Fig. 4).

In all questions about patient views on the questionnaire, differences in the distribution of answers by center were not statistically significant.

Discussion

The ultimate aim of this multicenter quality-improvement project is to contribute to the provision of excellent care through the routine assessment of patients’ self-reported QoL, which is 1 of the key outcome measures of effectiveness in the treatment and follow-up of patients with brain tumors.5,29 Originally proposed by senior clinical staff in each of the participating institutions, the aim was for this initiative to be both meaningful for professionals and patients, and sustainable in the long term. Thus, the project required a phase of progressive introduction of the new component within the patient care pathway (the routine assessment of their HRQoL) while involving all stakeholders and continuously evaluating and adapting the implementation process.

Given its intrinsically flexible and pragmatic methodology, conclusions derived from action research may not be directly generalizable to other contexts or institutions.30 However, this was considered the optimal approach for our aims of changing routine clinical practice in a meaningful and sustainable way. Continuous evaluation followed by strategic action, and the involvement both of the research participants and researchers, while taking into account the organizational context, are fundamental characteristics of action research,30 and are crucial for the successful implementation of quality-improvement projects.31 Other existing methodologies include structured or semistructured interviews with professionals and/or patients about how they experience the implementation process. This was beyond the scope of this project, and future studies based on such an approach are encouraged, including the evaluation of the longer-term project experience.

Our findings have confirmed that implementing a quality-improvement initiative may entail important challenges,32 which can however be mitigated or eliminated through realistic and collaboratively proposed measures.30 Practical issues were the most straightforward challenges to identify and find possible solutions for. Similarly, practical obstacles, such as initial struggles with the use of the software among some of the staff, were also identified in a similar study.9 Regular discussions, training, and close support were decisive to achieve correct usage and functioning of the information technologies implicated, as well as to make the patient information gathered easy to access and understand for the professionals. The importance of staff training in the use of QoL data and of the adaptation of technologies to the needs of professionals has also been pointed out in previous studies.9,33 Frequent meetings with stakeholders were also key, not only for the identification and implementation of feasible solutions to mitigate the existing limited time and resources, and other practical issues, but also to empower and motivate professionals toward the project. Similarly, the effectiveness of interactive, small-group meetings for the successful introduction of change in patients’ care has been suggested by a number of studies.34

The most challenging barriers for successful project implementation were professionals’ reluctance to change usual practice, and clinical inertia. This was a recurring theme at regular implementation meetings, and demonstrated in the transient decline in monthly submissions after withdrawal of support from the implementation team in Center 1. Difficulties in changing culture, especially when it entails additional workload, are well-known barriers to the success of quality-improvement initiatives,32,34,35 and have also been identified in similar projects.9 Strong leadership, and stakeholders’ involvement and focused strategy, were fundamental for the embracement of the project by professionals and, indirectly, by patients. The need for continued staff and leadership engagement for the sustainability of quality-improvement projects is largely supported by the literature.35,36 In fact, the European Foundation for Quality Management Excellence Model describes exemplar and flexible leaders, strategic direction, and valuing and involving stakeholders as 3 key criteria to enable excellent organizations to achieve their goals.31

On the other hand, professionals’ reluctance to change was helpful to challenge the project and foster more strategic thinking while refining the approach, making action plans more sensible and sustainable in the long run. In this regard, resistance to change has been described as a potentially positive process that can encourage learning among participants and improve the quality of the decision making.32

The resulting final workflow guiding the implementation included only 6 processes and 1 decision step, clearly identified the people involved, and depicted the main outputs. Such features of the guiding workflow are important, as clinical guidelines that are evidence based, concrete and precise, easy to follow, and not complex have been shown to improve adherence to them.37

Ascertaining whether the implementation has had a meaningful impact on patients’ QoL would require an evaluation of how data are utilized by clinicians and patients, but this was beyond the scope of this project. As a surrogate metric, however, most patients found that completing the questionnaire helped them remember the issues they wanted to discuss during their consultation, which indicates a positive impact of the project. This has also been suggested in previous studies in which patients recognized the value of monitoring their symptoms or QoL to facilitate or animate subsequent discussions with their clinician.9,10

Most patients were able to complete the questionnaire on their own, although about one-quarter required the help of a carer or professional. Age seemed to be a factor and, in some instances, this was due to visual impairments of the patient, but it could also be a result of the cognitive difficulties associated with brain cancer. There is evidence that cognitively impaired patients show large and statistically significant differences in a number of QoL domains or scores when compared to proxies answering on their behalf.38 Therefore, in our project we included only patients who could answer the questionnaires themselves, either alone or with the help of a professional or carer.

The time points selected for the questionnaire administration may vary among patients. While this could be seen as a limitation for comparability and research purposes, in practice it is not realistic to establish fixed time points for the questionnaire administration. If fixed time points were proposed, most patients would be missed because their appointments’ time points usually vary among patients. In line with this, it has been suggested that completion-time windows, relative to key events, for the completion of a questionnaire should be allowed, so that aggregate data can be used for audit or research purposes.7

Missing responses within questionnaires were not an issue in our project, as the electronic questionnaire submission is conditional on all answers being completed. While some patients were missed, recruitment rates were usually above 75%, and the reasons for missed patients mainly included practical issues such as initial difficulties with the use of tablets by clinical staff, insufficient time during busy clinics, or other priorities of the clinical team. Similarly, administrative failures (patient- or researcher-led) have been previously identified as the main source of missing data in collecting HRQoL data from glioma patients.7 In our project, most patients were willing to complete the questionnaire electronically and showed a positive view about this. In addition, the median age and sex distributions of our sample are very similar to the national figures of patients registered in the ONS Cancer registration statistics for England in 201628 (Fig. 2). Currently, we are evaluating measures for obtaining responses from missed patients in clinic, including during the preoperative assessment visits for baseline responses, and by emailing links for online access to the questionnaire for follow-up visits.

The high acceptance and positive views of patients about the project are among our most encouraging findings. The great majority found that questions were easy to understand, and rated the 6 to 9 minutes that it took them to submit the questionnaire as neither too long nor too short. Crucially, most of the patients described questions as important for them and even as useful to remind them of what they wanted to discuss during their subsequent consultation. Previous studies have concluded that the routine assessment and use of HRQoL information increases practitioners’ awareness of health problems in several domains of HRQoL,23 facilitates shared decision making and the raising of sensitive issues,8,22,25,39 enhances patient and clinician satisfaction,23 and can ultimately improve HRQoL scores.8,25 Additionally, the use of a widely accessible platform for capturing HRQoL enables the data to be shared and extended by different health care providers across the patient pathway. While patients did not have direct access to their own answers to the questionnaire, further work on implementing a patient-facing portal that allows this, together with providing additional relevant support information, is currently ongoing.

The two centers participating in this quality-improvement project have used the results from the HRQoL to evaluate the impact of surgery on patients’ HRQoL, for clinical and research purposes, and preliminary results of these impact evaluations have been reported previously.40,41 This demonstrates the usefulness of the project not only for clinical purposes, but also for building on current evidence on the impact of brain cancer and its treatment. In fact, 2 of the 10 priority questions for research identified by the James Lind Alliance Neuro-Oncology focus on the impact of care on QoL,42 and thus the data collected in our project institutions will constitute a powerful repository for future research studies.

Aspiring to the highest standards of excellence and professionalism is 1 of the 7 principles of the NHS Constitution.43 As health care enters an increasingly digital era, it is important that the implementation of innovations in clinical practice utilize current technologies and electronic formats that promote patient and professional involvement, and result in accessible information and scalable clinical data. The move toward the implementation of patient records that are accessible across health-care providers, wherever they are, and that allow patients to take a more active role in their own health care, is among the main NHS service improvement priorities over the next 2 years.44 On these grounds, and given the relevance and success of the project, we strongly encourage other clinical teams to introduce a similar project as part of the care they provide. The main principles of the project, involving the change in clinical practice and all the human involvement, would require a larger effort if an external support team is not available, but is nevertheless potentially feasible if there is willingness from the clinical team. The incorporation of an electronic platform on which patients submit and clinicians can access HRQoL answers in real time is essential for the aims of this project, and a number of similar software solutions exist for this besides the one developed for this project45–47

While implementation would require some financial commitment, we believe that this is an effort that institutions aiming for excellent care should consider. Although individual contexts and particular conditions may differ from this project, our results and approach are flexible and broad enough to help guide future successful implementations.

Conclusion

Evaluation of HRQoL in routine clinical practice is feasible, and has great potential for improving the provision of individualized high-quality care. Achieving this in a sustainable way can present diverse challenges that can be overcome through a systematic approach involving strong leadership, wide stakeholder engagement, and strategic planning.

Funding

This work was supported by Innovate UK as part of the DAMSEL project within the Digital Health in a Connected Hospital initiative [RG77262 to A.J.J.].

Supplementary Material

Supplementary Tables

Acknowledgments

We thank Innovate UK for its financial support of this project. Innovate UK is part of UK Research and Innovation, a nondepartmental public body funded by a grant-in-aid from the UK government (www.ukri.org). The research was supported by the National Institute for Health Research (NIHR) Brain Injury Healthcare Technology Co-operative based at Cambridge University Hospitals NHS Foundation Trust and University of Cambridge. The views are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health. We are very grateful to the team of Obex Technologies for its continuous support and flexibility in tailoring the software solution to the needs of the project. We also thank the Brain Tumour Charity and the Brain Trust for their key contribution in patient and stakeholder engagement, and Mountain Hare Consulting for their assistance with project management. Thanks also to our anonymous reviewers for their constructive and useful comments to improve the quality of the manuscript. Finally, we would like to acknowledge all professionals, patients, and their carers, who made the aims of this project significant, and its implementation possible.

Conflict of interest statement. A.J.J. is director of Obex Technologies, lead partner of the Detection and Assessment of Malignancy by Symptom Evaluation (DAMSEL) consortium, and developer of the software solution for this project.

References

  • 1. Fountain DM, Allen D, Joannides AJ Nandi D, Santarius T, Chari A. Reporting of patient-reported health-related quality of life in adults with diffuse low-grade glioma: a systematic review. Neuro Oncol. 2016;18(11):1475–1486. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Taphoorn MJ, Sizoo EM, Bottomley A. Review on quality of life issues in patients with primary brain tumors. Oncologist. 2010;15(6):618–626. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Baker PD, Bambrough J, Fox JRE, Kyle SD. Health-related quality of life and psychological functioning in patients with primary malignant brain tumors: a systematic review of clinical, demographic and mental health factors. Neuro-Oncol Pract. 2016;3(4):211–221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Wilbers J, Kappelle AC, Versteeg L, et al. . Cognitive function, depression, fatigue and quality of life among long-term survivors of head and neck cancer. Neuro-Oncol Pract. 2015;2(3):144–150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. National Collaborating Centre for Cancer. Improving Outcomes for People With Brain and Other Central Nervous System Tumours. National Institute for Health and Clinical Excellence; 2006. [Google Scholar]
  • 6. Bush NA, Chang SM, Berger MS. Current and future strategies for treatment of glioma. Neurosurg Rev. 2017;40(1):1–14. [DOI] [PubMed] [Google Scholar]
  • 7. Dirven L, Reijneveld JC, Aaronson NK Bottomley A, Uitdehaag BM, Taphoorn MJ. Health-related quality of life in patients with brain tumors: limitations and additional outcome measures. Curr Neurol Neurosci Rep. 2013;13(7):359. [DOI] [PubMed] [Google Scholar]
  • 8. Kotronoulas G, Kearney N, Maguire R et al. . What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol. 2014;32(14):1480–1501. [DOI] [PubMed] [Google Scholar]
  • 9. Erharter A, Giesinger J, Kemmler G et al. . Implementation of computer-based quality-of-life monitoring in brain tumor outpatients in routine clinical practice. J Pain Symptom Manage. 2010;39(2):219–229. [DOI] [PubMed] [Google Scholar]
  • 10. Boele FW, van Uden-Kraan CF, Hilverda K et al. . Attitudes and preferences toward monitoring symptoms, distress, and quality of life in glioma patients and their informal caregivers. Support Care Cancer. 2016;24(7):3011–3022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Hickmann AK, Hechtner M, Nadji-Ohl M et al. . Evaluating patients for psychosocial distress and supportive care needs based on health-related quality of life in primary brain tumors: a prospective multicenter analysis of patients with gliomas in an outpatient setting. J Neurooncol. 2017;131(1):135–151. [DOI] [PubMed] [Google Scholar]
  • 12. Dirven L, Koekkoek JAF, Reijneveld JC, Taphoorn MJB. Health-related quality of life in brain tumor patients: as an endpoint in clinical trials and its value in clinical care. Expert Rev Qual Life Cancer Care. 2016;1(1):37–44. [Google Scholar]
  • 13. Snyder CF, Jensen RE, Geller G Carducci MA Wu AW. Relevant content for a patient-reported outcomes questionnaire for use in oncology clinical practice: putting doctors and patients on the same page. Qual Life Res. 2010;19(7):1045–1055. [DOI] [PubMed] [Google Scholar]
  • 14. Aaronson NK, Ahmedzai S, Bergman B, et al. . The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85(5):365–376. [DOI] [PubMed] [Google Scholar]
  • 15. Osoba D, Aaronson NK, Muller M, et al. . The development and psychometric validation of a brain cancer quality-of-life questionnaire for use in combination with general cancer-specific questionnaires. Qual Life Res. 1996;5(1):139–150. [DOI] [PubMed] [Google Scholar]
  • 16. Cheng JX, Zhang X, Liu BL. Health-related quality of life in patients with high-grade glioma. Neuro Oncol. 2009;11(1):41–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. van Nieuwenhuizen D, Klein M, Stalpers LJ Leenstra S Heimans JJ Reijneveld JC. Differential effect of surgery and radiotherapy on neurocognitive functioning and health-related quality of life in WHO grade I meningioma patients. J Neurooncol. 2007;84(3):271–278. [DOI] [PubMed] [Google Scholar]
  • 18. Soon YY, Tham IW, Lim KH, Koh WY, Lu JJ. Surgery or radiosurgery plus whole brain radiotherapy versus surgery or radiosurgery alone for brain metastases. Cochrane Database Syst Rev. 2014(3):CD009454. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Stringer ET. Action Research, 4th Ed. Thousand Oaks, CA: SAGE Publications; 2013. [Google Scholar]
  • 20. Waterman H, Tillen D, Dickson R de Koning K. Action research: a systematic review and guidance for assessment. Health Technol Assess. 2001;5(23):iii–157. [PubMed] [Google Scholar]
  • 21. Braeken AP, Kempen GI, Eekers D van Gils FC Houben RM Lechner L. The usefulness and feasibility of a screening instrument to identify psychosocial problems in patients receiving curative radiotherapy: a process evaluation. BMC Cancer. 2011;11:479. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Hilarius DL, Kloeg PH, Gundy CM Aaronson NK. Use of health-related quality-of-life assessments in daily clinical oncology nursing practice: a community hospital-based intervention study. Cancer. 2008;113(3):628–637. [DOI] [PubMed] [Google Scholar]
  • 23. Detmar SB, Muller MJ, Schornagel JH Wever LD, Aaronson NK. Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. JAMA. 2002;288(23):3027–3034. [DOI] [PubMed] [Google Scholar]
  • 24. Thewes B, Butow P, Stuart-Harris R; Greater Southern Area Health Service Screening Collaborative Group Does routine psychological screening of newly diagnosed rural cancer patients lead to better patient outcomes? Results of a pilot study. Aust J Rural Health. 2009;17(6):298–304. [DOI] [PubMed] [Google Scholar]
  • 25. Velikova G, Keding A, Harley C et al. . Patients report improvements in continuity of care when quality of life assessments are used routinely in oncology practice: secondary outcomes of a randomised controlled trial. Eur J Cancer. 2010;46(13):2381–2388. [DOI] [PubMed] [Google Scholar]
  • 26. Cleeland CS, Wang XS, Shi Q, et al. . Automated symptom alerts reduce postoperative symptom severity after cancer surgery: a randomized controlled clinical trial. J Clin Oncol. 2011;29(8):994–1000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. ISO. ISO 5807: Information processing—document symbols and conventions for data, program and system flowcharts, program network charts and system resource charts. Geneva: International Organization for Standardization; 1985. http://www.iso.org/iso/catalogue_detail.htm?csnumber=11955. Accessed July 29, 2017. [Google Scholar]
  • 28. Office for National Statistics. Cancer registration statistics, England—dataset for 2016 https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/datasets/cancerregistrationstatisticscancerregistrationstatisticsengland. Published 2018. Accessed August 7, 2018.
  • 29. NHS England. NHS Standard contract for neurosurgery (adult) 2013. https://www.england.nhs.uk/wp-content/uploads/2013/06/d03-neurosurgery.pdf. Accessed March 16, 2017.
  • 30. Harvey G, Wensing M. Methods for evaluation of small scale quality improvement projects. Qual Saf Health Care. 2003;12(3):210–214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. EFQM. Fundamental Concepts. 2018. http://www.efqm.org/efqm-model/fundamental-concepts. Accessed February 19, 2018.
  • 32. Erwin DG, Garman AN. Resistance to organizational change: linking research and practice. Leadersh Organ Dev J. 2010;31(1):39–56. [Google Scholar]
  • 33. Wright EP, Selby PJ, Crawford M et al. . Feasibility and compliance of automated measurement of quality of life in oncology practice. J Clin Oncol. 2003;21(2):374–382. [DOI] [PubMed] [Google Scholar]
  • 34. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391):1225–1230. [DOI] [PubMed] [Google Scholar]
  • 35. Wiltsey Stirman S, Kimberly J, Cook N Calloway A Castro F Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Cranley LA, Hoben M, Yeung J Estabrooks CA, Norton PG, Wagg A. SCOPEOUT: sustainability and spread of quality improvement activities in long-term care—a mixed methods approach. BMC Health Serv Res. 2018;18(1):174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Francke AL, Smit MC, de Veer AJ Mistiaen P. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Ediebah DE, Reijneveld JC, Taphoorn MJ et al. ; EORTC Quality of Life Department and Patient Reported Outcome and Behavioral Evidence (PROBE) Impact of neurocognitive deficits on patient-proxy agreement regarding health-related quality of life in low-grade glioma patients. Qual Life Res. 2017;26(4):869–880. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Rotenstein LS, Huckman RS, Wagle NW. Making patients and doctors happier—the potential of patient-reported outcomes. N Engl J Med. 2017;377(14):1309–1312. [DOI] [PubMed] [Google Scholar]
  • 40. Fernández-Méndez R, Sage WA, Rastall RJ, Wan Y, Price SJ, Joannides AJ. Impact of surgical intervention on quality-of-life of patients with brain cancer. Neurocirugía 2017;28(Espec Congr):161. https://www.revistaneurocirugia.com/en-congresos-xxi-congreso-nacional-sociedad-espanola-47-sesion-oncologia-tumores-oncology-tumours-3245-impact-of-surgical-intervention-on-35765-pdf.
  • 41. Sage W, Mendez RF, Allen B, Brodbelt A, Price S, Joannides AJ. Heterogeneity in quality-of-life of patients before and after brain tumour surgery. Neuro Oncol. 2018;20(Suppl_1):i3. [Google Scholar]
  • 42. James Lind Alliance Neuro-Oncology Priority Setting Partnerships. Neuro-oncology top 10 http://www.jla.nihr.ac.uk/priority-setting-partnerships/neuro-oncology/top-10-priorities/. Accessed May 30, 2018.
  • 43. Department of Health. The NHS Constitution 2015. https://www.gov.uk/government/publications/the-nhs-constitution-for-england. Accessed February 15, 2017.
  • 44. NHS England. Next steps on the NHS five-year forward view 2017. https://www.england.nhs.uk/publication/next-steps-on-the-nhs-five-year-forward-view/. Accessed December 7, 2017.
  • 45. Jensen RE, Snyder CF, Abernethy AP et al. . Review of electronic patient-reported outcomes systems used in cancer clinical care. J Oncol Pract. 2014;10(4):e215–e222. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46. Bennett AV, Jensen RE, Basch E. Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin. 2012;62(5):337–347. [DOI] [PubMed] [Google Scholar]
  • 47. Evaluation Software Development. CHES platform https://ches.eortc.be/cms/. Published 2018. Accessed August 10, 2018.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Tables

Articles from Neuro-Oncology Practice are provided here courtesy of Oxford University Press

RESOURCES