The iPROMOS protocol: a stepped-wedge study to implement routine patient-reported outcomes in a medical oncology outpatient setting

Abstract

Introduction

Patient-reported outcome measures (PROMs) are data capture tools that collect information directly from patients. Several large research studies provide evidence that the use of PROMs in routine care provides benefits to mortality and morbidity outcomes in medical oncology patients. Despite this, implementation of PROMs in daily clinical routine is slow and challenging.

Methods and analysis

This study will use a stepped-wedge design to assess the implementation of a PROM intervention in highly frequented medical oncology outpatient clinics. During a lead-in period of 4 weeks, control data will be collected. The intervention will then be implemented for 4 weeks in Clinic 1 initially, then in Clinic 2 for another 4 weeks. 500 patient encounters will be measured over the 12 weeks in total. The process of implementation will be informed and evaluated using the Medical Research Council Guidelines for Implementing Complex Interventions. The study will be guided by the Promoting Action Research in Health Services framework approach for implementation. The intervention and implementation outcomes will be measured using qualitative and quantitative data.

Ethics and dissemination

Ethical approval has been obtained, approval number HREC/16/QRBW/100 by the Royal Brisbane and Women’s Hospital Human Research Ethics Committee. Results will be disseminated in peer-reviewed journals and at scientific meetings.

Trial registration

ACTRN12618000398202. Trial Status: Opened on 25 March 2018 and will continue until 12 months after the last PROMs reporting encounter.

Strengths and limitations of this study.

• One non-blinded researcher will implement the intervention, and collect and analyse the data.

• Response bias and social desirability bias (of both health professionals and patients that choose to participate).

• Bias by the Hawthorne effect whereby clinics being observed during the pre-implementation phase may start to change practice.

• A stepped-wedge design ensures an incremental implementation into clinical practice.

• Prospective use of an implementation framework will make sure that enablers and barriers in the setting are collected and reported allowing the findings from this study to inform future integration of patient reported outcomes into routine clinical care.

Introduction

What are patient-reported outcome measures?

The Federal Drug Administration defines patient reported outcome measures (PROMs) as ‘any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else’.¹ Revicki et al ² describe PROMs as validated self-reporting assessment tools that capture the patient experience. PROMs have been extensively evaluated for their sensitivity, specificity, overall accuracy and predictive value. They are now regarded to have excellent precision, similar to many other widely used clinical assessment tools including pathological tests or medical imaging reports.³ PROMs can provide an overview of a patient’s physical, emotional, functional or overall health status, or can be used to assess specific treatment outcomes or symptoms.⁴

PROMs in clinical practice

PROMs are commonly used as outcome measures in research. However, more recently, there is evidence that their real-time application in clinical practice can enhance clinical interactions and improve patient experience. Several studies have shown that using PROMs in routine care leads to improved quality of life (QOL)^{3 5} as well as improved communication, decision-making, care planning and patient satisfaction.^6–8 Two recent studies demonstrated improvements in patient mortality and morbidity when technology-facilitated PROMs data collection was incorporated in oncology care.^{5 9 10}

Given these evidence-based benefits, translating these findings into practice by integrating PROMs into routine clinical care is the next required step in the implementation cycle.

The complexities of implementing PROMs into the clinical setting

A number of systematic reviews^{3 11 12} reported that multiple organisational, technical and clinical factors need to be overcome before introducing PROMs. In particular, a lack of engagement from healthcare professionals, concerns about the workflow of generating and filing of PROM reports and lack of clearly defined approaches in how to respond to the PROM data that indicate a patient need (eg, elevated pain or depression) have been identified as barriers to successful implementation. The International Society of Quality of Life (ISOQOL) advocates a stepwise approach to implementing PROMs and provides a User’s Guide,¹³ which was updated in 2018. Klinkhammer-Schalke (2014) identified that a stepwise approach was most useful when integrating a PROM intervention into routine care, as it allows cycles of iterative learning during the implementation.⁷

Incorporating PROMs into clinical practice should be considered a complex intervention, with many elements impacting on the intervention, and vice versa¹⁴ Given these complexities, it has been recommended to use an implementation framework to increase the likelihood of success when aiming to integrate PROMs into routine care.¹⁵ Use of a framework approach can help to consider both the processes and intended outcomes of implementation. The Promoting Action Research in Health Services (i-PARIHS) framework appears well suited, as it highlights elements for consideration within the context (eg, the features of the particular clinic in which PROMs are to be integrated), the stakeholders (eg, patients, clinicians, administrative staff) impacted by the intervention and the evidence surrounding the intervention (eg, how much do stakeholders value the new PROM information presented to them).¹⁶ A unique feature of iPARIHS is that it stresses the central importance of a facilitator, who works with the local stakeholders to adapt the evidence-based intervention for the local context. Antunes et al’s ³ systematic review provided evidence for the important role of a facilitator of the implementation process, with enhanced successful uptake if one was present.^{17 18} For example, Baskerville et al ¹⁷ showed that medical practices were 2.76 more likely to adopt evidence-based guidelines when a facilitator was working in the local context.

Besides the implementation framework, the Medical Research Council (MRC) Guidelines for Implementation of Complex Interventions can provide guidance on how to best incorporate prespecified process measure. The Guidelines ‘can be used to assess fidelity and quality of implementation, clarify causal mechanisms and identify contextual factors associated with variation in outcomes’.¹⁸ The MRC approach ensures active evaluation throughout the implementation and highlights how to mitigate the impact that the introduction of new workflows has on the context, participants and the intervention.

In summary, the aim of this implementation study is to investigate implementation of symptom-reporting PROMs system into the outpatient oncology setting. The objective of the intervention will be to increase detection of symptoms by clinicians using the PROMs data. The implementation objectives include the successful engagement of clinicians to use PROMs in clinical practice, the successful use of technology to obtain PROMs data from patients and present reports to clinicians, and the identification of appropriate local strategies to respond to PROM information.

Methods and analysis

Study design

This mixed-methods study will use a stepped-wedge cluster design. PROMs will be introduced sequentially into two independent clinics, and all intervention and implementation outcomes will be prospectively evaluated. The stepped-wedge approach has been chosen as it is a pragmatic solution for the systematic introduction of a complex intervention¹⁹ and has been successfully used in a number of studies related to service delivery improvements.^{20 21} Another advantage of this study design is that it limits bias by randomly assigning the clinics to the intervention in sequential order. There are key elements that require attention with this study design including the consideration of timing of study time points, cluster equivalence within the setting and intervention uptake assessed by process measures.^{22 23}

The first clinic will be observed during a current standard practice lead-in period for 4 weeks, then introduced into the Integrating Patient Reported Outcomes in a Medical Oncology Setting (iPROMOS) intervention, whereas the other clinic will continue with current standard practice and await implementation of iPROMOS. Data collection and intervention time points are presented in table 1.

Table 1.

Cluster stepped-wedge study design for iPROMOS

Time point	T1 (weeks 0–4)	T2 (weeks 4–8)	T3 (weeks 8–12)
Clinic 1	Control data	Intervention	Intervention
Clinic 2	Control data	Control data	Intervention

This protocol was co-designed with clinicians, academics and patient representatives. The iPROMOS intervention was informed by pre-implementation data collected from health professionals and relevant local stakeholders (table 2). Reporting will follow Standards for Reporting Implementation Studies.²⁴

Table 2.

Summary of pre-implementation information and how it informed implementation design

Aim	Data collected	Description of findings	Implementation strategies
To engage health professionals and patients	Physical environment mapped. Field notes. Focus groups/interviews with multidisciplinary team members and patient representatives of enablers and barriers Staff survey of knowledge modelled on Rouette’s et al 37 assessing knowledge about PROMs including facilitators and barriers, PROMs data format, enablers and barriers. Questions are scored on a Likert scale with questions such as ‘My understanding of PROs is…(very poor, poor, fair, good, very good)’, ‘My lack of understanding of PROs is a barrier to using them in clinical practice (almost never, rarely, sometimes, often, almost always)’.	The physical environment is busy but movement of patients, staff and medical records is established. There are many established treatment pathways for patient care based on disease group, stage of disease and treatment regimen. Previous interventions have been unsuccessful due to a lack of collaboration with staff and patients. Knowledge about PROMs and current evidence is different across health discipline groups.	Touchscreen computers will be positioned for easy access by patients as they enter the clinic area. PROMs reports will be made available to staff prior to patient encounter. PROMs data entry design and equipment were sourced in collaboration with consumer representatives. Information resources were developed in collaboration with staff and patient representatives, including posters, information sheets, staff brochures and inservice material.
To effectively incorporate technology	Field notes. Map of Information Technology Systems that interact with patient care, including the physical environment.	Many electronic medical records systems interact with patients and staff but not with each other. If PROMs data become a report, it can be stored as such in the patient’s medical record. Paper-based reports can be more easily integrated into patient records. Development of a system specific for each individual health service is expensive and time-consuming. It is unclear whether this would be integrated into current IT systems, or become another log on for staff, which reduces their likelihood of engagement. No ready-made system could be identified for purchase.	A simple electronic data capture system (REDCap) will be used to collect PROMs data and generate reports. A simple set-up provides the flexibility needed for integration and implementation while ensuring the fidelity of the intervention. Developing/funding a more sophisticated platform for collecting PROMs from patients can be informed by the successful implementation process.
To manage and respond to PROMs data	Focus groups/interviews and field notes to map referral and communication pathways iPARIHS context assessments of clinical areas.15	Reports can inform referrals in the format of documentation in the medical record, verbal communication or by email. The best approach needs to be identified with the relevant clinical team/area. Symptom assessment by clinicians uses CTCAE v4.0 as standard practice. CTCAE is the Common Terminology Criteria for Adverse Events, developed by the US Department of Health and Human Services which offers universal assessment and grading of symptoms of disease and treatment. Allied health and specialist nurse roles are in place for management of specific symptoms.	Alerts criteria will be generated directly to the appropriate specialist nurse and allied health team member to integrate into their practice. PROMs reports will be used to inform assessment and clinical decision-making.

iPARIHS, Promoting Action Research in Health Services; PROMs, patient-reported outcome measures.

Patient and public involvement

The process of consumer engagement through protocol development informed the research question and study protocol. Consumer representatives within the health services, and on a research advisory group, were approached to discuss the project. They confirmed a need for patient self-reporting of symptoms that are integrated into routine care. Their reports would need to be available to staff so that their concerns could be actioned. During the development of the protocol, consumer representatives were involved in the development of patient resources and collection of pre-implementation data. They also assessed the anticipated burden of the intervention on patients, and this will continue to be evaluated with consumer input through the study. This will be done through Plan, Do, Study, Act (PDSA) cycle evaluation from qualitative data collected and ongoing consumer representative input.

Results will be disseminated on information boards in the health service, and reported back to Consumer Representative Forums.

Key features of the intervention

Based on the published evidence⁵ and data from local clinicians as summarised in table 2, the PRO-Common Terminology Criteria for Adverse Events (CTCAE) was selected as the PROM to be implemented, as it was developed to extend an assessment by clinicians using the CTCAE²⁵ and has been demonstrated to provide significant benefits for patient care and outcomes.¹⁰ PRO-CTCAE is a validated (119 of 124 items met at least one construct validity criterion) symptom-reporting PROM that has been demonstrated to be reliable (test–retest was 0.7 or greater for 39 of 49 prespecified terms) and responsive (item changes corresponded to the Quality of Life Questionnaire-Core 30 scale).²⁶ There are a number of studies that have demonstrated that the PRO-CTCAE is acceptable to patients from differing cancer populations internationally.^{27 28} This PROM allows patients to report how much they experience each symptom, and the impact on their daily activities, on a five-point Likert scale (ranging from ‘none’ to ‘very much’). The core set of questions includes anorexia, constipation, dyspnoea, diarrhoea, fatigue, nausea, pain, sensory neuropathy, vomiting, cough, low mood and anxiety. Basch et al’s ⁵ study used a weekly completion schedule on an app with alerts sent to clinicians in real time. However, use of apps for patient reporting was not compatible with the health service’s patient confidentiality policy. The intervention was adapted to include PROM reporting only during scheduled attendances for outpatient clinic appointments. Thus, reporting to clinicians will occur in line with existing clinic visits, which may be weekly or less frequently depending on cancer diagnosis, stage and treatment regimen. PROMs reports will be made available for health professionals to view and respond to. This could include referring the patient to allied health or supportive care, counselling, or additional pharmacological support (eg, adjusting pain medications). PROMs will be added in paper format to the patient chart, and in keeping with local practice, and then will be scanned into the electronic medical record at a later date.

In summary, the iPROMOS intervention consists of (a) patients self-reporting symptoms (PRO-CTCAE PROM) using a touchscreen computer with data captured on a custom-built REDCap database; (b) reports of this information are generated in real time; (c) these reports are available to all healthcare team members and filed in the patients’ medical record and (d) a copy of the report is also provided to the patient. Usual care is clinician assessment of symptoms without the additional use of a PROM.

In the co-design process, using the broader research evidence, investigated to support clinician’s recommendations, a reported symptom of grade 2 or higher for nausea, vomiting or anorexia, and grade 3 for all other symptoms is considered significant.⁵ If there is an increase in symptoms greater than 2 points from the previous visit, this will also trigger a referral by established pathways to the relevant allied health professional.

Setting of the implementation

This project will be conducted in a tertiary teaching/quaternary referral hospital located in South-East Queensland, Australia. The health service for this centre is the largest in Australia, with the oncology outpatients’ department running up to 14 clinics in 1 day. Each of these clinics is oncologist specific, providing service for treatment, surveillance and follow-up for the patients in their care.

Contextual pre-implementation information has revealed key factors for successful integration of the intervention (table 2). Most importantly, the intervention needs to engage all members of the multidisciplinary team and the staff who will have access to the PROM information to address symptoms, disease management and treatment. To make this likely, the facilitator will aim to integrate the PROM collection and reporting as much as possible into the existing workflow processes already in place at the clinic. Evidence shows that workflows differ greatly between hospitals and even within clinics in a hospital, and that staff are reluctant to change anything that interrupts established practice, given the very complex environment they are managing.²⁹ They are only willing to take on a new intervention when the benefits and processes for patient care are tangible and clear. For successful implementation, it has been identified that it is necessary to integrate with existing patient care pathways and technological infrastructure, rather than impose another layer, which would likely be met with resistance.²⁹

Participants

This study will collect data from two main groups of participants: (a) patients and (b) the clinicians caring for them.

1. Patients who attend the randomised medical oncology outpatients’ clinics for treatment, medical review, active surveillance or routine follow-up, with sufficient English knowledge to read the questionnaires. Patients with significant cognitive impairment, visual difficulties or from a non-English-speaking background who might have difficulty in completing the forms will be excluded from the study.

   Patient Screening and Recruitment: patients attending selected clinics will be invited to use touchscreen computer to complete PROM information. The first page of the PROM collection form provides a patient information sheet and consent form. Potential participants will need to read the information and accept to enter PROM-reporting platform. If they do not wish to, they can choose to decline. Patient information will also be visible on a poster displayed in the clinical waiting area.

2. Staff who care for these patients including nursing and medical staff, pharmacists, dietitians, welfare workers, social workers, psychologists, speech therapists, physiotherapists and other allied health workers are eligible.

   Staff participation: an opt-out approach to consent staff has been approved by the ethics committee. Multidisciplinary staff will be contacted using various communication channels, directly by the facilitator–researcher to collect pre-implementation information, as well as through distribution of information brochures and posters developed in collaboration with the clinical teams.

Methods of evaluation

Process measures used for implementation evaluation

In accordance with the MRC Guidelines for Complex Interventions, the iterative implementation will be evaluated using both quantitative and qualitative process measures as described in table 3.

Table 3.

Process measures of implementation evaluation

Process measuring tool	Method of collection	Approach to analysis
Context: Description of factors impacting and impacted. Description of barriers and enablers.	Facilitator field notes and site journal.	Qualitative: content analysis for a structured analysis.
Feasibility: Number of patients that approached the touchscreen computer without prompting. Time taken to complete PROM by patients. Time required to assist patients to complete PROM. Number of return completions by patients. Time taken to respond to report by staff.	Counts. Data from data-capture program. Self-report by staff. Field notes.	Quantitative: descriptive statistics. Qualitative: content analysis for a structured analysis.
Fidelity: Number of missing encounters by patients. Number of missing case report forms. Reasons for missing data.	Counts. Case report form data. Field notes.	Quantitative: descriptive statistics. Qualitative: content analysis for a structured analysis.
Reach: Number of staff that answered ‘yes’ to whether they knew about the implementation. Number of staff that stated that required education about PROMs. Number of staff that independently used PROMs report. Staff groups that responded to PROMs data.	Counts. Case report form data. Field notes.	Quantitative: descriptive statistics. Qualitative: content analysis for a structured analysis.

PROM, patient-reported outcome measure.

Following the iPARIHS framework, data will be collected by the facilitator who works closely within the context. In this protocol, the facilitator will collect and use process measures, with protocol-specified data collected at prespecified time points (table 4).

Table 4.

Outcomes of the implementation

Outcome measure	Method of data collection	Approach to analysis
% Patients completing PROM form	Nominator of PROMs in electronic data capture; denominator of booking schedule of patients that attended clinic; facilitator field notes of reasons for any missing data.	Quantitative: descriptive statistical analysis; longitudinal analyses of % change. Qualitative: content analysis.
% Staff acknowledging PROM data	Case report forms; facilitator field notes.	Quantitative: descriptive statistical analysis; longitudinal analyses of % change. Qualitative: content analysis.
% PROMs in medical record	Communication in the medical record; completed PROMs in electronic data capture; referral data.	Quantitative: descriptive statistical analysis. Qualitative: content analysis.
Acceptability of PROM reporting for staff and patients	Staff survey. Focus groups, interviews and field notes.	Quantitative: descriptive statistical analysis. Qualitative: content analysis to identify themes and interpret.

PROMs, patient-reported outcome measures.

PDSA cycles will be performed every 21 days as an interim data analysis to evaluate progress, and to report these findings to clinicians so that collaborative strategies can be established that maximise implementation. The purpose of each PDSA cycle is to summarise and reflect on the implementation process and improve it for the next cycle.¹⁶

Outcomes of the implementation

The primary outcome of interest is successful implementation and has been operationalised as ‘PROM reports are made available to clinicians in 85% of encounters, 70% of clinicians will respond to PROM data, and of those 50% of responses will be noted in the patients’ medical record’. This was selected as other studies reported that clinicians and patients are satisfied at such level of service when use is identified as feasible and acceptable.^{30 31}

Secondary outcomes will measure patient and staff acceptance. Staff surveys will be distributed at the end of the PROMs data collection to capture change from baseline in staff knowledge, and identified facilitators and barriers.

Outcomes of the intervention

The primary outcome measure of the intervention will be counts of health professional notes in the patients’ chart about a symptom being of concern (eg, pain). As well as the response to such symptoms will be recorded (eg, referral to pain specialist).

Secondary outcomes will be an improvement in patient QOL, presenting as a clinically significant reduction in measured symptoms. More detailed explanation of outcome measures is provided in table 5.

Table 5.

Outcome measures of the intervention

Outcome measure	Methods of collection	Approach to analysis
Symptoms assessment by clinicians	Medical record entries, case report forms.	Comparison of proportion of patients with symptom assessment between intervention and control group using χ2 test.
Response to symptom information	Medical record entries, case report forms.	Proportion of patients referred for supportive care interventions compared between intervention and control groups using χ2 test.
Change in symptom reporting and responding from pre-intervention to during intervention	Medical record entries, case report forms, PROM electronic data capture.	Proportion of patients before to during intervention period using χ2 analysis and process control analysis.
Presentations to the emergency department	Medical record entries.	Proportion of patients before to during intervention period using χ2 analysis and process control analysis.
Hospital admissions	Medical record entries.	Proportion of patients before to during intervention period using χ2 analysis and process control analysis.

PROM, patient-reported outcome measure.

Sample size

Berry et al ³² conducted a randomised controlled trial that compared symptom reports between clinics using an electronic reporting tool. They assessed both processes and outcomes of care, comparing the impact of PROM reports between the control and intervention clinics. It was used to guide the sample size calculations because this study measured the identification of symptoms in usual care versus a symptom-PROMs intervention. To obtain an estimate of a minimal number of observations that should be included in each cluster in this study, Berry et al’s³² results were used. These researchers identified that a PROMs intervention increased symptom detection by 10%. Using these findings, and 80% power, given a baseline detection level of 0.75, 500 participant encounters would be needed to show improvement by 10% or more.

Methods of analysis

Quantitative analyses

Quantitative measures have been designed for the process measures of implementation evaluation, the outcome measures of the implementation and the outcome measures of the intervention. Descriptive statistics including counts, frequencies and proportions will be used to summarise data collected. Other statistical analyses to be used will include χ²analysis for comparing proportions, linear mixed models for longitudinal analyses and statistical control process analysis to identify trends over time.

Data from both clusters will be analysed using inverse variance weighting so that the difference can be estimated for all patient encounters. This analysis can be used to adjust for cancer types, or clustering by clinicians.³³ This analysis will provide a measure of the intracluster effect, which can then be used for power calculations in future larger studies.³⁴

Qualitative data

The facilitator site journal will be used to record observations, and will be content analysed to identify key themes, as a part of each PDSA cycle every 21 days.

The analysis of the facilitator site field notes will be used to triangulate other research findings highlighting aspects in need of further investigation. The function of field notes is to identify processes in a given situation and describe how participants contribute to, and impact, these.³⁵ Extracted data will be interpreted in keeping with Miles et al’s³⁶ approach using field notes who propose an analysis of systematic coding, word by word, presenting the data visually to identify patterns.

Data monitoring

Data monitoring will ascertain high data quality, ensure rigour and mitigate biases.

Data monitoring will be done through the following three processes:

1. Quantitative data will be double entered for a random sample of 10% records, and all records will be double entered should the error rate be greater than 5%.

2. Monthly meetings with expert facilitators who are not involved with the project to reflect on the implementation and evaluation of the project.

3. Supervision and oversight by the study team not directly involved in the process of implementation.

Safety considerations

The main purpose of the secondary outcome measures of the intervention is to measure the safety of using this implementation approach. A potential safety issue is that when patients complete the PROMs, they expect that staff will act on that information. If the implementation is not successful, staff may not do this in a timely fashion or at all, and patients who report symptoms may not receive suitable treatment. Any such issues where a PROMs report was not acted on will be noted and described using the data collection tools for the project. The facilitator will raise any issues where patient safety is at risk.

Data deposition and curation

All de-identified data will be stored on a REDCap database, on a secure university server. Patient information will be stored on their medical record and hospital-based servers that are password protected. Data will be stored for 5 years. A formal data management plan has been developed and approved by the Queensland University of Technology Research Unit.

Dissemination of results

Results will be disseminated in peer-reviewed publications and presented at national and international scientific meetings.

Discussion

This study proposes that successful implementation of PROMs requires sophisticated attention to the local clinical setting and existing clinical workflows and can overcome barriers previously experienced in other settings by following a prespecified implementation approach with an experienced facilitator. It is important to investigate implementation strategies as clinical trials have demonstrated significant benefits for patients, but also reported the difficulties of using PROMs in complex health systems outside the highly structured context of a clinical trial. Systematic reviews recommend a structured implementation approach that considers the many elements present in the health system into which PROMs are introduced. The use of the iPARIHS framework with the MRC Guidelines for Implementation of Complex Interventions, built on the work of ISOQOL, offers an implementation strategy that addresses the issues identified in the research to date. This study offers an opportunity to scientifically measure implementation, potentially rapidly implement PROMs into clinical practice and to inform future research and clinical practice.