Table 2.
Results from round 1 (importance) and round 2 (ranking) across both Delphi rounds by domain of Knowledge to Action framework
| Round 1—importance | Round 2—ranking | ||||
|---|---|---|---|---|---|
| Consensus statements | Not important | Important | Critical | Median rank+ | Interquartile range (Q1–Q3) |
| (%) | (%) | (%) | |||
| Section 1: Readiness to implement PROs | |||||
| 1.1 Conduct a formal assessment of the willingness of the organisation to implement PROs | 0 | 20.9 | 77.6 | 5.0 | (3.0–11.0) |
| 1.2 Understand current beliefs and attitudes of staff related to PRO use | 0 | 22.4 | 11.6 | 6.0 | (3.0–10.0) |
| 1.3 Identify current staff capabilities, skills and service requirements to implement routine PRO collection | 0 | 10.5 | 89.6 | 4.5 | (2.0–7.0) |
| 1.4 Assess existing resources and infrastructure that can be deployed to aid the collection of PROs | 0 | 9 | 91 | 5.0 | (3.0–7.3) |
| 1.5 Conduct an assessment of patient and clinician willingness to participate in PRO collection | 3 | 25.4 | 71.7 | 6.0 | (3.0–10.0) |
| 1.6 Determine which populations of patients should be invited to participate in PRO collection* | 4.5 | 26.9 | 68.7 | 10.0 | (7.0–11.3) |
| 1.7 Ensure equity, diversity and inclusion when determining which patients will be invited to participate in PRO collection | 1.5 | 22.4 | 76.1 | 9.0 | (7.0–11.0) |
| 1.8 Identify potential local barriers and facilitators that will help or hinder PRO implementation | 0 | 10.5 | 89.6 | 5.0 | (3.8–7.3) |
| 1.9 Form strong partnerships with stakeholders who will be involved in PRO collection and use (e.g. consumers, clinicians, IT/infrastructure developers, health service administrators) | 0 | 7.5 | 92.5 | 5.0 | (2.0–7.3) |
| 1.10 Consult with relevant stakeholders that will be impacted by the routine implementation of PROs (i.e., patients, family members, clinicians, administration, IT, service administrators) to discuss strategies for implementation | 0 | 17.9 | 82.1 | 6.0 | (2.8–8.0) |
| 1.11 Establish implementation teams that incorporate all stakeholders (mentioned above) to guide the implementation of PROs | 0 | 13.4 | 86.6 | 6.5 | (3.0–7.3) |
| 1.12 Assess, with stakeholder consultation, when PROs should be collected | 0 | 11.9 | 88.1 | 11.0 | (8.0–12.0) |
| 1.13 Consider if additional resourcing streams are necessary to enable the implementation of PRO collection | 0 | 10.5 | 88.1 | 10.0 | (7.8–13.0) |
| Section 2: Addressing barriers | |||||
| 2.1 Clearly articulate the evidence for and value of using PROs to all stakeholders | 1.5 | 9.1 | 89.4 | 3.0 | (1.0–6.0) |
| 2.2 Select which existing validated tools (surveys) should be used for the collection of PROs | 0 | 18.2 | 81.8 | 3.5 | (2.0–7.3) |
| 2.3 Decide on the appropriate format for PRO collection (i.e. paper based or electronic) | 0 | 27.3 | 72.7 | 4.0 | (3.0–8.3) |
| 2.4 Establish how PRO data will be stored and who will have access to the data | 0 | 19.7 | 80.3 | 7.0 | (4.0–11.0) |
| 2.5 Map workflows and processes needed for PRO implementation | 0 | 10.6 | 89.4 | 2.5 | (1.0–4.3) |
| 2.6 Develop systems that will allow the PRO information to be used for quality improvement | 0 | 16.7 | 83.3 | 8.0 | (5.0–11.3) |
| 2.7 Develop systems that will allow the PRO information to be used for performance measurement* | 4.6 | 31.8 | 62.1 | 12.0 | (6.8–15.0) |
| 2.8 Develop systems that will allow the PRO information to be used for research* | 1.5 | 28.8 | 69.7 | 11.0 | (8.0–13.3) |
| 2.9 Provide training for clerical staff on the role of PROs and local administrative processes associated with their collection | 3 | 18.2 | 78.8 | 9.0 | (6.0–12.0) |
| 2.10 Run dedicated training sessions for clinicians on how to utilise and interpret PRO reports | 0 | 22.7 | 77.3 | 8.0 | (6.0–10.0) |
| 2.11 Run education programs to help patients and families who are first entering the cancer system understanding why PROs are being collected and how to use the PRO platform/system (as appropriate)* | 6.1 | 31.8 | 62.1 | 10.5 | (7.0–13.0) |
| 2.12 Ensure patient-centred education programs (item above) will include why PROs are collected and how PROs can improve their health outcomes* | 6.1 | 25.8 | 68.2 | 11.0 | (7.5–13.0) |
| 2.13 Develop guidelines to help clinicians respond to patient concerns and issues identified in collected PRO data | 0 | 10.6 | 81.8 | 7.0 | (4.8–10.0) |
| 2.14 Supply patients and families with guidance about actions to take regarding PRO results | 3 | 18.2 | 78.8 | 10.0 | (7.0–13.0) |
| 2.15 Develop and adopt clinician-facing and patient friendly reports that summarise PRO scores for ease of interpretation and actionability | 1.5 | 13.7 | 84.9 | 8.0 | (3.8–10.0) |
| Section 3: Developing implementation strategies | |||||
| 3.1 Decide on an implementation approach using an implementation framework* | 1.5 | 27.3 | 69.7 | 4.0 | (2.0–6.3) |
| 3.2 Identify and support champions within the organisation who can drive the implementation of PROs | 0 | 13.6 | 86.4 | 2.0 | (2.0–4.0) |
| 3.3 Seek feedback from patients regarding the process of PRO collection and use of data in clinical care, and use this feedback to guide further initiatives | 0 | 25.8 | 74.2 | 6.0 | (4.8–7.0) |
| 3.4 Integrate PROs into clinical workflow, care pathways, and the way team members work together so that the use of PROs becomes routine practice | 0 | 10.6 | 89.4 | 2.0 | (1.0–3.3) |
| 3.5 Establish a centralised system to deliver technical assistance focused on the implementation of PROs | 1.5 | 21.2 | 75.8 | 6.0 | (4.0–7.0) |
| 3.6 Pilot the collection of PROs in a smaller controlled group of patients before service wide rollout | 6.1 | 15.2 | 77.3 | 4.0 | (2.8–5.0) |
| 3.7 Develop reminder systems to help prompt clinicians to collect PROs* | 7.6 | 22.7 | 68.2 | 7.0 | (5.0–8.0) |
| 3.8 Identify and learn from other sites where the collection of PROs has been successful | 0 | 13.6 | 86.4 | 4.0 | (3.0–6.0) |
| Section 4: Monitoring use and evaluating outcomes | |||||
| 4.1 Collect and summarise clinical performance data and report performance back to clinicians and administrators | 1.5 | 21.2 | 77.3 | 3.5 | (2.8–5.0) |
| 4.2 Monitor how many clinicians/clinics are collecting and using PROs | 1.5 | 27.3 | 71.2 | 4.0 | (2.0–5.0) |
| 4.3 Evaluate the reasons for non-use of PROs in clinical care | 0 | 16.7 | 83.3 | 4.0 | (3.0–6.0) |
| 4.4 Record any changes to clinical practice once PROs are implemented | 0 | 7.6 | 92.4 | 3.5 | (2.0–5.0) |
| 4.5 Ensure ongoing communication with administrative, clinical, Information Technology staff, and patients to understand their evolving needs in collecting PROs | 0 | 7.6 | 92.4 | 4.0 | (2.0–5.0) |
| 4.6 Monitor patient and clinician engagement with PROs once they are implemented for quality assurance and intervention refinement | 1.5 | 15.2 | 83.3 | 2.0 | (1.0–3.0) |
| Section 5: Sustainability | |||||
| 5.1 Conduct an assessment of what is necessary for long-term continuation of routine PRO collection | 0 | 10.6 | 89.4 | 3.0 | (1.0–5.0) |
| 5.2 If appropriate, use collected PROs to guide and improve clinical care at the health service level | 1.5 | 6.1 | 92.4 | 2.0 | (1.0–4.0) |
| 5.3 Continue regular training of health service staff in PRO collection and response | 0 | 22.7 | 77.3 | 3.0 | (2.8–5.0) |
| 5.4 Identify dedicated resources that are needed to keep PRO collection ongoing | 0 | 12.1 | 87.9 | 2.0 | (2.0–4.0) |
| 5.5 Develop a protocol for PRO collection that is regularly evaluated and refined when necessary | 0 | 21.2 | 78.8 | 3.0 | (3.0–4.0) |
*Items that did not meet 70% agreement in round 1, + lower scores= higher levels of importance. Grading of Recommendations Assessment, Development and Evaluation (GRADE) scales16 were used for round 1 results, scaling from ‘1’ to ‘9’. Scores of 7–9 indicated the item was of ‘critical importance’, scores of 4–6 indicated ‘importance’, and scores of 1–3 indicated ‘limited importance’. Abbreviations: PRO, patient reported outcome; IT, information technology