Leveraging co-production in the systematic review of evidence to update the pan-Canadian Oncology Symptom Triage and Remote Support (COSTaRS) practice guides

Abstract

We used co-production: I) to conduct a systematic review to update the evidence on assessment, triage, and management in 17 pan-Canadian Oncology Symptom Triage and Remote Support (COSTaRS) practice guides; and II) to evaluate the co-production approach used by our team to contribute evidence on the impact of engaging knowledge users (patients, caregivers, nurses, managers, policy-makers).

Of 17,618 citations, 175 were eligible, including 124 clinical practice guidelines and 51 systematic reviews with meta-analysis since the last update published in 2020. Findings were used to update the 17 COSTaRS practice guides and add one new COSTaRS practice guide for difficulty swallowing. Evidence was removed from 32 outdated sources. The COSTaRS practice guides required a few updates in the assessment, triage, medications, and self-care domains to reflect the new identified evidence.

A baseline survey assessed our team members’ previous experience with the COSTaRS practice guides, previous experience with conducting systematic reviews, and their preferred level of involvement for each step of the systematic review. The baseline survey yielded a response rate of 14/15 team members (93.3%). The follow-up survey including the Patient Engagement in Research Scale (PEIRS-22), completed by 12 team members (2 patient/caregivers, 5 researchers and 5 nurses), showed an extremely high level of meaningful engagement on all sub scores and the total PEIRS-22 score.

Narrative feedback from participants indicated that they felt valued as equal partners and engaged through regular project updates. They enjoyed being able to select individual levels of involvement. The inclusion of multiple perspectives enhanced the richness of the final product, according to participants. While their involvement was influenced by workload time constraints, a strong sense of ownership of the final product was identified as a positive impact of the co-production process.

INTRODUCTION

Adults with cancer experience many treatment-related symptoms. Prompt and accurate cancer symptom management is an important safety issue given that symptoms can become life-threatening and often occur when patients are at home. For patients with cancer, the telephone is the quickest and the easiest route to access guidance in self-care and triaging symptoms to the appropriate level of care (Cancer Care Ontario, 2019; Stacey et al., 2007; Wilson & Hubert, 2002). Nurse-delivered telephone services, including virtual triage, are becoming more prevalent in the provision of ambulatory oncology care and have been positively evaluated (Piazza & Drury, 2023). Telehealth is the remote delivery of health care through a variety of telecommunications technologies and services aimed at improving patient care and healthcare delivery (Dorsey & Topol, 2016). Telephone-based support can improve patients’ ability to navigate the complex health care system, provides them with reassurance related to symptom self-management (Piazza & Drury, 2023) and helps keep them out of the emergency department when they could have been managed at home (Elkaddoum et al., 2020; Piazza & Drury, 2023; Vandyk et al., 2012).

The symptoms identified by adult cancer patients and effectiveness of telephone-delivered support in the community has been explored previously. Ream et al. (2020) undertook a systematic review to identify symptom priorities and the effectiveness of telephone-delivered interventions for reducing symptoms associated with cancer and its treatment. Despite challenges with respect to patient heterogeneity, small sample sizes, and poor-quality studies, the authors determined that telephone interventions provided a convenient way to support patient self-care of cancer-related symptoms in adults with cancer. Some interventions provided by telephone particularly related to depression, anxiety, fatigue (Ream et al., 2020; Sezgin & Bektas, 2024), and nausea and vomiting (Sezgin & Bektas, 2024) have been described as effective. Telephone support has been identified as an effective decision support system in the reduction of pain in patients with cancer (Sezgin & Bektas, 2022, 2024).

Nurses typically provide telehealth symptom management, including telephone services; a service that has become more critical due to overcrowded emergency rooms and inpatient units (Rowe et al., 2020). Since the COVID-19 pandemic, many patients fear risking infection when entering healthcare facilities (Al-Shamsi et al., 2020; Karacin et al., 2020). A recent systematic review of the effectiveness of nurse-led telehealth services showed reduced symptom severity without increases in hospitalizations, emergency department visits, or unscheduled clinic visits (Kwok et al., 2022). When providing telehealth services, nurses are expected to use evidence-informed decision support tools, apply critical thinking, and use clinical judgement (College of Nurses of Ontario, 2020). Despite widespread availability of clinical practice guidelines for symptom management, many are not formatted for use in clinical practice.

In 2007, the pan-Canadian Oncology Symptom Triage and Remote Support (COSTaRS) team was established, including researchers and knowledge users (KUs) from eight provinces to improve the quality and consistency of evidence-informed cancer symptom management (Stacey et al., 2012). The COSTaRS practice guides are evidence-informed decision support tools developed to guide nurses providing symptom management, on the telephone, to adults receiving cancer treatments. The 17 practice guides were developed using the CAN-IMPLEMENT methodology that involved conducting a systematic review to identify clinical practice guidelines, appraising guideline quality, establishing a user-friendly template, validating the guidelines with nurses across Canada, using plain language to facilitate use with patients, and formatted based on how nurses think and what nurses do (Stacey et al., 2013; Stacey & Carley, 2017; Stacey & Pan-Canadian Oncology Symptom Triage, 2012). Nurses using the COSTaRS practice guides a) assess symptom severity; b) triage patients for symptom management based on overall severity; c) review medications being used for the symptom; d) discuss self-care strategies; and e) document the plan agreed upon with the patient. Previous literature searches to create and update COSTaRS practice guides occurred in 2012, 2015, and 2018.

Evaluation of COSTaRS practice guides has consistently shown positive outcomes. In a descriptive study in California exploring nurse-initiated outreach symptom management, the COSTaRS guides were found to be trustworthy, evidence-based resources that identified common treatment-related symptoms early and involved patients in decision making (Davis et al., 2020). Analysis of nurse-patient calls in Canada revealed that nurses provided higher quality symptom management and worked more fully within their scope of practice when COSTaRS were used (Stacey et al., 2021). A randomized controlled trial in Turkey found patient care guided by COSTaRS showed decreased symptom severity (Chemo Symptom Assessment Scale); improved quality of life (FACT-G) and increased self-care management compared to usual care (Cetin et al., 2022). The COSTaRS guides are in various stages of implementation to guide nursing practice in Canada, the United States, China, Turkey, Australia, and Brazil. For example, when COSTaRS guides were integrated into the after-hours oncology telehealth program in Ontario, there was a 33% reduction in ER visits and increase in patient satisfaction (Martelli et al., 2023).

Co-production involves including community partners in the development of health improvement initiatives (Grindell et al., 2022), while few studies have been able to fully capture the extent or meaningfulness of partners’ perspectives (Grindell et al., 2022; Hopkins et al., 2024; Lewis et al., 2024; Norton, 2021). Measuring the benefits of co-production in research studies has also been identified as a challenge (Nordin et al., 2023). Three perspectives are important to consider when trying to measure the benefits of co-production: the processes, the experience of research partners, and the outcomes (Nordin et al., 2023). The goal of including diverse perspectives is to develop a more meaningful product (Lewis et al., 2024).

Using a co-production approach, we aimed to: I) update the evidence on assessment, triage, and management in 17 COSTaRS practice guides; and II) evaluate the co-production approach used by our team to contribute evidence on the impact of engaging KUs (patients, caregivers, nurses, managers, policy-makers) throughout the research project to determine the impact on cancer researchers, KUs, and the research itself.

METHODS

Study design

A systematic review was co-produced by a team of researchers and KUs. The executive committee (DS, GM, CK, JT), including researchers, oncology nurses, a nurse practitioner, and a policy maker, met every two weeks throughout the project. Our full research team included researchers and KUs of varying levels of influence in the healthcare system (nurses, managers, patients, caregivers, policymakers) and collaborators (e.g., Canadian Association of Nurses in Oncology (CANO), de Souza Institute, Cancer Care Ontario). We followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guidelines (Page et al., 2021). To facilitate a meaningful experience for our research team members, team members were asked at baseline their preferred level of involvement in the steps of the systematic review (e.g., screening, data extraction, updated guideline review, quality appraisal) and were invited to participate in each step based on their availability. Ethics approval for evaluating co-production was obtained from the Ottawa Health Science Network Research Ethics Board (OHSN-REB #20230015-01H).

Eligible sources of evidence

According to PICAR, we included synthesized evidence in the form of clinical practice guidelines and systematic reviews focused on adults (Population); cancer symptom management for each of the 17 cancer symptoms (Interventions); with any comparator (Comparators); evidence-based development with systematic rating of evidence and published since 2018 (Attributes); and reports’ level of confidence with recommendations (Recommendations) (Johnston et al., 2019). Eligible systematic reviews included meta-analyses including reported certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework (Oxman, 2004).

Information sources and search strategy

The librarian scientist (ARW) reviewed the literature search strategy from the 2020 update and revised it based on team members’ feedback. Searches were run in MEDLINE, CINAHL, Embase, and PsycINFO between May and June 2023. Using key terms developed for the electronic searches, grey literature searches were conducted on websites known or suspected to have practice guidelines related to cancer care and known guideline clearinghouse websites (e.g., Oncology Nursing Society, American Society of Clinical Oncology, Cancer Care Ontario, British Columbia Cancer Agency).

To screen for eligible citations, findings from electronic databases and hand searching were merged with duplicates removed in Covidence. Using Covidence, two reviewers independently screened titles and abstracts for inclusion. We retrieved full texts for any citations identified as potentially relevant and screened them using independent pairs of reviewers. Any disagreements were resolved in discussion between reviewers or with the executive team. The GRIPP2 (Guidance for Reporting Involvement of Patients and the Public) reporting checklists were used to report the patient and public involvement (Staniszewska et al., 2017).

Findings were synthesized by symptom (e.g., diarrhea, constipation, breathlessness) using a standardized data extraction form and incorporated into the COSTaRS 2020 version using track changes. Each updated symptom practice guide was sent for peer review to the full research team members, who were asked to share it within their oncology program and provide feedback. One of the patient caregivers on the team (previous elementary school teacher) reviewed all practice guides to ensure use of plain language.

Evaluation of co-production approach

All team members (excluding the research coordinator and collaborators) were invited to participate. Feedback was elicited mid project and at the end of the study with respect to the level of perceived engagement in the systematic review of the guidelines and authenticity, impact, and challenges of the co-production process. The baseline survey asked team members about their previous experience with the COSTaRS practice guides, previous experience with conducting systematic reviews, their preferred level of involvement for each step of the systematic review, and demographic questions. Follow-up surveys asked team members questions about their satisfaction with their involvement in the study, the Patient Engagement in Research Scale (PEIRS-22), and open questions to explore their experience with the co-production approach (Hamilton et al., 2018). Following Dillman methods, we sent up to three email reminders to survey non-responders (Dillman et al., 2024).

The PEIRS-22 was previously shown to have structural validity, construct validity, and good test-retest reliability with scores showing significant differences across three levels of meaningful patient engagement in research (Hamilton et al., 2018). This tool has been found to have good psychometric properties, making it a good proxy for assessing the degree of meaningful engagement of patients and caregivers in research (Hamilton et al., 2021). Cut-points for engagement are established for each of the PEIRS-22 subdomains and the total score. The PIERS-22 engagement scores above 92% are interpreted as ‘extremely meaningful’; scores between 82.7 to <92% are ‘very meaningful’, 70.0 to <82.7% are ‘moderately meaningful ‘and <70% are deemed ‘low’ for meaningfulness. Quantitative findings were analyzed descriptively. Inductive content analysis was used for narrative feedback to open-ended questions by participant group, including patient/caregivers, nurses and researchers (Elo & Kyngäs, 2008).

RESULTS

Evidence update

Of 17,618 citations yielded, 75 were eligible clinical practice guidelines and systematic reviews with meta-analysis (see Figure 1). Hand search of the grey literature yielded an additional 100 clinical practice guidelines. In total, 175 articles were used to update the 17 COSTaRS practice guides and create one new practice guide for ‘swallowing difficulty’. COVID-19 was only mentioned in one evidence source for fever with neutropenia that recommended COVID-19 vaccinations.

Figure 1.

PRISMA Diagram

Compared to the 2020 version of COSTaRS, the authors added evidence from 124 new or updated clinical practice guidelines and 51 systematic reviews. For the new practice guide, we sought permission of the clinical practice guideline developer. Evidence from 54 sources were updated and 32 outdated sources were removed.

A summary of changes to the COSTaRS guidelines since 2020 is outlined in Table 1. Overall, there were few updates required. Changes related primarily to assessment, medications, and self-care domains. The assessment criteria for immune checkpoint inhibitor therapy – organ specific (e.g. thyroid) or system specific (e.g. endocrine) – adverse effects and toxicities were updated and standardized across all guidelines. Other examples of updates include the broadening of the assessment history for anxiety and depression to include consideration of past traumas, abuse, cognitive impairment or communication difficulties. New medications were added (e.g., trazodone for sleep changes) and some were removed (e.g., calendula ointment for skin reaction prophylaxis). The level of evidence supporting specific medications was upgraded (e.g., serotonin reuptake inhibitors for anxiety) or downgraded (e.g., corticosteroids for appetite loss) according to newer literature. The strength of the evidence related to the effectiveness of complementary self-care strategies, such as acupuncture, yoga, meditation, and cognitive behavioral therapy, has been evolving and practice guides were updated accordingly to reflect this.

Table 1.

Summary of Changes in COSTaRS from 2020 to 2024

All practice guides
Assessment for ICI signs of a medical-related condition	signs of hyperthyroidism: ADDED fatigue/weakness, swollen base of neck
	signs of hypothyroidism: ADDED dry skin; REMOVED headaches
	signs of endocrine toxicity: ADDED weakness, abdominal pain, vomiting, weight gain or loss, dizziness, mood or behaviour changes, decreased libido, feeling cold
	• CHANGED eyes sensitive to light to vision changes
	• REMOVED fever
	signs of GI toxicity: abdominal pain, diarrhea, blood or mucus in stool, fever, nausea, vomiting, weight loss
	signs of hepatic toxicity: ADDED “right side” to abdominal pain
	signs of hematological adverse effects: ADDED less urination, abdominal pain, vomiting, irritability, confusion, seizures, blood pressure changes, swelling of face, hands, feet, or entire body; REMOVED yellow skin/eyes
	signs of renal adverse effects: decreased urine output, blood in urine, swelling of hands or legs, face, abdomen, sudden weight gain, abdominal or pelvic pain, nausea, vomiting, high blood pressure, drowsiness
	signs of pneumonitis: ADDED bluish coloured nail beds
	signs of cardiovascular toxicity: ADDED chest pain
	signs of myositis: ADDED muscle pain
	signs of aseptic meningitis: headache, eyes sensitive to light, neck stiffness, low-grade fever, nausea, vomiting
Anxiety
Assessment	“Do any of these apply to you?” (risk factors for anxiety):
	• ADDED past trauma/abuse, cognitive impairment, difficulty communicating, other health issues
	• REMOVED not exercising, age limit (<30) for younger age
Medications	• evidence: SSRIs/SNRIs upgraded from Expert Opinion to Likely Effective
	• ADDED SNRIs, venlafaxine (Effexor XR®), duloxetine (Cymbalta®)
	• REMOVED fluvoxamine (Luvox®)
	• footnote: No guidance on use of cannabinoids due to lack of studies and potential negative effects on mood.
Self-care	ADDED progressive muscle relaxation, acupuncture/acupressure, other creative therapies, couple counselling
Appetite Loss
Assessment	“Do you have any other symptoms?” ADDED dry mouth
Medications	corticosteroids downgraded from Effective to Likely effective
	megestrol downgraded from Effective to Benefits balanced with harm
	footnote: ADDED prokinetics have the potential for serious side effects; metoclopramide on the central nervous system and domperidone on cardiac rhythm.
Bleeding
Assessment	ADDED Do you have (signs of hepatic adverse effects) – see ICI above
	ADDED Do you have (signs of renal adverse effects): see ICI above
Breathlessness/Dyspnea
Medications	ADDED benzodiazepines if anxiety related – lorazepam (Ativan®), diazepam (Valium®), alprazolam (Xanax®) – level of evidence Expert Opinion
Self-care	ADDED: meditation, music therapy, acupressure/acupuncture
	REMOVED ♥ drinking fluids, 6–8 glasses per day
Constipation
Assessment	ADDED: Tell me what number from 0 to 10 best describes your constipation (0 = “no constipation”; 10 = “worst possible constipation”)
Medications	ADDED naloxegol for opioid as cause – level of evidence Expert Opinion
	footnote: ADDED naloxegol and methylnaltrexone are contraindicated in bowel obstruction
Self-care	ADDED consider trying acupuncture.
Depression
Assessment	“Do any of these apply to you?” (risk factors for depression):
	• ADDED past trauma, cognitive impairment, difficulty communicating, other health issues unrelated to cancer
	• REMOVED the specific age limit (<30) for younger age
	CHANGED hyperthyroidism to hypothyroidism
Medications	footnote: ADDED No guidance for the use of cannabinoids due to lack of studies and potential negative effects on mood.
Self-care	• ADDED meditation/breathing techniques, listening to music, progressive muscle relaxation, massage therapy with or without aromatherapy, acupuncture/acupressure
Diarrhea
Assessment	Do you wake in the night to have bowel movements?
	ADDED vomiting
Medications	loperamide (Imodium®) upgraded from Expert Opinion to Likely effective
	corticosteroids/prednisone, infliximab, vedolizumab, or budesonide upgraded from Expert Opinion to Likely effective
	Probiotics for radiation-induced / for chemo-induced. Evidence: Effective / Likely effective
Fatigue/Tiredness
Assessment	ADDED neurologic (condition that may cause fatigue)
	ADDED appetite loss, increase in bleeding/bruising
Self-care	ADDED acupressure or acupuncture
Fever with Neutropenia
Assessment	Do you have an oral temperature of ≥38.0°C: ADDED adjust if measured by other methods (e.g., ear, forehead)
	Do you have any other symptoms? ADDED urinary symptoms (burning, urgency, frequency)
	ADDED Do you have (signs of GI toxicity):
	ADDED Do you have (signs of pneumonitis)
	ADDED Do you have (signs of hepatic toxicity)
	ADDED Do you have (signs of aseptic meningitis)
Triage	ADDED alert if on immunotherapy
Medications	footnote: REMOVED 2G-CSF is generally recommended for patients with >20% risk of developing febrile neutropenia; prophylactic antibiotic use should be limited to high risk patients with an expected duration of neutropenia for >7 days as it may promote antibiotic resistance; antifungal prophylaxis should be reserved for a targeted group of high-risk patients with an expected duration of neutropenia for >7 days; antiviral prophylaxis is recommended for select patients at risk for certain viral infections or reactivation of viral infection.
Self-care	ADDED COVID-19 vaccinations
Mouth Dryness/Xerostomia
Assessment	ADDED Do you have (signs of sicca syndrome): abrupt onset of dry mouth, thick saliva, saliva pooling on the floor of your mouth, dry eyes
	ADDED Are you able to eat?
	Do you have (signs of uveitis): ADDED new floaters, eyes sensitive to light, eyelid swelling, change in colour vision
Medications	footnote: ADDED older adults may be more sensitive to the side effects of pilocarpine
Mouth Sores/Stomatitis
Assessment	ADDED Do you have (signs of skin toxicity): sores/ulcers/blisters in your mouth, redness or white patchy areas in your mouth, irritated gums and/or throat
	ADDED Are you able to eat
Triage	ADDED alert if on immunotherapy
Medications	ADDED dexamethasone mouthwash. Evidence: Likely effective
	REMOVED topical steroids for mouth sores from targeted therapies
	footnote: ADDED advise not to swallow morphine mouthwash or lidocaine due to systemic side effects including fatal arrhythmia
Self-care	ADDED xylitol-containing lozenges, gum, or popsicles, consider using low level laser therapy
Nausea & Vomiting
Assessment	Do you have (signs of aseptic meningitis): ADDED low-grade fever
Medications	For Lorazepam (Ativan®), haloperidol (Haldol®): ADDED anticipatory
Pain
Assessment	Do you feel confused…: ADDED nauseous
	Do you have (signs of ocular toxicity): ADDED dry eyes, eye redness, new floaters
	• CHANGED vision changes to blurred/double vision and change in colour vision
Self-care	ADDED acupressure, Yoga, meditation, cognitive behavioural therapy (CBT), biofeedback, progressive muscle relaxation, participate in patient and family counselling
Peripheral Neuropathy
Assessment	Do you have: ADDED facial weakness, drooping eyelid(s)
	CHANGED vision changes to double vision
Self-care	ADDED avoid going barefoot, acupressure
	Lower the temperature of your hot water heater: CHANGED <120ºF/49ºC to <110ºF/43ºC
Skin Rash
Assessment	ADDED graphic to illustrate a frame of reference for body surface area (BSA) percentages
	REMOVED “Is the rash moist or weeping?”
Triage	CHANGED for MODERATE advise to notify in 12–24 hours to 1–2 weeks
Medications	footnote: ADDED Higher potency topical steroids are preferred for short-term use (days to a few weeks) for immune-related dermatitis, compared to longer term use (several weeks to months) of lower potency steroids.
Self-care	ADDED Have you seen or spoken to a dermatologist?
	ADDED: over the counter acne medications, tight fitting clothes or irritating fabrics like wool
Skin Reaction
Medications	REMOVED Prevention: calendula ointment
	Footnotes:
	“Insufficient evidence to support or refute other agents for skin reaction”
	ADDED oral antihistamines, emu oil, oral curcumin, specialty non-steroidal creams (e.g., Cavilon™)
	REMOVED ascorbic acid, almond ointment, polymer adhesive skin sealant
	REMOVED Emerging evidence for proteolytic enzymes for treatment of skin reaction from radiation.
	Under “not recommended for radiation skin reaction”: ADDED calendula ointment
Self-care	ADDED “Avoid talcum powder, baby powder, and cornstarch especially on treatment areas. These products promote fungus growth and infections.”
	ADDED “Eat a well-balanced diet that includes fruit, vegetables, whole grains, and lean protein and drink fluids, 6–8 glasses per day.”
	CHANGED “Are you using non-metallic deodorant?” To “Use deodorant on intact skin only. Stop using if skin becomes irritated, blisters, or peels.”
	REMOVED “Are you avoiding skin creams or gels in the treatment area before treatment?”
Sleep Changes
Assessment	Do you have other symptoms: ADDED itchy skin, breathlessness
Medications	ADDED trazadone (Desyrel®). Evidence: Expert opinion
	ADDED antihistamines: diphenhydramine (Benadryl®), hydroxyzine (Atarax®). Evidence: Expert opinion
Self-care	ADDED muscle relaxation, aromatherapy; if you are hungry a protein snack is best; try acupressure or acupuncture.

Note. ICI = immune checkpoint inhibitor; GI = gastrointestinal; SSRIs = selective serotonin reuptake inhibitors; SNRIs = serotonin and norepinephrine reuptake inhibitors.

Survey

Of 15 team members, 14 completed the baseline survey (response rate 93.3%). Our team included members with disabilities, those from the LGBTQ2S community, and those who are members of visible minorities (Table 2). All respondents had advanced degrees (master’s or doctoral) and 50% of respondents described themselves as nurses with experience using systematic review summaries. Six provinces were represented with most respondents coming from Ontario (57%). Of 14 team members who completed baseline survey, 12 completed the second survey (Table 3). On average, respondents described extremely high levels of meaningful engagement on all sub-scores and the total PEIRS-22 score (range 91.9 to 98.9%).

Table 2.

Participant Characteristics at Baseline (n = 14)

Characteristic	n (%)
Decade of birth
1950s	1 (7.1%)
1960s	6 (42.9%)
1970s	4 (28.6%)
1980s	3 (21.4%)
1990s	0 (0.0%)
Sex
Female	13 (92.9%)
Male	1 (7.1%)
Gender*
Woman	13 (92.9%)
Man	1 (7.1%)
Other	0 (0.0%)
Current level of education
Master’s degree	8 (57.1%)
PhD degree	6 (42.9%)
Nursing certification*
Oncology Nursing	5 (35.7%)
Palliative Care Nursing	1 (7.1%)
Primary language(s)*
English	14 (100.0%)
French	1 (7.1%)
Province
Ontario	8 (57.1%)
Nova Scotia	2 (14.3%)
Alberta	1 (7.1%)
British Columbia	1 (7.1%)
New Brunswick	1 (7.1%)
Prince Edward Island	1 (7.1%)
Ethnicity*
White	13 (92.9%)
East Asian	1 (7.1%)
Lived experience as a member of the following groups*
Disabled or caregiver of a person with a disability	2 (14.3%)
LGBTQ+	1 (7.1%)
None of the above	8 (57.1%)
Prefer not to say	3 (21.4%)
Employment status
Employed full time	12 (85.7%)
Retired	2 (14.3%)
Role that best reflects primary expertise on this research project (select one)
Nurse	7 (50.0%)
Researcher	5 (35.7%)
Patient/Caregiver	2 (14.3%)
Previous experience with the COSTaRS Practice Guides*
• Member of COSTaRS Practice Guides Research Group	9 (64.3%)
• A nurse used the COSTaRS Practice Guides to support me or the person I care for while experiencing cancer treatment-related symptoms	3 (21.4%)
• I have used the COSTaRS Practice Guides to support myself or someone experiencing cancer treatment-related symptoms	6 (42.9%)
• I participated in training on how to use the COSTaRS Practice Guides	5 (35.7%)
• I developed or participated in previous updates of the COSTaRS Practice Guides	8 (57.1%)
• I participated in a research study about the COSTaRS Practice Guides (e.g., responded to surveys)	2 (14.3%)
• I developed and/or promoted health policy that supports the COSTaRS Practice Guides	4 (28.6%)
Previous experience with systematic reviews*
• No previous experience	2 (14.3%)
• Read or reviewed abstracts/consumer summaries of a systematic review(s)	7 (50.0%)
• Reviewed search strategies to be used in electronic databases	7 (50.0%)
• Screened titles and abstracts of citations	6 (42.9%)
• Screened full text of citations	6 (42.9%)
• Searched grey literature sources	4 (28.6%)
• Extracted data into data collection forms	5 (35.7%)
• Assessed the quality of the data sources	4 (28.6%)
• Conducted descriptive analyses of findings from eligible studies	4 (28.6%)
• Drafted a systematic review article(s)	5 (35.7%)
• Provided feedback on a systematic review article(s)	9 (64.3%)
• Co-authored a systematic review article(s)	7 (50.0%)
• Contributed to a plain language summary for a systematic review(s)	6 (42.9%)
• Shared findings with those who can use them	9 (64.3%)
• Peer-reviewed a systematic review article(s) for a journal	4 (28.6%)

Note.

^*Multiple responses accepted.

LGBTQ+ = lesbian, gay, bisexual, transgender, queer/questioning, plus other; COSTaRS = pan-Canadian oncology symptom triage and remote support.

Table 3.

Patient Engagement in Research Scale PEIRS-22 (n = 12)

Sub-Domains & Total Score	Mean Score (%)
	Patient/Caregiver (n = 2)	Researcher (n = 5)	Nurse (n = 5)
Procedural Requirements (/28)	28 (100)	28 (100)	26.2 (94)
Convenience (/12)	12 (100)	11.6 (97)	11.2 (93)
Contributions (/12)	11.5 (96)	11.6 (97)	11.4 (95)
Team Environment & Interaction (/8)	8 (100)	8 (100)	7 (88)
Support (/8)	8 (100)	8 (100)	7.4 (93)
Feel Valued (/8)	8 (100)	8 (100)	7.2 (90)
Benefits (/12)	11.5 (96)	11.8 (98)	10.8 (90)
Total PEIRS-22 Score (/88)	87 (98.9)	87 (98.9)	81.2 (91.9)

Note. PEIRS = patient engagement in research scale.

Content analysis

Content analysis of narrative feedback showed that all groups felt appropriately engaged throughout the project. Individuals understood the complexity of the project, enjoyed being able to contribute, and appreciated regular project updates. One researcher commented: “I have never been involved in research where such a considered and well-actioned approach to engagement and involvement was taken” (Researcher 2). Participants described enjoying being able to select individual levels of involvement over the course of the project. They described feeling respected and valued as “equal partners” in the co-production process. Transparency was apparent when feedback was incorporated from multiple sources. One patient-caregiver commented: “Team leaders establish an environment of co-production by valuing all team members, offering options for how they wish to be engaged in the co-production process, and communicating at all stages of the project” (Patient-Caregiver 1).

In terms of the benefits or impact of co-production, participants across stakeholder groups described feeling very proud with a “strong sense of ownership” of the final product. For this reason, participants felt the product would be easily taken up into practice: “It (co-production) provides a well-rounded perspective from all those involved; from those producing research to using research, whether they are clinicians, administrators or patients/caregivers” (Patient-Caregiver 2). One researcher added that it “helps us create knowledge products that meet the needs of society as the voices of the users shape the design” (Researcher 3).

There were very few challenges identified related to operationalizing the co-production process. Participants described time constraints affecting their availability and personal workloads impacting contributions to the project. They also acknowledged that engaging multiple stakeholders takes more time but that the results were “worth it in the end”:

“Sometimes the other roles that we have limit one’s ability to contribute to the desired degree and you can feel like you are letting the project team down” (Nurse 3).

DISCUSSION

This study highlights the benefits of using a co-production approach in the context of updating the COSTaRS practice guides for use by oncology nurses. Although it had been 5 years since the previous evidence update of COSTaRS, numerous new evidence sources produced few changes to the content in COSTaRS. The biggest change was having access to a new clinical practice guideline on swallowing difficulty to create a new practice guide. Using a co-production approach with planned evaluation, there was increased involvement of team members tailored to their desired level of involvement. Team members described feeling valued for their contributions. Our findings led us to the following points for discussion.

The frequency for which evidence needs to be updated within COSTaRS or the extent to which new evidence needs to be searched is not clear. This was the third systematic review to update the evidence in the COSTaRS practice guides and interestingly, there were few changes required. The new evidence sources continued to support the assessment, triage, medications, and self-care strategies included with the COSTaRS practice guides. Future updates could potentially focus on the key sources of evidence used.

Co-production of the systematic review was successful, given team members indicated high satisfaction and feeling meaningfully engaged. Several authors have explored co-production, co-design, and co-creation in health (Grindell et al., 2022; Hopkins et al., 2024; Norton, 2021), but many have focused on the use of co-production focused on healthcare interventions. Only one other study evaluated use of a co-production approach for conducting a systematic review (Lewis et al., 2024). Respondents included researchers, patients/consumers, nurses, and a graduate student. Most respondents reported extremely meaningful engagement on the PEIRS-22 instrument indicating similar results to our study. Despite time, resource, and logistic issues, respondents valued being engaged in the research process contributing to a better output. Findings from previous studies have indicated that sharing power can be a turbulent challenge mitigated by the establishment of trusting relationships in co-production (Hopkins et al., 2024). Co-production can foster a shared understanding of needs, joint ownership and engendered trust (Grindell et al., 2022). The use of co-production in our study helped to improve the relevance of the COSTaRS practice guides and enhanced our relationship with patients, caregivers, and other research team partners.

Limitations

Seventeen COSTaRS practice guides were updated from 124 clinical practice guidelines and 51 systematic reviews with meta-analysis. Concurrently, the co-production process was quantitatively and qualitatively assessed. While a validated tool was used to quantify the co-production experience, a more rigorous approach to qualitative analysis would have strengthened the study.

CONCLUSION

Our study showed that co-production was a useful strategy for meaningfully engaging team members in the conduct of a systematic review to update the evidence in the COSTaRS practice guides and create one new guide for swallowing difficulty. Although the PEIRS-22 is designed to measure co-production with patients, it was also useful for measuring meaningful involvement of all team members, including policy makers, clinicians, and researchers. The COSTaRS guidelines required a few updates, since 2020, in the areas of assessment, triage, medications, and self-care domains to reflect the new evidence base explored in this review.

The updated COSTaRS practice guides and online training are available in English and French on the Canadian Association of Nurses in Oncology and the COSTaRS websites. An electronic PDF booklet format is available for printing. A document tracking changes between 2020 and 2024 versions is available to facilitate updates of electronic health records. These tools are invaluable assets for oncology nurses to support patients experiencing treatment-related side effects and to improve patient outcomes.