Partnering with survivors & families to determine research priorities for adult out-of-hospital cardiac arrest: A James Lind Alliance Priority Setting Partnership

Abstract

Background

Research priority setting in health care has historically been done by expert health care providers and researchers and has not involved patients, family or the public. Survivors & family members have been particularly absent from this process in the field of resuscitation research and specifically adult out of hospital cardiac arrest (OHCA). As such, we sought to conduct a priority setting exercise in partnership with survivors, lay responders and their families in order to ensure that their priorities were visible. We partnered with the James Lind Alliance (UK) and used their commonly used consensus methodology for Public Priority Setting Partnerships (PSPs) to identify research priorities that reflected the perspectives of all stakeholders.

Methods

We used two rounds of public and health care professional surveys to create the initial priority lists. The initial survey collected open-ended questions while the second round consolidated the list of initial questions into a refined list for prioritization. This was done by reviewing existing evidence and thematic categorization by the multi-disciplinary steering committee. An in-person consensus workshop was conducted to come to consensus on the top ten priorities from all perspectives. The McMaster PPEET tool was used to measure engagement.

Results

The initial survey yielded more than 425 responses and 1450 “questions” from survivors and family members (18%), lay responders, health care providers and others. The second survey asked participants to rank a short list of 125 questions. The final top 25 questions were brought to the in-person meeting, and a top ten were selected through the JLA consensus process. The final list of top ten questions included how to improve the rate of lay responder CPR, what interventions used at the scene of an arrest can improve resuscitation and survival, how survival can be improved in rural areas of Canada, what resuscitation medications are most effective, what care patient’s family members need, what post-discharge support is needed for survivors, how communication should work for everyone involved with a cardiac arrest, what factors best predict neurologically intact survival, whether biomarkers/genetic tests are effective in predicting OHCA and more research on the short and long-term psycho-social impacts of OHCA on survivors. The PPEET showed overwhelmingly positive results for the patient and family engagement experience during the final workshop.

Conclusions

This inclusive research priority setting provides essential information for those doing resuscitation research internationally. The results provide a guide for priority areas of research and should drive our community to focus on questions that matter to survivors and their families in our work. In particular the Canadian Resuscitation Outcomes Consortium will be incorporating the top ten list into its strategic plan for the future.

Introduction

The role of patients in research ranges from a passive one, with patients primarily being study subjects, to an active one, with patients as co-researchers involved in the planning, design, conduct, or translation of research studies.¹ Active engagement of patients in research has many advantages, including the potential to improve the credibility of research results (through higher rates of enrollment and participant retention), and the direct applicability of research findings (when questions that are important to patients are answered).² Over the last ten years there has been greater patient participation in research as a manifestation of the “democratization” of the research process.3, 4 Patient engagement in the planning and execution of research has been shown to improve its translation into clinical practice⁵; the unique perspective provided by patients based on their lived experience of a condition or illness leads them to suggest measuring outcomes that are more applicable to patient care.⁵ Furthermore, there is growing consensus from the international research community about the crucial role of patient engagement in fulfilling an overarching ethical mandate to make the research process more transparent and holistic3, 4 and improve the overall value of healthcare research.²

In the challenging research funding environments of many developed countries, the relative “value” of research has taken on a new importance, with health research priority setting deemed essential to maximize the impact of investments.⁶ Health research prioritization, the process of obtaining a range of opinions on what research questions are the most important or timely in a particular field, is currently regarded as a key part of efforts needed to strengthen national health research systems,7, 8, 9, 10, 11, 12 particularly in combination with analyses of return on investment in health research¹³ and our understanding of the burden of disease both clinically and socially.14, 15 Accordingly, a number of comprehensive approaches to health research priority setting have been developed to guide researchers through this process.¹⁶ Historically, however, this has been done with a group of expert health care providers and researchers without the involvement of patients, family members, or the public. As priority setting techniques have been refined, the most common addition is the incorporation of public values as part of the assessment. Thus, research priority setting is increasingly being seen as combining an objective assessment of costs and effects with a more subjective assessment of patient or public preferences.¹⁷ To date, patients and members of the public have been engaged in research priority setting exercises for a variety of conditions in Canada, the United Kingdom, and elsewhere, including gestational diabetes, end-stage renal disease, early pregnancy loss, asthma, vertigo, and type 1 diabetes.¹⁸ However, survivors and family members have been particularly absent from this process in the field of resuscitation research.

The Canadian Resuscitation Outcomes Consortium (CanROC) is a pan-Canadian resuscitation research network aimed at a collaborative approach to research to improve survival from life-threatening illness and injury, including out-of-hospital cardiac arrest (OHCA).¹⁹ OHCA is defined as a sudden stop in effective blood circulation due to the heart’s unexpected failure to contract effectively or at all, therefore impeding oxygen delivery to the body and brain.²² Out-of-hospital cardiac arrest is a significant cause of death worldwide²⁰; in Canada, approximately 35,000 occur each year or one every 13 minutes.²¹ Although past research efforts have nearly doubled survival rates for OHCA victims, the current average survival rate remains low at approximately 6–8%,22, 23 meaning there is still much work to do in the field of resuscitation research.

In order to build on the CanROC vision, and create meaningful engagement opportunities for our public partners, we sought to identify research priorities in adult out-of-hospital cardiac arrest from all perspectives, in particular examining where priorities converged and diverged between different stakeholders so that we can build a robust research agenda that speaks to everyone impacted by OHCA.

Methods

Design

We adopted the Priority Setting Partnerships (PSP) methodology developed and used in multiple conditions by the James Lind Alliance (JLA) in the United Kingdom.²⁴ The methodology involves establishing a steering committee for the partnership, distributing surveys to gather uncertainties from a variety of stakeholders, and collating the collected uncertainties into a shortlist to be discussed at an in-person consensus workshop where the final top ten research priorities are selected.²⁵ This technique has been tested through over 100 PSPs and proven to help groups work effectively and reach credible and useful outcomes.¹⁸ In order for the JLA to fully endorse the final top ten research priorities, the founding principles of patient and clinician involvement, transparency and systematic rigour must be respected.²⁵ A JLA adviser (KC) was appointed by the National Institutes of Health Research (NIHR) Evaluation, Trials and Studies Coordinating Centre to support and liaise with the PSP lead (KND) and guide our cardiac arrest research priority setting process.

Setting, scope and ethics

Our PSP was pan-Canadian in scope and was specifically focused on the treatment adult out-of-hospital cardiac arrest. The study was approved by the North York General Hospital REB on August 30, 2018.

The Cardiac Arrest-Setting Priorities for Research (CA-SPR) JLA Process

We formed a steering group with diverse OHCA experiences and perspectives consisting of 2 survivors, 1 family member, 4 clinicians (2 emergency department physicians, an intensive care unit nurse and a paramedic), 2 resuscitation researchers; an experienced JLA facilitator (AL), a JLA advisor (KC) and a project coordinator (MBS). The survivor and family member partners had participated with us before on a previous project but were relatively new to research. The entire steering committee was trained in the JLA methodology by our JLA facilitator (KC). The steering committee met monthly by teleconference for the duration of the project (September 2018 to January 2020). The JLA priority setting methodology consists of four stages.

Step one: Identifying uncertainties

An online survey was used to collect feedback on potential research uncertainties from all relevant stakeholder groups. Part A of Survey 1 included four open-ended questions developed by the Steering Committee and designed to prompt people to think through the trajectory of an out-of-hospital cardiac arrest from the incident to hospital discharge or death. Part B included demographic questions about the survey respondents, such as the category of respondent (ie. survivor, family member, health care provider, lay responder, etc.), gender, age and postal code. Survey 1 was pilot tested on 5 patients and 5 health care providers.

Survey 1 was open for 6 weeks via a link on the CA-SPR PSP website (www.cardiacarrestresearch.com). A copy of the survey is included in Appendix A. Survivors, family members, health care providers, lay responders, and other stakeholders including researchers were invited to participate through emails, newsletters, and advertisements sent to relevant Canadian health professional associations, survivor and family member support groups and networks, through the professional networks of steering committee members, and posted on social media (Appendix B). Respondents were encouraged to keep questions/comments concise and were advised that there was no limit to the number of questions or topics they could pose. This was a non-probabilistic self-selection sampling strategy, however at the three-week mark of the survey being open, respondent demographics were reviewed and targeted communications were used to encourage responses from underrepresented groups.

Step 2: Refining questions and uncertainties

The responses received in Survey 1 were compiled by two team members and analyzed by the steering committee in collaboration with an information specialist. We first removed uncertainties that were out-of-scope (i.e. not concerned with adult, out-of-hospital cardiac arrest) or unclear (not researchable questions). We then grouped the in-scope responses into eight categories reflecting broad “themes” including as causes of cardiac arrest; signs, symptoms and screening; bystander response; CPR & AEDs; treatments for cardiac arrest; survivor & family experience; gender questions; and outcomes. Steering committee members then worked together in pairs or triads consisting of 1 survivor or family member and 1–2 health care providers and/or a researcher to analyze each category by converting, where possible, the uncertainties into Patient, Intervention, Comparison and Outcome (PICO) format, and then developing summary or indicative uncertainties by combining together questions addressing similar topics and eliminating duplicates. The “long list” of research uncertainties produced through this process was then assessed by an information specialist who systematically searched the literature using the Medline database, PubMed interface, and Cochrane database for existing systematic reviews and randomized controlled trials published within the last ten years. Uncertainties that had already been the focus of research that adequately answered the question were removed from the list. When the literature showed either conflicting results or there were substantial methodological shortcomings, we deemed the existing research evidence inconclusive and left the uncertainties on the long list. The steering committee subsequently met by teleconference to review the long list to ensure that the wording was clear and that indicative questions adequately summarized the relevant uncertainties.

Step 3: Interim ranking process

We programmed the remaining long list of uncertainties into a second online survey (Survey 2) and asked respondents to identify the ten most important questions from their perspective from within the longer list. Survey 2 (Appendix B) was available online for 4 weeks and distributed to all stakeholder groups using the same strategies as the first survey.

Responses from Survey 2 were analyzed by the project lead and coordinator (KND and MBS) and curated to produce a “short list” of 20 unique uncertainties for the final prioritization workshop. This was done by ordering the response statistics and selecting the top 5 for each respondent group (patients & families, lay responders, health professionals, and EMS providers). There was significant agreement at this stage between all of the stakeholder groups and so this method resulted in a final list which included the ten questions most frequently selected as important by all survey respondents and the top ten from the health care provider group (physicians, nurses, paramedics) and the survivors/family members/lay responder group.

Step 4: Final prioritization workshop

The final in-person prioritization workshop included 20 participants (5 survivors, 5 family members, 1 lay responder, 3 emergency department physicians, 2 critical care nurses, 2 paramedics, 1 CPR trainer, and 1 cardiac rehab specialist), and 3 independent facilitators with experience of the JLA method and with no previous experience in cardiac arrest, to ensure unbiased moderation of the workshop. Participants were invited from those that volunteered via Survey 2 and personal networks, to ensure diversity in terms of experience with OHCA, gender and geography.

To ensure that all voices in the workshop are heard, the JLA supports an adapted nominal group technique (NGT)²⁵ or PSPs when choosing their priorities. Care was taken to respect all participants’ views and ensure the confidentiality of each individual’s responses throughout the three rounds of small group discussion and three rounds of large group discussion at the workshop.

Participants were sent the short list of 20 uncertainties in advance, asked to individually reflect on and rank the research questions in order of priority from 1 to 20, and to bring their rankings to the workshop. This allowed the participants to be familiar with the uncertainties for discussion at the workshop. The established JLA process was followed to reach consensus.²⁵

Measuring engagement

At the end of the in-person workshop, all participants were asked to complete the validated Public and Patient Engagement Evaluation Tool (PPEET) V2.0²⁶ as a measure of engagement in the priority-setting process. The PPEET was developed by McMaster University for Canadian health care organizations to measure patient engagement. During its early development, the PPEET underwent usability testing with respondents and end-users (including patients and members of the public) who provided feedback on the structure, layout, comprehensibility, ease of use and overall utility of the survey.²⁶ It has had widespread use of the tool since its launch in 2015 and Version 2.0 was developed after use in a large comprehensive implementation study.²⁷ The PPEET is designed to explore existing enablers and barriers related to patient engagement processes, as well as the impacts and influences of patient engagement. The tool contains 21 questions/statements – five are open‐ended, and 18 were statements in four categories: (1) communication and support for participation; (2) sharing views and perspectives; (3) impacts and influence of the engagement initiative; and (4) final thoughts and rated using a 5‐point Likert scale, ranging from ‘strongly disagree’ to ‘strongly agree’. The tool was administered in paper format onsite and anonymously collected in a locked box placed at the exit of the meeting room.

Results from completed PPEET surveys were entered into a Microsoft Excel worksheet for analysis. Descriptive statistics (frequencies, means, standard deviations, ranges) were calculated for the demographic variables and PPEET's quantitative responses.

Results

The initial survey generated a total of 1500 questions from 408 individuals. Survivors and family members represented 18% of the respondents. Survey 2 generated a total of 314 responses from the same respondent groups and a similar number of those who identified as survivors and family members. The characteristics of respondents from both surveys and the final workshop is provided in Table 1.

Table 1.

Participant characteristics.

Participant type*	Survey 1 (372/408 provided demographic data)	Survey 2 (312/314 provided demographic data)	Final workshop (20 total participants)
Person who has survived a cardiac arrest	54 (14.52%)	40 (12.82%)	5 (25%)
Spouse/relative/friend/caregiver of someone who survived a cardiac arrest	13 (3.49%)	15 (4.81%)	4 (20%)
Spouse/relative/friend/caregiver of someone who did not survive a cardiac arrest	6 (1.61%)	5 (1.60%)	1 (5%)
Health care professional	279 (75%)	233 (75%)	6 (30%)
Physician	88 (27.94%)	34 (14.59%)	3 (15%)
Nurse	13 (41.90%)	142 (61%)	2 (10%)
Rehabilitation specialist (PT, OT, etc)	1 (0.32%)	2 (0.86%)	1 (5%)
Paramedic/EMS provider	32 (10.16%)	42 (19.74%)
Dentist	1 (0.32%)	–
Optometrist	–	1 (0.43%)
Other	61 (19.37%)	8 (3.43%)	2 (10%)
Other (please specify)	20 (5.38%)	19 (6.09%)	2 (10%)
*Responses were mostly those who fit in more than one category.
Female gender	221 (59.73%)	225 (72.82%)	10 (50%)

After analysis of Survey 1 responses, 556 were determined to be out of scope, (e.g., recount of a personal story, questions relating to paediatric cardiac arrest, etc.). This left 944 responses that were in-scope. These were then grouped into a preliminary list of 75 broader summary questions by the committee by grouping similar questions together using summative wording. For example, several responses which asked about risk factors related to cardiac arrest were represented by “Which risk factors best predict sudden cardiac arrest?”. Eight of the responses provided were deemed to have already been answered by existing research and were not included in stage 2.

The remaining 67 research questions were programmed into a second online survey (Survey 2). A total of 312 individuals participated in Survey 2 which asked them to select the ten questions which they felt were most important for research to answer. Of these participants, 60 (19.2%) were survivors or family members, and the balance were a mix of healthcare professionals. While some participants would have completed both survey 1 and 2 and some might have only done one or the other, due to the anonymous nature of the surveys there is no way to determine what, if any bias this may have on the results.

The steering group analyzed the responses to Survey 2 and created a shortlist of 20 uncertainties that were considered at the final prioritisation workshop, where the final top ten unanswered research questions (Fig. 1) were agreed to by all participants. The shortlist of 20 was created by including the uncertainties ranked as the top 5 by each group (survivors, families, health professionals and lay responders).

Fig. 1.

Top ten priorities.

The top ten research priorities included questions about how to improve the rate of lay responder CPR, what interventions used at the scene of an arrest can improve resuscitation and survival, how we improve response times and survival in rural areas of Canada, what resuscitation medications are most effective, what care patient’s family members need following an arrest, what post-discharge care should include for cardiac arrest survivors, how we can communicate back to everyone involved with a cardiac arrest, what factors best predict neurologically intact survival, whether there are biomarkers or genetic tests that are effective in predicting OHCA and what are the short and long-term psycho-social impacts of OHCA on survivors.

Although these results seem easily summarized here, the path to the top ten at the final workshop was not an easy one. At the beginning, the top question for patients and families was the bottom question for health care providers and vice versa. For example, the final decision to place the most effective mechanisms for improving the rate of lay responder/bystander CPR as the number one priority came out of very pragmatic discussion about the fact that without effective bystander CPR to increase the chances of survival, the rest of the questions about treatment, etc. are irrelevant. Interestingly it also seemed that clinicians were initially less concerned than the other groups with what happens before a patient arrives at hospital which is unique to this JLA because of the importance of the prehospital care in the chain of survival. However, throughout the discussion and in hearing from patients and family members, the clinicians could better appreciate why this was of high importance to survivors and their families. This was a strong example of why having all perspectives at the table is so important.

Perceptions of engagement during the in-person workshop

Of the 20 participants in the final workshop, 15 completed and submitted a PPEET survey tool (5 survivors, 5 family members, four health care providers and 1 lay responder). Feedback was overwhelmingly positive with 97% of all respondents indicating they agreed or strongly agreed with the statements provided.

In the section of communication and supports for participation, respondents strongly agreed or agreed (93–100% response frequency) with: (1) having a clear understanding of the purpose of the project; (2) having the supports needed to participate; (3) having enough information to contribute to the topics being discussed. In the sharing views and perspectives section, respondents ‘strongly agreed’ (73–93% response rate) that: (1) they can express their views freely; and (2) that their views are being heard; (3) a wide range of views on the topics discussed were shared; and (4) the individuals at the priority setting workshop represented a broad range of perspectives on the topics being discussed.

In the impact and influences section, most participants (53–73% response rate) indicated strong agreement that the engagement initiative fulfilled its objectives and that the input provided through this project will make a difference to the work of the project. They were also confident (93% response rate) that the input provided through the priority setting partnership will be used by the project.

In the final thoughts section, respondents (93–100% response rate) conveyed that they are now better informed about cardiac arrest and resuscitation research, they were satisfied with the group and the initiative overall, and that this patient engagement activity was a good use of their time.

Discussion

We feel we were able to conduct a fair and inclusive PSP on adult cardiac arrest, a topic which had not been previously worked on within the JLA. We are very pleased to see that our top ten list represents a balance of clinical, systems and patient-oriented questions. This not only demonstrates the equal value of the various perspectives involved in this process but also the complexity of out-of-hospital cardiac arrest and the multiple care systems involved.

The top ranked question in the final top ten list related to understanding the most effective mechanisms for improving the rate of lay responder/bystander CPR; the group discussed how this was most important because “without this, the rest doesn’t matter…” (quote from a participant). Two of the top ten priorities related to prehospital issues, four related to support and communication for the survivors, families and those involved in the resuscitation and the remaining priorities included developing knowledge about the most effective medications and predictive tools for OHCA such as genetic testing. The list represented what we feel to be a very balanced set of research priorities that highlights the significance of focusing on increasing our knowledge at all points in the chain of survival.²⁸ A very clear message was also the prioritization of research in areas that fall under the new sixth link of recovery and support.²⁹ In addition, our final priorities include several of those focused on understanding the post-discharge support needed for survivors and families as outlined in several recent studies30, 31, 32, 33 including the American Heart Association Survivorship Scientific Statement³⁴ and gaps in our knowledge on bystander CPR, resuscitation medications and neuroprognostication identified by the 2015 ILCOR consensus statement on scientific knowledge gaps and clinical research priorities.³⁵

Over the past decade, patient and public involvement (PPI) has been highlighted worldwide in both health research agendas and the development of next-step research projects.³⁶ Researchers have noted that involving healthcare service users, the public and patients, improves research quality, relevance, implementation and cost-effectiveness; it also improves researchers’ understanding of and insight into the medical and social conditions they are studying.³ The JLA process enabled us to ensure the topics related to each group of people affected by OHCA were thoroughly discussed and purposefully included in the final top ten list. First responders, health care providers, patients and families were very aware of all the pieces that come together to improve a victim’s chance of survival from OHCA. Having said that, the idea of saving a life and the variability in lived experience for all the participant groups meant that this particular topic was particularly sensitive at times. The entire journey of this PSP made us highly aware of how the list would be very different if the groups had been polled separately and more importantly if we hadn’t had a such an open, respectful and supportive process. This was truly a co-creation experience that saw everyone look at research priorities for OHCA in a different way. The positive experience of engagement was not only palpable by the research team but was supported by the overwhelmingly positive outcomes and comments we captured with the PPEET tool.

As a community, CanROC has made significant headway on creating the infrastructure to achieve our goals and originally stated deliverables around public engagement in resuscitation research. We have had successful public engagement to date, but we are cognizant of the need to ensure we provide as many opportunities to co-design the future of resuscitation science in Canada as possible and that there are tangible returns on investment, particular of time and feedback from our public partners.

Strengths & limitations

This PSP is the first one that has been conducted with the JLA in cardiac arrest. Previous PSPs related to stroke and intensive care are similar in terms of a sudden condition but they were much broader in scope.37, 38 Our study has a number of strengths, most notably our transparent and successful process of engaging patients and clinicians from across Canada. That said, we did struggle to recruit physicians from certain key specialties including critical care and cardiology. This is likely due to the fact that cardiac arrest is just one of a number of clinical conditions these specialists treat. We also worked very hard to reach survivors and family members because of the lack of centralized coordination of this group in Canada. Having said that we were able to recruit strong numbers of patients and families for the final workshop so we feel their voice was very present in the final results.

As expected, many of the submissions, especially those from patients, were not worded as research questions but rather as comments, concerns or stories. Steering committee members used judgement when turning these comments into research uncertainties. Also, the processing and collation of uncertainties was subjective due to the quick turn around required between surveys and did not have the rigour associated with qualitative techniques such as thematic analysis. We did however publish our full methods protocol, process and results on the JLA website for full transparency. Despite these limitations, we feel the PSP provided a robust list of questions that will inform more holistic discussions for resuscitation research investment in the future.

Conclusion

Using the methodology of JLA Priority Setting Partnership we were able to develop consensus on a top ten list of research priorities for out-of-hospital cardiac arrest in adults. This priority setting exercise valued the perspectives of survivors, family members, bystanders and health care professionals equally. The results of the CA-SPR PSP will be used to focus on active co-design of the CanROC research agenda moving forward and will be shared with our international counterparts to support public engagement in resuscitation science worldwide.

Conflict of interest disclosures

The authors have no conflicts of interest to disclose related to this research.

CRediT authorship contribution statement

KND and MBS led the conception and design of the study and had full access to all of the data in the study, collected and/or analyzed data and prepared the first draft of the manuscript; KC was the JLA advisor assigned and provided guidance throughout the PSP process; MD, PD, JG, JG, DS, DS, CT and CV served on the steering committee, reviewed and analyzed the data and reviewed and edited the manuscript; AL served as a consultant in the implementation of this study, facilitated the process and the final prioritization workshop and reviewed and edited the manuscript.

Appendix A. Supplementary data

The following is Supplementary data to this article: