Abstract
Purpose
The importance of patient-oriented research (POR) has been recognized by the scientific community and governmental agencies, and its development is exponential across most health-related disciplines. The current Canadian Institutes of Health Research (CIHR) definition of POR is overly broad and hinders the reliable selection of POR-related publications from bibliographic databases. The present study was aimed to adapt CIHR’s definition of POR into an operational definition that can be used by stakeholders for selecting POR publications.
Methods
Eighteen POR experts in Québec, Canada, were invited to participate in a modified e-Delphi study. Two rounds of Delphi surveys were undertaken to reach consensus. Round-1 sought consensus on clarity and indispensability of POR characteristics. Round-2 included modifications planned in POR characteristics and obtained final consensus leading to an adapted POR definition. Finally, POR experts across Canada were consulted to assess generalizability of this adapted POR definition.
Results
The item that achieved 75% of consensus was removal of the POR characteristic “POR can be conducted in partnership with relevant stakeholders,” because it was considered redundant and confusing. Additionally, participants suggested defining unclear concepts such as “continuum,” “direct impact,” and “patient.” Finally, based on results of Round-1 and Round-2 and the consultation with POR experts across Canada, an operational POR definition was developed.
Conclusions
This study was a novel attempt to adapt an operational POR definition to help patients and POR stakeholders have a common understanding of what POR is, focus on important outcomes that matter to patients, and improve care quality.
Keywords: patient-oriented research, definition, modified e-Delphi, patient-centered outcomes, patient engagement
Patient-oriented research (POR) is a continuum of research that engages patients as partners.1 The roots of POR can be traced back to Rufus Cole’s concepts apparent from his letter written in 1911 to Simon Flexner. Both Cole and Flexner were directors at the Rockefeller Institute for Medical Research in the United States. Cole stated in his letter that to understand diseases and improve treatments, more refined methods of investigation were required. Furthermore, he expressed that in a hospital environment physicians should, like researchers, undertake an exhaustive scientific study of disease by engaging their patients.2
With widespread agreement globally about the importance of patient engagement in health research, the way research is conducted has shifted.3 Progressively, the importance of POR was recognized by the scientific community4 and governmental agencies.1,5 Subsequently, POR-related infrastructure and programs developed across most health-related disciplines.6–8 In the United Kingdom, the National Institute of Health Research established the national advisory group INVOLVE to engage individual and community stakeholders.9 In the United States, the Patient-Centered Outcomes Research Institute (PCORI) emphasizes patient engagement in research.10 In Canada, the Strategy for Patient-Oriented Research (SPOR) was initiated “to foster evidence-informed health care by bringing innovative, diagnostic, and therapeutic approaches to the point of care, so as to ensure greater quality, accountability, and accessibility of care.”1
While there is agreement regarding engaging patients in decision-making about health research, the definitions of POR vary worldwide.8,11,12 A scoping review on POR found that the language used for describing POR is inconsistent.11 For example, the U.S. National Institutes of Health has defined POR as “research conducted with human subjects (or on material of human origin, such as tissues, specimens and cognitive phenomena)” that requires direct interactions with human subjects.13 Shaywitz et al defined POR as “integrative approaches to understanding how component molecules and physiological systems function in the context of the whole person.”14 The Canadian Institutes of Health Research (CIHR) has defined POR as “a continuum of research that engages patients as partners, focusses on patient-identified priorities and improves patient outcomes. This research, conducted by multidisciplinary teams in partnership with relevant stakeholders, aims to apply the knowledge generated to improve healthcare systems and practices.”1 SPOR adheres to the following principles: patients need to be involved in all aspects of the research to ensure questions and results are relevant; decision-makers and clinicians need to be involved throughout the entire research process to ensure integration into policy and practice; and effective patient-oriented research requires a multidisciplinary approach.1
Based on our experience with POR and discussions with POR stakeholders (researchers, clinicians, patient partners, and decision-makers), we concluded that there is a lack of an agreed-upon POR definition worldwide. Furthermore, CIHR’s definition of POR is not sufficiently operational to filter and retrieve POR-related publications from bibliographic databases in a reliable manner.15 Consequently, it is challenging to monitor the trends in POR publications and products (eg, protocols, guidelines, checklists). In May 2017 at the Association Francophone pour le Savoir conference, a 1-day colloquium,16 the need for consensus on an operational (actionable) definition of POR was expressed by delegates.
Such a definition would benefit POR stakeholders in two ways. First, it would help them to monitor the latest POR trends and to reliably select POR publications. Second, they will be able to use this definition to identify potential collaborators (eg, authors of such publications) and form multidisciplinary or transdisciplinary teams for addressing broad POR issues. In turn, this can provide clarity to focus on important outcomes that specifically matter to patients and can improve care quality. Thus, based on the literature pertaining to building concepts and theories in information science,17 the present study was aimed to adapt CIHR’s current definition of POR into an operational definition.
METHODS
Study Design
The CIHR definition of POR essentially describes the characteristics of POR. We sought to go beyond this basic description and conducted a two-round modified e-Delphi study18,19 using online surveys to reach consensus among POR experts on operational characteristics of POR. For our study’s purpose, an operational POR characteristic is defined as a feature or quality of POR that can serve to identify POR publications and products. The breakdown of POR characteristics described in CIHR’s definition is presented in Table 1.
Table 1.
List of Characteristics Delineated From the CIHR Definition of Patient-Oriented Research (POR)
| Characteristic |
|---|
|
CIHR, Canadian Institutes of Health Research.
Ethical Considerations
This study was approved (#A06-E48-17A) by the institutional review board for the Faculty of Medicine at McGill University (Montréal, Canada). Delphi panel experts provided informed consent to participate and agreed to be acknowledged in associated publications. All survey responses were rendered anonymous.
Participants and Setting
A POR expert was defined as an individual having knowledge of, experience with, and a leadership role in POR. The POR experts invited to participate had diverse backgrounds and were the directors, coordinators, patient partners, health and social care practitioners, managers, and other stakeholders involved in the Québec SPOR-SUPPORT Unit, which provides support for people and patient-oriented research and trials. In total, 18 POR experts were invited via email and 15 agreed to participate. As per the snowballing technique, experts were asked to recommend other potential Delphi panelists. Our sample size is justified given that 10 to 30 is sufficient for building consensus using the Delphi method.20,21
SPOR-SUPPORT units within their respective Canadian provinces “provide decision-makers and health care providers with the ways and means to connect research with patient needs so that evidence-based solutions can be applied to health care and then shared throughout the country.”22 The main partners of Québec’s SPOR-SUPPORT Unit are funding agencies both federal (CIHR) and provincial (Ministère de la Santé et des Services sociaux [Québec City], Fonds de recherche du Québec [Montréal]), as well as 4 Québec universities (McGill, Université Laval, Université de Montréal, and Université de Sherbrooke).23
Data Collection
We developed the online survey and pilot tested with 1 research coordinator, 2 graduate students, and 1 research associate at McGill’s Department of Family Medicine. Data were collected from August 2017 to March 2018. POR experts were sent individualized email invitations including the link to the survey posted on LimeSurvey software hosted on the McGill University server. They were asked to complete the survey within 2 weeks. Those who did not respond were sent an email reminder. Modifications to the POR definition were made based on results obtained from Round-1, and the online survey for Round-2 was developed accordingly. Again, participants were provided 2 weeks to respond to the Round-2 survey and were sent an email reminder, as needed.
Round-1
The Round-1 survey asked experts to rate the level of clarity and indispensability (whether necessary or not) of the POR characteristics using a 5-grade Likert scale. Results from Round-1 were used to formulate modifications to POR characteristics for dissemination in Round-2.
Round-2
The Round-2 survey asked participants to rate their level of agreement or disagreement with the proposed modifications of POR characteristics (after incorporating the suggestions provided in Round-1). Participants also provided comments regarding 3 parts of Round-2: (i) removal of the POR characteristic(s) from the current POR definition; (ii) modifications planned in the current POR characteristic(s); and (iii) indispensability of each POR characteristic. POR experts were asked to comment on POR characteristics, to suggest possible modification, and to highlight any feature that may have been missed in POR characteristics. In both Round-1 and Round-2, space was provided for adding comments or suggestions for all questions.
Consultation Exercise
In order to gather a broader perspective on our adapted POR definition, we conducted a consultation exercise among POR experts from SPOR-SUPPORT units and networks across Canada. In brief, we sent out individualized emails in June 2018 to POR experts (all of whom were directors of units or networks outside Québec) that asked them to read the adapted definition of POR, express their agreement or disagreement, and provide suggestions.
Data Analysis
Descriptive statistics (mean, median, standard deviation, and percentage of agreement) were calculated for all rated items using SPSS statistical software (Version 24, IBM, Armonk, NY). As previously described by Foth et al,24 we predefined consensus. For each item we determined that consensus was achieved when the percentage of agreement was at least 60%. To calculate percentage of agreement, the following formulae were used:
- Round-1 consensus:
- Round-2 consensus:
We performed content analysis of the qualitative data (panelists’ comments) to identify and to describe any implicit and explicit ideas. Figure 1 presents the flow chart for the two-round Delphi.
Figure 1.
Flow chart describing both rounds of modified e-Delphi study methods. CIHR, Canadian Institutes of Health Research; POR, patient-oriented research.
RESULTS
Description of POR Experts
Of the 18 experts invited to Round-1, 13 completed the entire survey, 2 left it incomplete, and 3 did not respond. The participants reported having between 1 and 30 years of POR experience and involvement in 1 to more than 15 POR studies. Five panelists had been either a principal or co-investigator in POR projects, 3 were a research associate or coordinator or assistant, 1 was a manager, 1 was a clinical trial coordinator, 1 was an administrative coordinator, 1 was a clinician, and 1 was a patient. The patient partner is a member of the Centre of Excellence on Partnership with Patients and the Public at Université de Montréal.
Round-1
Regarding the clarity of POR characteristics (Table 2), consensus (ie, ≥60% agreement) was achieved for all but one characteristic: “G. POR can be conducted in partnership with relevant stakeholders” (53.8%).
Table 2.
Level of Clarity* of POR Experts (n=13) Regarding POR Characteristics
| POR Characteristic | Mean | SD | Median | Agreement† |
|---|---|---|---|---|
| A. POR is a continuum of research that involves patients (from being consulted to being engaged as a partner) in all aspects of research. | 3.77 | 1.17 | 4 | 69.2% |
| B. POR is focused on patient’s priorities. | 3.85 | 0.55 | 4 | 76.9% |
| C. POR is aimed to have direct impact on patients. | 3.92 | 0.86 | 4 | 76.9% |
| D. POR is aimed to improve health care practices or systems. | 4.08 | 0.64 | 4 | 84.6% |
| E. POR involves decision-makers and clinicians throughout the entire research process to ensure integration into policy and practice. | 3.62 | 0.96 | 4 | 61.5% |
| F. POR can be conducted by multidisciplinary team. | 3.77 | 0.93 | 4 | 61.5% |
| G. POR can be conducted in partnership with relevant stakeholders. | 3.54 | 1.05 | 4 | 53.8% |
Scale: 1 = very unclear; 2 = unclear; 3 = neither clear or unclear; 4 = clear; 5 = very clear.
Percentage of agreement in bold had an agreement less than 60%.
POR, patient-oriented research; SD, standard deviation.
The level of agreement reported by POR experts regarding indispensability (whether necessary or not) of the POR characteristics showed that only 54% of participants agreed that all POR characteristics were necessary. Based on analyses of panelist comments from Round-1, the following modifications were planned:
Clarify or define the “continuum.”
Replace “focused on patient’s priorities” with “addresses patient’s priorities.”
Define the terms “direct impact” and “patient.”
Address the opinion that “[characteristic] D seems to be too restrictive.”
Replace “decision-makers” with “decision/policy-makers.”
Specify “multidisciplinary or transdisciplinary team.”
Remove characteristic G because it is “redundant and confusing.”
Round-2
The 13 full respondents of Round-1 as well as the 2 who partially completed the Round-1 survey were invited to participate in Round-2. Of these 15 POR experts, 12 completed the survey and 3 did not respond. Table 3 presents the modifications proposed to Delphi panelists based on Round-1 results, along with the percentage of POR panelists who agreed with the modification. POR characteristic G achieved a consensus (75%) of POR experts agreeing with its removal from the current POR definition due to redundancy.
Table 3.
Consensus of POR Experts on the Planned Modifications to Current POR Characteristics
| Round-1 POR Characteristics | Suggestions of POR Experts | Modifications Planned | Agreement† |
|---|---|---|---|
| A. POR is a continuum of research that involves patients (from being consulted to being engaged as a partner) in all aspects of research. | Clarify or define the “continuum.” | Phrase “(from being consulted to being engaged as a partner)” will be replaced with “(continuum from being consulted to being engaged as research partners).” | 66.7% |
| B. POR is focused on patient’s priorities. | Replace “focused on patient’s priorities” with “addresses patient’s priorities.” | This will be done. | 100% |
| C. POR is aimed to have direct impact on patients. | Define terms “direct impact” and “patient.” | Phrase “to have direct impact on patients” will be replaced with “to address patient outcomes (patients including relatives, family caregivers and the public).” | 83.3% |
| D. POR is aimed to improve health care practices or systems. | “D seems to be too restrictive.” | No change is planned. | 66.7% |
| E. POR involves decision-makers and clinicians throughout the entire research process to ensure integration into policy and practice. | Replace “decision makers” with “decision/policy-makers.” | This will be done. | 100% |
| F. POR can be conducted by multidisciplinary team. | Specify “multidisciplinary or transdisciplinary team.” | Phrase “multidisciplinary team” will be replaced with “multidisciplinary or transdisciplinary team (including decision/policy-makers and clinicians).” | 83.3% |
| G. POR can be conducted in partnership with relevant stakeholders. | Remove G (“redundant and confusing”). | This will be done. | 75% |
Percentage of agreement in bold had an agreement greater than or equal to 60%.
POR, patient-oriented research.
Table 4 provides POR experts’ comments regarding the modifications proposed.
Table 4.
Comments of POR Experts Regarding Planned Modifications to Current POR Characteristics
| Suggestions From POR Experts (Labeled P1, P2, P3, and P5) | Modification |
|---|---|
| P1: To add “patients” as an option of team member | Done |
| P2: Notion of “partner” is essential and must be preserved | Done |
| P3: (a) Confirm definition of continuum of CIHR | (a) Done |
| (b) Include improve patient outcomes | (b) Done |
| (c) Include “patients” to relevant stakeholders list | (c) Done |
| P5: (i) Revise to “addresses outcomes deemed important by patients” | (i) Done |
| (ii) “POR is a continuum of research that involves patients as members of the research team, beyond their role as research subject” | (ii) Done |
CIHR, Canadian Institutes of Health Research; POR, patient-oriented research.
Consensus, Comments Regarding Indispensability of POR Characteristics
Only 2 of the POR characteristics were rated as unnecessary (Table 5). Specifically, 66.7% of panelists considered POR characteristic F unnecessary and 75% rated POR characteristic G unnecessary. It should be noted that even regarding its indispensability, characteristic G achieved a 75% consensus of POR experts considering it “not necessary.” However, POR experts suggested including “transdisciplinary team” in characteristic F. Table 5 presents consensus ratings, and Table 6 presents the comments of POR experts.
Table 5.
Consensus of POR Experts on Necessity of POR Characteristics
| Round-1 POR Characteristic | Necessary? | Agreement* |
|---|---|---|
| A. POR is a continuum of research that involves patients (from being consulted to being engaged as a partner) in all aspects of research. | Necessary | 100% |
| B. POR is focused on patient’s priorities. | Necessary | 100% |
| C. POR is aimed to have direct impact on patients. | Necessary | 91.7% |
| D. POR is aimed to improve health care practices or systems. | Necessary | 100% |
| E. POR involves decision-makers and clinicians throughout the entire research process to ensure integration into policy and practice. | Necessary | 91.7% |
| F. POR can be conducted by multidisciplinary team. | Not necessary | 66.7% |
| G. POR can be conducted in partnership with relevant stakeholders. | Not necessary | 75% |
Percentage of agreement in bold had an agreement greater than or equal to 60%.
POR, patient-oriented research.
Table 6.
Comments of POR Experts Regarding Indispensability of POR Characteristics
| Suggestions for Characteristics (A), (C), and (F) | Modification |
|---|---|
| A: Reformulate continuum as “research that involves patients, from being consulted to being engaged as a research partner, in all aspects of research when relevant.” | Done |
| C: “Direct impact might be strong; perhaps the aim is to ultimately improve patient health.” | Done |
| F: Include “transdisciplinary team.” | Done |
POR, patient-oriented research.
Adapted Definition of POR
In view of findings from the two rounds of Delphi, one POR characteristic, G, was removed from CIHR’s definition of POR. The following POR definition represents the consensus of the panelists: POR satisfies at least condition 1 (C1), or condition 2 (C2), or conditions 1 and 2.
Condition 1: Patients (including relatives, family caregivers, and the public) are involved as research partners with multidisciplinary or transdisciplinary research team members (including decision/policy-makers, patients, and clinicians) along a continuum (from being consulted to being engaged) in addressing patient priorities or planning/conducting research (eg, formulation of the question; data collection/analysis; interpretation, diffusion, dissemination, or application of results), or both addressing patient priorities and planning/conducting research.
Condition 2: Studies are aimed to (a) address outcomes deemed important by patients; (b) have a direct impact on at least one of the following targets: patient health and experiences, health professionals’ practice, or health care services and policies; or (c) achieve both objectives C2(a) and C2(b).
Results of the Consultation Exercise
In total, we sent invitations to 17 participants (directors of SPOR units and networks outside Québec or their coordinators). Of these, 12 (70.5%) responded and 5 did not respond. Table 7 presents the suggestions of these Canadian POR experts.
Table 7.
Comments of POR Experts Across Canada Regarding Agreement of Adapted POR Definition
| Suggestions From Canadian POR Experts (Labeled P1, P2, P3, and P4) | Action Taken |
|---|---|
| P1: For Condition 2(b), add “patient experience.” | Done |
| P2: For Condition 2, would recommend a narrower definition rather than “deemed important by patients” because it does not specify how exactly this was determined and just about any research can be deemed important by any one patient. Would suggest wording to the effect of “deemed important/relevant by patients through patient priority-setting process.” | While comment is interesting, study authors determined the suggested criterion would be useful for screening full texts |
| P3: Condition 2(b) would pick up a lot of false-positives, as it would pick up studies that have a direct impact on patient outcomes, health professionals’ practice, or health care services and policies that represent a much wider field than POR (ie, a study might address one of those areas but not be patient-oriented). | No action taken |
| P4: Condition 2 is not enough to be considered POR without Condition 1. | No action taken |
POR, patient-oriented research.
Of the 12 participants who responded, 10 (83.3%) completely agreed and only 2 disagreed with our adapted definition of POR. All participants completely agreed with the C1 part of the adapted definition, and few comments were received concerning the C2 part. We incorporated the suggestion to add “patient experience” to C2(b) and revised it accordingly. Interestingly, contradictory comments were received regarding the C2 part of the adapted definition. One participant commented that C2 is a “narrower definition,” whereas another participant perceived it as too broad. Given that a majority of directors or coordinators of SPOR-SUPPORT units and networks across Canada expressed their complete agreement with the adapted definition (83.3%), we decided to retain C2 as accepted.
DISCUSSION
The results of the present study provide POR experts’ consensus on an operational definition of POR. The use of a modified e-Delphi design enabled our busy POR experts to access and respond to online surveys at their convenience, which is not always possible in face-to-face group meetings. Furthermore, based on their diverse perspectives and experiences, the POR experts enriched this POR definition with their valuable suggestions.
The adapted POR definition was made operational (actionable) for selecting publications in 5 ways: First, we added the Boolean operators (AND, OR) to (a) include publications that combine the two conditions (condition 1 and condition 2) of the adapted definition, and (b) exclude publications satisfying only one of the conditions (condition 1 only or condition 2 only). It is expected that this will enable stakeholders to select POR publications in a more reliable manner. Second, all nonintuitive (not immediately clear without complete definition) concepts such as “continuum,” “direct impact,” and “patient” were defined to bring further clarity. Third, the role of patient in POR (from being consulted to being engaged) was specified. Fourth, a redundant and confusing POR characteristic, “POR can be conducted in partnership with relevant stakeholders,” was removed. Fifth, as suggested by panelists, “transdisciplinary team” was included in the POR definition.
We acknowledge that this Delphi is limited by its small sample size, and the response rate slightly declined in the subsequent round (15 respondents in Round-1, 12 respondents in Round-2). However, a sample size ranging from 10 to 30 is sufficient for building consensus using a Delphi method,20,21 and the reported dropout is not exclusive to the present Delphi.25 Other limitations were that this Delphi was conducted with POR experts who are part of the Québec SPOR-SUPPORT Unit and involved only one patient partner. This study would have profited from a recruitment of POR experts from other networks across Canada and more patient partners. Recognizing this, we gathered a broader perspective on our adapted POR definition among POR experts by conducting a consultation exercise among POR experts from SPOR-SUPPORT units and networks across Canada.
Despite aforementioned limitations, this study provided an operational POR definition that can be beneficial to patient partners, researchers, clinicians, and POR stakeholders to select POR publications, monitor POR trends, and focus on outcomes that matter to patients.
CONCLUSIONS
This study used modified e-Delphi design to build consensus on an operational definition of patient-oriented research. Any stakeholders can use this POR definition to reliably retrieve POR publications from bibliographic databases for monitoring POR trends. This information can provide researchers with a common understanding of “what is” POR and help focus study designs on important outcomes that specifically matter to patients, thereby improving care quality.
Future plans include testing the accuracy and reliability of this POR definition and its subsequent use to measure the performance (precision and recall) of a map of POR publications in health and social sciences currently under development by the Québec SPOR-SUPPORT Unit (Method development platform – fiche resume des initiatives en cours, unpublished document, January 2018). This map will serve as a directory for POR experts to identify potential collaborators, assist networking, and monitor POR trends to focus on improving patient outcomes.
Patient-Friendly Recap.
Patient-oriented research (sometimes called patient-centered or patient-engaged research) is nebulously defined by various health research agencies.
The authors queried experts throughout Quebec, Canada, to pinpoint an operational definition of patient-oriented research.
Two conditions — patient involvement in planning and outcomes deemed important by patients — were identified as required for a study to qualify as patient-oriented research.
The ultimately adapted definition can be used to more reliably select existing literature on patient-oriented research.
Acknowledgments
We sincerely thank all the patient-oriented research experts (presented alphabetically) who dedicated their valuable time to participate in this e-Delphi study and agreed to be acknowledged: Alain Vanesse (Université de Sherbrooke, Sherbrooke, Canada); Alexandre Gregoire (Centre hospitalier de l’Université de Montréal, Montréal, Canada); Antoine Boivin (Université de Montréal, Montréal, Canada); Anne-Marie Cloutier (Université de Sherbrooke); Bernadette Diedhiou (CIUSSS du Centre-Sud-de-l’Île-de-Montréal, Montréal, Canada); Elham Rahme (McGill University); Émilie Grenier (CIUSSS du Centre-Sudde-l’Île-de-Montréal); France Légaré (Université Laval, Québec City, Canada); François Fournier (CIUSSS du Centre-Sud-de-l’Île-de-Montréal); Geneviève David (Centre of Excellence on Partnership with Patients and the Public, Montréal, Canada); Hervé Zomahoun (Université Laval); Jean-François Ethier (Université de Sherbrooke); Janusz Kaczorowski (Université de Montréal); Martine Fournier (Centre hospitalier de l’Université de Montréal); Paula L. Bush (McGill University and Institut national d’excellence en santé et en services sociaux, Québec City, Canada) and Vincent Dumez (Université de Montréal). We also thank the professional editor for editing this manuscript.
Footnotes
Author Contributions
Study design: Kaur. Data acquisition or analysis: Kaur, Pluye. Manuscript drafting: Kaur. Critical revision: Pluye.
Conflicts of Interest
None.
Funding Sources
Navdeep Kaur holds a postdoctoral fellowship from the Québec SPOR-SUPPORT Unit. Pierre Pluye holds a senior scholarship award from the Fonds de recherche du Québec – Santé.
References
- 1.Canadian Institutes of Health Research. Strategy for patient-oriented research – patient engagement framework. [Accessed June 20, 2018]. Date modified 2014 Jul 2. http://www.cihr-irsc.gc.ca/e/48413.html.
- 2.Ahrens EH., Jr The birth of patient-oriented research as a science (1911) Perspect Biol Med. 1995;38:548–53. doi: 10.1353/pbm.1995.0073. [DOI] [PubMed] [Google Scholar]
- 3.Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14(89) doi: 10.1186/1472-6963-14-89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Sacristán JA. Patient-centered medicine and patient-oriented research: improving health outcomes for individual patients. BMC Med Inform Decis Mak. 2013;13(6) doi: 10.1186/1472-6947-13-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Collier R. Federal government unveils patient-oriented research strategy. CMAJ. 2011;183(13):E993–4. doi: 10.1503/cmaj.109-3978. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Denig P, Schuling J, Haaijer-Ruskamp F, Voorham J. Effects of a patient oriented decision aid for prioritising treatment goals in diabetes: pragmatic randomised controlled trial. BMJ. 2014;349:g5651. doi: 10.1136/bmj.g5651. [DOI] [PubMed] [Google Scholar]
- 7.Gabrielsson S, Sävenstedt S, Zingmark K. Person-centred care: clarifying the concept in the context of inpatient psychiatry. Scand J Caring Sci. 2015;29:552–62. doi: 10.1111/scs.12189. [DOI] [PubMed] [Google Scholar]
- 8.Strom BL, Norman S, Margolis DJ. Patient-oriented research: definitions and new paradigms. Am J Med. 2000;109:164–5. doi: 10.1016/S0002-9343(00)00502-7. [DOI] [PubMed] [Google Scholar]
- 9.INVOLVE, National Institute for Health Research. [Accessed April 14, 2018];About INVOLVE. http://www.invo.org.uk/about-involve/ [Google Scholar]
- 10.Patient-Centered Outcomes Research Institute. [Accessed April 14, 2018];Engagement. https://www.pcori.org/engagement [Google Scholar]
- 11.Manafo E, Petermann L, Mason-Lai P, Vandall-Walker V. Patient engagement in Canada: a scoping review of ‘how’ and ‘what’ of patient engagement in health research. Health Res Policy Syst. 2018;16(1):5. doi: 10.1186/s12961-018-0282-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Lough S. Need to define patient engagement in research. CMAJ. 2015;187(12):E385–6. doi: 10.1503/cmaj.109-5109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.National Institutes of Health. Research Involving Human Subjects. Glossary. [Accessed April 14, 2018]. Last updated 2018 Mar 22. https://humansubjects.nih.gov/glossary.
- 14.Shaywitz DA, Martin JB, Ausiello DA. Patient-oriented research: principles and new approaches to training. Am J Med. 2000;109:136–40. doi: 10.1016/S0002-9343(00)00452-6. [DOI] [PubMed] [Google Scholar]
- 15.Granikov V, Tang DL, Bouthillier F, Pluye P. eSRAP: a system for collaborative monitoring of latest trends in patient-oriented research. (abstr.) J Can Health Libr Assoc. 2016;37(2):99. [Google Scholar]
- 16.Association Francophone pour la Savoir (ACFAS) [Accessed June 20, 2018]. http://www.acfas.ca/evenements/congres/programme/85/par-journee.
- 17.Gregor S. The nature of theory in information system. MIS Q. 2006;30:611–42. [Google Scholar]
- 18.Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32:1008–15. doi: 10.1046/j.1365-2648.2000.t01-1-01567.x. [DOI] [PubMed] [Google Scholar]
- 19.Keeney S, Hasson F, McKenna H. The Delphi Technique in Nursing and Health Research. West Sussex, UK: Wiley-Blackwell; 2011. [Google Scholar]
- 20.von der Gracht HA. Consensus measurement in Delphi studies: review and implications for future quality assurance. Technol Forecast Soc Change. 2012;79:1525–36. doi: 10.1016/j.techfore.2012.04.013. [DOI] [Google Scholar]
- 21.Rayens MK, Hahn EJ. Building consensus using the policy Delphi method. Policy Polit Nurs Pract. 2000;1:308–15. doi: 10.1177/152715440000100409. [DOI] [Google Scholar]
- 22.Canadian Institutes of Health Research. SPOR SUPPORT Units. [Accessed August 13, 2018]. Date modified 2018 Jul 23. http://www.cihr-irsc.gc.ca/e/45859.html.
- 23.SPOR Support Unit in Québec. The mission, vision, values and main directions of the unit. [Accessed August 13, 2018]. http://unitesoutiensrapqc.ca/a-propos/#vision.
- 24.Foth T, Efstathiou N, Vanderspank-Wright B, et al. The use of Delphi and nominal group technique in nursing education: a review. Int J Nurs Stud. 2016;60:112–20. doi: 10.1016/j.ijnurstu.2016.04.015. [DOI] [PubMed] [Google Scholar]
- 25.de Meyrick J. The Delphi method and health research. Health Educ. 2003;103:7–16. doi: 10.1108/09654280310459112. [DOI] [Google Scholar]