Abstract
Background
Concerns over the need to improve translational aspects of genetics research studies and engaging community members in the research process have been noted in the literature and raised by patient advocates. In addition to the work done by patient advocacy groups, organizations such as the Patient-Centered Outcomes Research Institute advocate for a change in the culture of research from being researcher-driven to becoming more patient-driven.
Objective
Our project, Autism Genetics and Outcomes (AutGO), consists of two phases. The goal for phase I was to initiate a general discussion around the main topic (i.e., linking genetics and outcomes research). We used the Patient-Centered Outcomes Research Institute engagement approach to: (aim 1) develop a partnership with a wide range of stakeholders to assess their perspective on developing projects that use both genetics and outcomes research data/principles; (aim 2) identify barriers, facilitators, and needs to promote engagement in patient-centered genetics research; and (aim 3) distill and describe actions that may facilitate utilization of patient/parent perspectives in designing genetics research studies.
Methods
In phase I, we formed a community advisory board composed of 33 participants, including outcomes and genetics researchers, clinicians, healthcare providers, patients/family members, and community/industry representatives, and convened six sessions over the 12-month period. We structured the sessions as a combination of online PowerPoint presentations, surveys, and in-person group discussions. During the sessions, we discussed topics pertaining to linking genetics and outcomes research and reviewed relevant materials, including patient stories, research projects, and existing resources.
Results
Two sets of surveys, project evaluations (k = 2) and session evaluations (k = 6), were distributed among participants. Feedback was analyzed using content analysis strategies to identify the themes and subthemes. Herein, we describe: the established partnership (aim 1), the identified barriers, facilitators, and needs (aim 2), as well as the lessons learned and suggested recommendations for the research community (aim 3). Following phase I participants’ recommendation, in phase II, we will focus on a specific disease (i.e., autism); this projected plan is briefly outlined to highlight the overarching goal of the project and its potential significance. We also discuss the study limitations, challenges for conducting this type of multidisciplinary work, as well as potential ways to address them.
Conclusions
The AutGO project has created a unique collaborative forum to facilitate the much needed dialogue between genetics and outcomes researchers, which may contribute to finding ways to improve the translational aspects of genetics research studies.
Electronic supplementary material
The online version of this article (10.1007/s40271-018-0302-z) contains supplementary material, which is available to authorized users.
Key Points
| Currently, there is no systematic platform to gather and process patient/parent perspectives and clinical observations for research use (gap), therefore such critical information is not commonly considered in designing genetics studies. |
| Despite the recent emphasis on personalized medicine, it is not yet clear how genetic information may be used in patient-centered outcomes research, in part owing to the lack of communication between genetics and outcomes researchers. |
| The Autism Genetics and Outcomes (AutGO) project aims to make a conceptual connection between the two disciplines (outcomes and genetics research). We outline how a partnership was established among a wide range of stakeholders and report the findings and recommendations for the research community. |
Introduction
Recently, there has been a major push toward incorporating patients’ voices in research studies with the goal of promoting research that addresses patients’ needs [1–3]. In addition to the work done by patient advocacy groups [4], organizations such as the Patient-Centered Outcomes Research Institute (PCORI) [5, 6] advocate for a change in the culture of research from being researcher-driven to becoming more patient-driven.
There is also an emerging move toward using genetic information in patient healthcare. In particular, the Precision Medicine Initiative [7] aims to take into consideration an individual’s characteristics, including genetic make-up, to personalize treatment [8]. Despite the recent emphasis on precision medicine, it is not yet clear how genetic information may be used in “patient-centered outcomes research” (for simplicity, hereafter it will be referred to as “outcomes research”). One of the main reasons why this promising potential has been overlooked is the lack of communication between genetics and outcomes researchers. This barrier could be addressed by developing a practical engagement protocol to promote synergistic relationships between the two respective disciplines.
Our project, Autism Genetics and Outcomes (AutGO), consists of two phases. The goal for phase I was to initiate a general discussion around the main topic (i.e., linking genetics and outcomes research). The aims were to (1) develop a partnership with a wide range of stakeholders to assess their perspective and interest on creating projects that use both genetics and outcomes research data/principles, (2) identify barriers, facilitators, and needs to promote engagement in patient-centered genetics research, and (3) distill and describe actions that may facilitate utilization of patient/parent perspectives in designing genetics research studies. This paper outlines how a partnership was established and reports the findings (barriers, facilitators, and needs) and recommended tasks distilled from the study participants’ views. Following phase I participants’ recommendations, in phase II, we will focus on a specific disease (i.e., autism); this projected plan is briefly outlined to highlight the overarching goal of the project and its potential significance. Considering the continuous nature of the two phases, we chose AutGO as an overall project title to emphasize the current and future direction of this initiative.
Background
Wealth of Existing Clinical/Genetic Resources
In the USA, PCORI [5] was established in 2010 to generate research evidence to assist patients and providers in making informed health decisions by promoting patient-centered studies (i.e., focusing on questions that matter most to patients). Other international initiatives, such as INVOLVE, the UK-based program, also promote public involvement in health research (http://www.invo.org.uk). Traditionally, research hypotheses/questions have been developed based on scientific facts and/or pilot data generated by researchers. As a result, patient concerns may not directly influence the process. Alternatively, outcomes studies intend to incorporate patient perspectives throughout the study, from the selection of the research questions to the dissemination of the findings [1].
In addition to supporting comparative effectiveness research (CER) and engagement projects, PCORI has also invested in building research infrastructures by launching PCORnet, a National Patient-Centered Clinical Research Network [9, 10]. PCORnet combines electronic health records and patient-generated data (i.e., clinical and research networks led by patients, advocacy organizations, and clinical research partners), and may provide a valuable resource for research purposes, if an opportunity is developed to query the archived data. Recently, several disease-specific networks partnered with PCORnet to use their existing infrastructures to promote patient-centered research. Such partnerships may reinforce using these resources in patient-centered studies. There are also non-PCORI resources, including the eMERGE Network, a National Institutes of Health-organized and funded consortium (http://emerge.mc.vanderbilt.edu/) that combines DNA biorepositories with electronic health records from patients.
In addition to these disease-independent resources, several disease-specific resources have been developed by the research community. Data collections in the existing resources need to be assessed to better understand to what extent they may fulfill the data structure and capacity required for testing patient-centered genetic questions.
Genetic Data and the Potential Application in Outcomes Studies
The role of genetics in patient health outcomes cannot be ignored because: (1) genetic factors contribute to the underlying mechanism of most complex conditions, and (2) it is well documented that drug metabolism and efficacy are regulated by and dependent on a patient’s genetic make-up.
Research approaches using genetic information to address a patient-centered question have been investigated in some disciplines. Oncology and cardiology are two fields where the importance of incorporating patient perspectives in improving medical care has already been recognized, for example, in predicting patient outcomes [11] or guiding treatment decisions [12]. It needs to be explored how experiences gained in these conditions could be applied to develop successful patient outcome predictors for other conditions.
Methods
Ethics Statement
Ethical approval for this study to form an advisory board, conduct focus groups, distribute surveys, and recruit to-be-determined members was granted by the Office of Research Integrity at our institution.
Partnership Formation and Focus Groups
A wide range of stakeholders were either involved from inception or recruited at the beginning of the study. We formed a community advisory board (CAB) composed of 33 participants, including genetics researchers, PCORI awardees working on different conditions, bioinformaticists, physicians, patients/family members, and industry/community representatives. Furthermore, we invited specialists involved in policy making, managing public genetic databases, bioethics, and developing software for managing patient data. See Tables S1 and S1a of the Electronic Supplementary Material (ESM) for more details on participants’ background/expertise.
We convened six sessions over the 12-month period. During the sessions, we discussed topics pertaining to linking genetics and outcomes research, and reviewed relevant materials (i.e., patient stories, research projects, and existing resources). Feedback from CAB members was collected and processed to identify barriers, facilitators, and needs, as illustrated in Fig. 1. More details on methods, research design, and data analysis are provided in Table S2 of the ESM, using the COREQ checklist [13].
Fig. 1.
Schematic of the methods used in phase I. CAB community advisory board, CER comparative effectiveness research, PCORI Patient-Centered Outcomes Research Institute
We structured the sessions as a combination of online PowerPoint presentations, surveys, and in-person group discussions (focus groups) to share educational materials with CAB members and listen and learn from their experiences, preferences, and needs in this context. This structure was suggested by CAB members (when surveyed, 100% of participants approved it). We employed the principles of patient engagement, community-based participatory research (i.e., by equitably involving a wide range of stakeholders in all aspects of our research process, including contributing expertise, sharing decision making, and authorship), and a semi-structured iterative communication process (questionnaires/surveys) throughout all stages of the project.
Two sets of surveys, project evaluations, and session evaluations were distributed using the SurveyMonkey tool to obtain participants’ feedback. See the ESM for the list of questions included in each survey.
Project evaluations (k = 2): The baseline project evaluation survey was conducted at the beginning of the study and assessed the initial level of knowledge, attitudes, and beliefs of the topic. The summative project evaluation survey was conducted 11 months post-baseline to assess the changes in knowledge and overall participants’ satisfaction at the end of the project.
Session evaluations (k = 6): We held six sessions between the two project evaluations. Each session consisted of a PowerPoint presentation distributed by e-mail, followed by a session evaluation survey, and concluded by an in-person group discussion. Session evaluation surveys were conducted to evaluate CAB members’ satisfaction with the session quality and content. Results obtained from each survey were discussed in subsequent in-person group discussions (7–13 CAB members in each). Suggestions were employed to improve the effectiveness and quality of future sessions.
Responses obtained were compared and used as the metrics to measure the study impact and significance. Likert scale and free text responses were used to assess participants’ responses obtained in surveys. A 0–10 point scale was used to record a change of opinion (“0” indicating “not changed at all” and “10” indicating “has changed a great deal”) between the two project evaluations and/or level of agreement with given statements (“0” indicating “strongly disagree” and “10” indicating “strongly agree”) for session evaluations.
Feedback collected via surveys and other communication means (see Fig. 1) was analyzed using content analysis strategies [14]. Three coders (ABE, ZT, and AS) familiarized themselves with the data, and identified and finalized the themes and subthemes. Each coder independently coded the data, then the coders met regularly to discuss and resolve discrepancies and reach consensus. After completion of standard coding and content analysis, we further processed the study findings from the perspectives of community engagement (ABE) and scientists (ZT and AS). As a result, our approaches/input were rather complementary in the further processing. For example, ZT’s focus was tailored toward processing participants’ feedback and finding applicable ways to communicate the identified results to the basic research community. ABE focused on ensuring adequate community engagement throughout the study process and dissemination plans.
Project Website
The project website (http://genetics-outcomes.net/AutGO-PhaseI) was developed under the principal investigator’s supervision with input from study participants. It was used to archive a summary of all communications, meeting minutes, and related documents. Participants were granted access to the website to allow them to review progress and give feedback at their convenience.
Results
Summary: Tasks and Findings
Thirty-three participants took part to assess if/how genetic information may be used in outcomes studies. Eighteen CAB members had been recruited and engaged from inception. ZT, SB, AS, and KS developed the original research plan. ABE and AM were involved in further refinement of the research plan and designing a community outreach protocol. To-be-determined members (n = 15) were identified and recruited by existing members through professional/community/personal contacts. See Table 1 and Table S1 of the ESM for participants’ demographic data and background with respect to contribution to the project. The CAB consisted of the following groups of stakeholders (63.6% female, 84.8% Caucasian): parents/patients 36.4% (n = 12); physicians 27.3% (n = 9); PCORI awardees 18.2% (n = 6); bioinformaticists 12.1% (n = 4); scientists 45.5% (n = 15); and others 21.2% (n = 7).
Table 1.
Demographic data: phase I
| n | % | |
|---|---|---|
| Sex | ||
| Female | 21 | 63.6 |
| Male | 12 | 36.4 |
| Age (years) | ||
| 30–39 | 11 | 33.3 |
| 40–49 | 5 | 15.2 |
| 50–59 | 10 | 30.3 |
| 60–69 | 2 | 6 |
| Skipped question | 5 | 15.2 |
| Race | ||
| White/Caucasian | 28 | 84.8 |
| Black/African American | 3 | 9.1 |
| Asian | 2 | 6.1 |
| Education level | ||
| Bachelor’s degree | 10 | 30.3 |
| Master’s degree | 3 | 9.1 |
| Professional degree | 1 | 3 |
| Doctorate degree | 19 | 57.6 |
In the baseline survey, we asked participants to identify their disease of interest, as shown in Table 2 and Table S1 of the ESM. Corresponding numbers and percentages were calculated separately per each stakeholder category. Some members selected more than one disease category; however, autism was selected as a priority topic by 39.4% of participants.
Table 2.
Disease of interest per stakeholder category: phase I
| Disease of interest | Stakeholder category | Mean (total)* (%) | |||||
|---|---|---|---|---|---|---|---|
| Parents/patients | Physicians | PCORI awardees | Bioinformaticists | Scientists | Others | ||
| 1. Autism | 4 (33.3%) | 1 (11.1%) | 1 (25%) | 8 (53.3%) | 5 (71.4%) | 39.40 | |
| 2. CVD | 4 (33.3%) | 4 (44.4%) | 3 (50%) | 2 (50%) | 3 (20%) | 27.30 | |
| 3. Cancer | 6 (50%) | 3 (33.3%) | 3 (50%) | 1 (25%) | 4 (26.7%) | 1 (14.2%) | 27.30 |
| 4. Other | 27.30 | ||||||
| Details for “Other” disease category | |||||||
| Turner syndrome | 1 (8.3%) | ||||||
| Metabolic disease | 1 (8.3%) | ||||||
| Rehabilitation medicine | 1 (11.1%) | 1 (6.7%) | |||||
| Nephrology | 1 (11.1%) | ||||||
| Infectious Diseases | 1 (11.1%) | 1 (16.7%) | 1 (6.7%) | ||||
| Research Informatics | 1 (25%) | 1 (6.7%) | 1 (14.2%) | ||||
| Sleep disorders | 1 (6.7%) | ||||||
| Genetics | 1 (6.7%) | ||||||
| Bipolar disorder | 1 (14.2%) | ||||||
| ADHD | 1 (14.2%) | ||||||
| Allergies | 1 (14.2%) | ||||||
| Asthma | 1 (14.2%) | ||||||
Numbers and percentages are shown per each stakeholder category
ADHD attention-deficit hyperactivity disorder, CAB community advisory board, CVD cardiovascular disease, PCORI Patient-Centered Outcomes Research Institute
*The average total was calculated based on the total number of CAB members (N = 33). Of note, some CAB members selected more than one disease category
Feedback from participants indicated their high level of interest on the topics covered, willingness to share personal experiences and to learn other stakeholders’ perspective, as well as appreciation for the educational aspect of the project. Topics covered were carefully selected to provide a meaningful overview of the most relevant areas, considering participants’ diverse backgrounds. Successful implementation of this aspect (100% approved by participants) was monitored via surveys, as exemplified in Table 3. See Tables S3 and S4 of the ESM for more details on participants’ assessment of topics covered per stakeholder category.
Table 3.
Participants’ assessment of topics covered: phase I
| Mean (SD*) | |
|---|---|
| Participants’ personal perspectives | |
| Topic | |
| Familiarity with PCOR and CER | 7.9/10 (1.5) |
| Importance of incorporating genetic information into CER | 8.0/10 (0.5) |
| Interest level in using genetic information in CER studies | 7.9/10 (0.6) |
| Participants’ assessment of educational aspect of the project | |
| Approval of the topics covered | 100% |
| Overall clarity of presentations | 8.8/10 (0.4) |
| Topics with the highest ratings | |
| Patient personal stories | 9.3/10 (1.1) |
| Examples of electronic medical record systems | 8.9/10 (1.1) |
| Overview of the ethical issues and genetic testing | 8.8/10 (1.0) |
| Overview of existing resources (PCORnet, eMERGE) | 8.8/10 (1.0) |
| Examples of genetic research done by our CAB members (scientists) | 8.8/10 (1.3) |
| Precision Medicine Initiative | 8.8/10 (1.2) |
Participants’ responses are based on a scale from “0- Strongly disagree” to “10- Strongly agree”
CAB community advisory board, CER comparative effectiveness research, SD standard deviation, PCOR patient-centered outcomes research
*SD equals the mean SD across groups
Questions in the baseline and summative surveys (called Qa and Qb, respectively), assessed knowledge gained by participants pertaining to key topics (PCOR/CER and genetics), are summarized in Table 4. Some participants were already familiar with PCOR/CER concepts prior to being involved with the AutGO project, whereas others had no previous knowledge. Therefore, responses related to the questions Q1a (I am familiar with PCOR and CER) and Q1b (How much has your knowledge about PCOR and CER changed?) were divided into two groups: Group 1 (n = 16) had limited experience with PCOR/CER [average 5.6 (standard deviation [SD] 1.9) for Q1a; on a 0–10 point scale] at baseline, and reported significant knowledge gain [average 7.9 (SD 1.1) for Q1b] at the end. In contrast, Group 2 (n = 17), reported a high level of experience with PCOR/CER at baseline [average 9.4 (SD 0.9) for Q1a]. Nevertheless, they acknowledged a moderate change [average 6.8 (SD 2.1) for Q2b] in their perspective/knowledge about this topic as a result of participation in the study.
Table 4.
Participants’ assessment of two key topics [patient-centered outcomes research/comparative effectiveness research (PCOR/CER) and genetics]: phase I
| Survey topics | Baseline (Qa) | Summative (Qb) | n | ||
|---|---|---|---|---|---|
| Questions | Mean (SD) | Questions | Mean (SD) | ||
| CER/PCOR | Q1a. I am familiar with PCOR and CER | Q1b. How much has your knowledge about PCOR and CER changed? | |||
| Group 1* | 5.6 (1.9) | 7.9 (1.1) | 16 | ||
| Group 2** | 9.4 (0.9) | 6.8 (2.1) | 17 | ||
| Genetics | |||||
| Q2a. I find it important to incorporate genetic information into CER | 8.1 (2.1) | Q2b. How much has your opinion on “the importance of incorporating genetic information into outcomes research” changed? | 8.4 (1.7) | 29 | |
| Q3a. I have a clear understanding of how genetic information may be incorporated in CER | 6.1 (1.8) | Q3b. How much has your understanding of “how genetic information may be incorporated in PCOR and CER” changed? | 8.2 (1.2) | 29 | |
| Q4a. I am interested in using genetic information in CER studies | 8.2 (1.6) | Q4b. How much has your interest in using genetic information in PCOR and CER studies changed? | 8.2 (1.7) | 29 | |
n number of responses, SD standard deviation
*Limited experience with PCOR/CER prior to participation in phase I
**High level of experience with PCOR/CER prior to participation in phase I
Three sets of questions (Q2a,b; Q3a,b; Q4a,b) were used to assess knowledge/interest about using genetic information in outcomes research. Average scores for baseline and summative questions are shown in Table 4. Four respondents who reported a prior high level of knowledge/interest of genetics, interpreted summative assessment questions differently than the rest of the respondents. Because they have already reported a high level of interest (average 10) at the baseline, in the summative assessment, they only acknowledged that their interest level remained the same, instead of reporting changes. Therefore, results for questions related to the use of genetic information (Q2a,b; Q3a,b; Q4a,b) were adjusted by removing these four responses (n = 29 out of 33). Of note, responses to Q3b (How much has your understanding of “how genetic information may be incorporated in PCOR and CER” changed?), which reflects the main objective of the AutGO project, clearly shows an increase [from average 6.1 (SD 1.8) for Q3a to 8.2 (SD 1.2) for Q3b] in the participants’ level of understanding of this topic as a result of participation in the study.
Participant Feedback
The general themes identified by CAB members, reflecting barriers, facilitators, and needs (i.e., for if/how genetic information may be used in outcomes studies), are summarized in Table 5. For example, 16 participants reported difficult subject matter (subtheme 1.1.3). See Table 6 for examples of actual quotes.
Table 5.
Barriers, facilitators, and needs identified based on participants’ perspectives (phase I): themes and subthemes
| Themes | Subthemes | Qty | |
|---|---|---|---|
| 1. Barriers | 1.1. Limited knowledge | 1.1.1. Provider limited knowledge/training | 3 |
| 1.1.2. Patient/community limited knowledge | 9 | ||
| 1.1.3. Difficult subject matter/information too dense | 16 | ||
| 1.2. Ethical issues | 1.2.1. Ethical concerns | 2 | |
| 1.2.2. Stigma | 3 | ||
| 1.3. Logistical issues | 1.3.1. Insufficient patient/parent engagement | 1 | |
| 1.3.2. Insufficient resources/genetic counselors | 1 | ||
| 1.3.3. Technical challenges/EMR/internet access | 4 | ||
| 1.3.4. Logistical challenges | 3 | ||
| 1.4. Other | 1.4.1. Cost/insurance coverage | 1 | |
| 1.4.2. Other | 1 | ||
| 2. Facilitators | 2.1. Knowledge/training | 2.1.1. Knowledge/awareness/information | 8 |
| 2.1.2. Providing training | 1 | ||
| 2.2. Establishing partnership | 2.2.1. Collaborative environment | 4 | |
| 2.2.2. New/future research priorities | 1 | ||
| 2.3. Other | 2.3.1. Improve care/health outcomes | 3 | |
| 2.3.2. Easy/Interactive presentation style /videos | 8 | ||
| 3. Needs | 3.1. Education | 3.1.1. Providing education | 4 |
| 3.1.2. Easy to understand and interactive resources/glossary | 13 | ||
| 3.2. Engagement | 3.2.1. Patient/parent engagement/more overall interaction | 6 | |
| 3.2.2. Social and media outreach | 2 | ||
| 3.2.3. Future research collaboration | 2 | ||
| 3.3. Infrastructure | 3.3.1. Developing Infrastructures (linking clinical and research works) | 1 | |
| 3.4. Other | 3.4.1. Patient-centered | 3 | |
| 3.4.2. General comments about project | 17 | ||
| 3.4.3. Other | 3 |
EMR electronic medical records, Qty number of times a given subtheme was noted in participants’ feedback
Table 6.
Example of stakeholder quotes grouped based on the identified themes/subthemes
| Subthemes* | Stakeholder category**: Example quotes |
|---|---|
| 1.1.2 | Patients/Parents: “More personal interaction would be helpful. I think there is some additional benefit to be gained by interaction between the presenter and audience.” |
| 1.1.2 and 3.1.1 | Scientists/Bioinformaticists: “Patients and families can help this study by sharing a list of questions they have about the condition, their concerns and worries, related to possible prevention or treatment protocols and so on.” |
| 1.1.2 and 3.1.1 | Scientists/Bioinformaticists: “I would consider providing patients/families with the resources that they can learn about the disease condition and related risk factors or in general raising awareness related to genetic diseases and risk factors is as a prerequisite (for participation in such projects).” |
| 1.1.3 and 3.1.2 | PCORI awardees: “Visual representation is often helpful in explaining complex context. Otherwise, poor level of understanding may be intimidating for lay people.” |
| 1.1.2 and 1.1.3 and 3.1.2 | Patients/Parents: “Investigators do not have time to make a conversion of terminology to bring us (patient members) along with them. It will remain a big obstacle until they come up with some systematic approach.” |
| 1.2.1 and 1.2.2 | Scientists/Bioinformaticists: “It would also be great to talk about fear that people have about having EMR, the same as about genetic testing, regarding who exactly will be able to see the data. How much protection the patients have, considering sometimes tricky language in consent forms?” |
| 1.2.1 and 1.2.2 | Physicians/Scientists: “There is often lack of communication and explanations between medical field and patients, which produces misinterpretations and lack of trust.” |
| 1.2.1 and 1.3.4 | Patients/Parents: “I found the genetic research tied to autism of great interest to me both as a parent as well as a provider. We have often discussed the implication of autism like symptoms that exist in our girls with Turner Syndrome (TS). Sleep disturbances are common in our TS population. The discussions (about sleep problems in autism) make me wonder if we are seeing the same subset of problems (in TS).” |
| 1.3.2 | Scientists/Bioinformaticists: “Number of genetic counselors is insufficient. We cannot expect genetic counseling to be done by basic physicians, they are not geneticists.” |
| 1.3.2 | Patients/Parents: “Whose responsibility will it be to review the genetic information gathered about a patient as he/she grows older? Will the patient have to remember that they had genetic information obtained?” |
| 1.3.3 and 1.4.1 | Patients/Parents: “There are challenges, like data being collected based on different platform that makes integration difficult. It brings along additional expenses for hospitals to be able to share patients’ data between health care providers.” |
| 1.4.1 | Physicians: “In clinical practice I frequently have parents who want to do genetic testing but have trouble obtaining prior authorization from insurance companies to cover testing which is often quite expensive.” |
| 2.1.1 | Physicians: “So far the information has been really helpful! I’m learning so much about PCORI and PCORnet. Very exciting!” |
| 2.2.1 | Physicians/Scientists: “I have enjoyed the patient experience formats. It is great to hear a perspective from a fellow scientist who has had a family member affected by genotypic variations.” |
| 2.2.1 | Physicians/Scientists: “Especially loved being able to interact with patients and their families and to hear their opinions.” |
| 2.2.2 and 2.3.1 | PCORI awardees: “The definition and description of PCORnet was extremely helpful in providing a framework for how we researchers could access genetic data and incorporate it in research to improve health outcomes.” |
| 2.3.2 | Scientists/Bioinformaticists: “With regard to having collaborators on this study from different time zones and schedules, I think PowerPoint presentations are very useful and convenient.” |
| 2.1.1 and 3.4.2 | Physicians: “eMERGE seems like it is a great opportunity. I was not aware of this project previously.” |
| 2.1.1 and 3.4.2 | Industry representatives: “I was not aware of the information about drug resistance (and genetic risk factors).” |
| 3.1.1 | Scientists/Bioinformaticists: “Providing educational materials about environmental risk factors.” |
| 3.1.2 | Patients/Parents: “A glossary would be nice to have, or connect to other sources that may have already been developed.” |
| 3.1.2 | Scientists/Bioinformaticists: “Would be good to have a summary slide with the most important features of these initiatives (eMERGE, PCORnet) and how these compare to one and other. The same comment applies to information shared about technologies and health record systems.” |
| 3.2.1 | PCORI awardees: “With respect to future disease specific project, may be useful to take an opportunity to get input beyond this group, from providers in the hospital and the community, patients and researchers.” |
| 3.2.2 | PCORI awardees: “What if we give more visibility to the project website? Can we open it up and make it accessible for the public and make it more interactive to be able to reach out to people doing similar type of work? People could find each other, initiate potential collaborations, collect additional feedback.” |
| 3.2.3 | PCORI awardees: “There are many different hospitals engaged in that network (PCORI funded Greater Plains Collaborative project). Do they include any genetic data? For future, when they have the infrastructure established, it would make sense if they added a genetic component.” |
| 3.4.2 | Scientists/Bioinformaticists: “Having the actual findings from eMERGE phase I and II would be helpful – and/or a list of publications that were a result of the effort. Same comment maybe for PCORnet.” |
CAB community advisory board, PCORI Patient-Centered Outcomes Research Institute, EMR electronic medical records
*Themes/subthemes are listed in Table 5
**Some CAB members represented more than one stakeholder category
Lessons Learned
Based on our experience from working with participants with diverse backgrounds and expectations, we learned that some elements may enhance study participants’ contribution and overall implementation of the results (see Fig. 2). These lessons learned can be taken into consideration for future studies.
Many useful resources containing genetic/clinical data have already been developed, but study participants, including scientific/clinical members, may not be aware of them. Familiarizing stakeholders with such resources would promote the development of studies that use both genetics and outcomes research data/principles.
Setting reasonable expectations for study participants is needed to maintain each member’s meaningful contribution without slowing down the overall research process.
Study participants’ motivation and time devotion are essential in reviewing the provided educational materials.
Reviewing practical research examples would facilitate a better understanding of the topic, particularly, for non-technical study participants.
Visual aids (e.g., videos, webinars, and illustrations) are great educational tools and help with explaining complex contexts.
Hearing patients’ stories help researchers frame the explanatory materials.
Focusing on one disease would be helpful for: developing practical example(s) that link genetics and outcomes research; stimulating interest for stakeholders to participate in a genetics outcomes study; identifying more specific barriers and needs, as well as implementing results among the research community.
Fig. 2.
Study aims and findings in phase I. CAB community advisory board
Suggested Recommendations for the Research Community
The unique composition of our CAB provided us with a rare opportunity to assess the study findings in a complementary manner, from both outcomes and basic research standpoints. Not only did it enable us to use principles of qualitative assessments, but also allowed us to further process the study findings and distill actionable recommendations that would facilitate their implementation. As a result of applying this complementary assessment, we suggest the following recommendations/tasks:
Building Genetics and Outcomes Multidisciplinary Teams
We refer to integrated research approaches that use data and principles of both genetics and outcomes research as GO projects. Unlike other multidisciplinary approaches, building a team for a GO project, in addition to the inclusion of different expertise, requires further considerations. It is essential to maintain patient centeredness throughout the process of GO team building. This notion should be considered in defining study team composition (both technical and non-technical members); furthermore, all members must either have a prior experience with or a willingness to adopt a patient-centered approach; i.e., be vigilant about the overall goal (improving patient outcomes, NOT identifying disease causality). Owing to the multidisciplinary nature of GO projects, in addition to patient involvement, they also imply a bi-directional approach: developed by incorporating genetic information into outcomes studies and/or bringing the outcomes research approach into genetics studies.
Raising Awareness in the Research Community
Despite a growing interest among different groups of stakeholders, it appears that the research community is not fully aware of the need to promote developing GO projects. To stimulate interest towards developing patient-centered genetics research projects, awareness needs to be raised among the research community about GO studies, and patient-powered resources such as PCORnet, as well as initiatives such as the AutGO project. This would encourage the research community to consider developing projects that prioritize translational aspects of the study over finding genetic causes.
Developing Effective Educational Protocols
A combination of online courses, website postings, and in-person meetings/workshops can be used to design an educational protocol with the aim of motivating the research community to consider developing GO projects.
Developing Disease-Specific Examples
We identified several key issues on the general topic of linking genetics and outcomes research. To build a more detailed engagement and implementation plan, disease-specific projects need to be developed. Focusing on a given disease will enable the implementation of lessons learned from such engagement activities to establish research partnerships with the relevant stakeholders. Step-by-step instructions for a practical workflow and potential gaps (i.e., patient/community engagement, hypothesis development, data retrieval from genetic/phenotypic databases, and statistical analysis) pertaining to each condition could be crafted by developing disease-specific research examples.
Assessing Existing Resources
To identify to what extent patient-centered questions could be addressed using the genetic/clinical data archived in the existing resources (e.g., PCORnet and eMERGE), they need to be assessed by a GO team. One potential technical pitfall for conducting queries on multiple resources (i.e., de-identified electronic health records, and/or disease-specific databases) is the incompatibility of their informatics platforms. Addressing this technical obstacle could facilitate such assessments and stimulate more effective engagement and contribution of participants, especially non-technical members, throughout the study process.
Future Directions
Development of an Autism-Specific Educational Protocol (Phase II)
In phase II, we will apply lessons learned from phase I to develop a disease-specific educational protocol. Autism was selected as the disease of interest based on the following: (1) this condition has a strong genetic basis, and developing GO studies may have a profound impact on improving health-related decisions for this patient population, (2) the majority of phase I members (39.4%) identified autism as a disease of interest (as a parent, healthcare provider, scientist, or industry representative), and (3) a rapidly growing amount of genetic information has been generated from subjects with autism. Furthermore, mental health is among PCORI’s top priority conditions, resulting in a major investment in autism in outcomes research, including the recent partnership of PCORnet with two autism-related networks, Interactive Autism Network and Phelan–McDermid syndrome.
To develop a practical autism-specific protocol, first, a GO team has to be built to develop a hypothesis for autism that fits with both outcomes and genetics research. One area where patient engagement could be crucial is the identification of environmental risk factors, which may help develop a research hypothesis related to gene–environment interactions, a largely unknown area of research that also carries a considerable potential to identify meaningful outcomes for improving patient health. Another avenue for selecting research priorities to be assessed by a GO team would be via findings from other studies on this topic, conducted with input from patients with autism.
Subsequently, the genetic/phenotypic data archived in the existing autism-specific and non-autism resources need to be evaluated to establish to what extent they can be used to assess the identified patient-centered research hypothesis, as well as relevant limitations. The success and impact of new initiatives such as the AutGO project depend on the involvement of a larger autism research community. One effective method of raising awareness would be through organizing educational workshops around this topic, particularly, at the scientific meetings such as the International Society for Autism Research.
A lack of translational aspects and an often inadequate inclusion of patient concerns/priorities in the work process is a recognized gap in many conditions, including autism. Several autism genetic initiatives have been formed with the overall goal of identifying causality, leading to important gene discoveries. However, the overarching goal of the AutGO initiative is on developing genetics research questions aimed at addressing outcomes, NOT finding genetic causes. This may sound a laudable goal, but by setting patient centeredness and outcomes, NOT causality, as the main goal, we feel the AutGO project may bring more attention to this critical existing gap and serve as an effort complementary to other autism genetic initiatives.
Limitations
Owing to the time constraints and budget restrictions, we gave priority to recruiting local participants. As a result, several participants were our institution employees (i.e., clinicians and parent representatives). Therefore, our findings/recommendations may not be viewed as a comprehensive and uniform representation of the local community perspectives, particularly with respect to minority groups and hard-to-reach participants. This limitation could be addressed by informing and engaging a wider community, which could be facilitated through this publication. During survey analysis, we noticed that using more specific rating scales in certain questions would have reduced potential confusion for interpretation (e.g., anchor the 0–10 ratings as “increased” rather than “changed” because the latter does not imply a direction).
A potential challenge to this as well as other patient and community engagement projects is the real or perceived imbalance of power between study participants, particularly, technical and non-technical members. We prepared a memorandum of agreement that outlined the roles and responsibilities of each member to make communication more open, honest, and transparent. Providing training and ongoing follow-up support may be beneficial to further ensure inclusive involvement of all participants, particularly more dynamic engagement of patients/family members. Another potential challenge is to ensure patient centeredness in GO studies. This aspect cannot be reached without tangible patient engagement throughout the study. For our project, patient engagement was evaluated by the funding agency through a strict monitoring process including regular/monthly reports, which facilitated addressing this challenge. Another way to reinforce this aspect would be through involvement of outcomes experts/PCORI investigators experienced in conducting patient-centered studies, when forming GO teams. Other areas for further investigation, which are beyond the scope of our work, include: ethical issues related to using genetic information for improving patient outcomes and effective communication methods for multidisciplinary work.
Conclusions
The principles of conducting outcomes studies, such as patient involvement in every aspect of the process, have been well defined elsewhere [3, 15] and are intrinsic elements of the AutGO project. However, a lack of communication between genetics and outcomes researchers is a unique gap pertaining to conducting GO projects; therefore, to address this gap, we are targeting the research community as a main audience for this paper. To make a connection between genetic information and patient-centered studies, in addition to engaging patients in the process, the research community, particularly, the genetics research community needs to: (1) be aware of the communication gap and (2) dynamically participate in relevant dialogues to become an active partner in designing/conducting GO projects. One effective method of stimulating interest in developing this type of unique collaborative effort is to educate stakeholders about why and how team members may work in synergy on developing GO projects. This process may involve (1) developing disease-specific educational protocols that integrate relevant concepts (e.g., patient centeredness and translational elements) and demonstrate their applicability at different stages of the study process, (2) identifying potential connections between outcomes research and genetic information, and (3) assessing the potential applicability of the existing phenotypic/genotypic data in this context. Our motivation in developing the AutGO initiative is to create a hybrid concept connecting the outcomes and genetics disciplines to demonstrate the potential and benefits of interplay between them, which may facilitate dialogue for finding practical ways to take into consideration patients/parents perspectives when designing GO studies.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Acknowledgements
The “PCORI EAIN-2419 Working Group” participants (in alphabetical order): Seth Bittker, Patient/parent representative, Darien, CT, USA; Jeff Blackwood, Pathfinder Health Innovations, Kansas City, MO, USA; Jamie Bolen, Children’s Mercy Hospital, Kansas City, MO, USA; Andrea Bradley-Ewing, Children’s Mercy Hospital, Kansas City, MO, USA; Amy Brower, American College of Medical Genetics, Bethesda, MD, USA; Mark Bryant, Patient/parent representative, Kansas City, MO, USA; Sheryl Chadwick, Children’s Mercy Hospital, Kansas City, MO, USA; Broderick Crawford, Faith community representative, Kansas City, KS, USA; Emily Farrow, Children’s Mercy Hospital, Kansas City, MO, USA and Kansas City School of Medicine, University of Missouri, Kansas City, MO, USA; Laura Fitzmaurice, Children’s Mercy Hospital, Kansas City, MO, USA and Kansas City School of Medicine, University of Missouri, Kansas City, MO, USA; Mary Anne Hammond, Children’s Mercy Hospital, Kansas City, MO, USA; Mark Hoffman, Children’s Mercy Hospital, Kansas City, MO, USA and Kansas City School of Medicine, University of Missouri, Kansas City, MO, USA; Valerie Hu, George Washington University, Washington, DC, USA; Monirul Islam, University of Nebraska Medical Center, Omaha, NE, USA; Mary Kinart, Children’s Mercy Hospital, Kansas City, MO, USA; Angie Knackstedt, Children’s Mercy Hospital, Kansas City, MO, USA; John Lantos, Children’s Mercy Hospital, Kansas City, MO, USA and Kansas City School of Medicine, University of Missouri, Kansas City, MO, USA; Paul Law, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA; Matthew Mclaughlin, Children’s Mercy Hospital, Kansas City, MO, USA and Kansas City School of Medicine, University of Missouri, Kansas City, MO, USA; James McClay, University of Nebraska Medical Center, Omaha, NE, USA; DeeJo Miller, Children’s Mercy Hospital, Kansas City, MO, USA; Angela Myers, Children’s Mercy Hospital, Kansas City, MO, USA and Kansas City School of Medicine, University of Missouri, Kansas City, MO, USA; Ginger Nicol, Washington University in St. Louis, St. Louis, MO, USA; Kelly Ranallo, Patient/parent representative, Overland Park, KS, USA; Tayebeh Rezaie, National Center for Biotechnology Information, National Institutes of Health, Bethesda, MD, USA; Leon Rozenblit, Prometheus Research, LLC, New Haven, CT, USA; Hossein Saiedian, University of Kansas, Lawrence, KS, USA; Kim Smolderen, Kansas City School of Medicine, University of Missouri, Kansas City, MO, USA; John Spertus, Kansas City School of Medicine, University of Missouri, Kansas City, MO, USA; Olivia Veatch, University of Pennsylvania, Philadelphia, PA, USA; Darcy Weidemann, Children’s Mercy Hospital, Kansas City, MO, USA and Kansas City School of Medicine, University of Missouri, Kansas City, MO, USA; Kristen Worden, Patient/parent representative, Overland Park, KS, USA.
Authors Contributions
ZT, SB, AS, and KS developed the original research plan. ABE and AM were involved in further refinement of the research plan and designing a community outreach protocol. ZT and AS drafted the initial manuscript. Other authors (listed as the PCORI EAIN-2419 Working Group) contributed to the conceptualization, writing, revising, and final approval of the manuscript. HS provided technical support (project website developer). See Table S1 of the Electronic Supplementary Material for the full names of the authors.
Funding
This study was funded by the Patient-Centered Outcomes Research Institute (Contract numbers: EAIN-2419 and EAIN-3885). All statements in this report, including its findings and conclusions, are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute, its Board of Governors, or Methodology Committee.
Compliance with Ethical Standards
Conflict of interest
Zohreh Talebizadeh, Ayten Shah, and the PCORI EAIN-2419 Working Group participants have no conflicts of interest directly relevant to the content of this study.
Data Availability Statement
All data generated or analyzed during this study are included in this published article, the Electronic Supplementary Material, as well as on the project website (http://genetics-outcomes.net/AutGO-PhaseI).
Footnotes
The members of the PCORI EAIN-2419 Working Group are listed in the Acknowledgements section.
Electronic supplementary material
The online version of this article (10.1007/s40271-018-0302-z) contains supplementary material, which is available to authorized users.
Contributor Information
Zohreh Talebizadeh, Phone: 816-983-6506, Email: ztalebi@cmh.edu.
PCORI EAIN-2419 Working Group:
Seth Bittker, Jeff Blackwood, Jamie Bolen, Andrea Bradley-Ewing, Amy Brower, Mark Bryant, Sheryl Chadwick, Broderick Crawford, Emily Farrow, Laura Fitzmaurice, Mary Anne Hammond, Mark Hoffman, Valerie Hu, Monirul Islam, Mary Kinart, Angie Knackstedt, John Lantos, Paul Law, Matthew Mclaughlin, James McClay, DeeJo Miller, Angela Myers, Ginger Nicol, Kelly Ranallo, Tayebeh Rezaie, Leon Rozenblit, Hossein Saiedian, Kim Smolderen, John Spertus, Olivia Veatch, Darcy Weidemann, and Kristen Worden
References
- 1.Anderson M, McCleary KK. From passengers to co-pilots: patient roles expand. Sci Transl Med. 2015;7(291):291fs25. doi: 10.1126/scitranslmed.aac6023. [DOI] [PubMed] [Google Scholar]
- 2.Chiauzzi E, DasMahapatra P, Cochin E, et al. Factors in patient empowerment: a survey of an online patient research network. Patient. 2016;9(6):511–523. doi: 10.1007/s40271-016-0171-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Holm KE, Casaburi R, Cerreta S, et al. Patient involvement in the design of a patient-centered clinical trial to promote adherence to supplemental oxygen therapy in COPD. Patient. 2016;9(3):271–279. doi: 10.1007/s40271-015-0150-z. [DOI] [PubMed] [Google Scholar]
- 4.Wong-Rieger D. Moving from patient advocacy to partnership: a long and bumpy road. Patient. 2017;10(3):271–276. doi: 10.1007/s40271-017-0216-1. [DOI] [PubMed] [Google Scholar]
- 5.Selby JV, Lipstein SH. PCORI at 3 years: progress, lessons, and plans. N Engl J Med. 2014;370(7):592–595. doi: 10.1056/NEJMp1313061. [DOI] [PubMed] [Google Scholar]
- 6.Selby JV, Forsythe L. HC Sox HC. Stakeholder-driven comparative effectiveness research: an update from PCORI. JAMA. 2015;314(21):2235–2236. doi: 10.1001/jama.2015.15139. [DOI] [PubMed] [Google Scholar]
- 7.Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372(9):793–795. doi: 10.1056/NEJMp1500523. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Buchanan J, Wordsworth S, Schuh A. Patients’ preferences for genomic diagnostic testing in chronic lymphocytic leukaemia: a discrete choice experiment. Patient. 2016;9(6):525–536. doi: 10.1007/s40271-016-0172-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Fleurence RL, Curtis LH, Califf RM, et al. Launching PCORnet, a national patient-centered clinical research network. J Am Med Inform Assoc. 2014;21(4):578–582. doi: 10.1136/amiajnl-2014-002747. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Corley DA, Feigelson HS, Lieu TA, et al. Building data infrastructure to evaluate and improve quality: PCORnet. J Oncol Pract. 2015;11(3):204–206. doi: 10.1200/JOP.2014.003194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Kullo IJ, Jouni H, Austin EE, et al. Incorporating a genetic risk score into coronary heart disease risk estimates: effect on low-density lipoprotein cholesterol levels (the MI-GENES clinical trial) Circulation. 2016;133(12):1181–1188. doi: 10.1161/CIRCULATIONAHA.115.020109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Cardoso F, van’t Veer LJ, Bogaerts J, et al. 70-Gene signature as an aid to treatment decisions in early-stage breast cancer. N Engl J Med. 2016;375(8):717–729. doi: 10.1056/NEJMoa1602253. [DOI] [PubMed] [Google Scholar]
- 13.Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–357. doi: 10.1093/intqhc/mzm042. [DOI] [PubMed] [Google Scholar]
- 14.Morgan DL. Qualitative content analysis: a guide to paths not taken. Qual Health Res. 1993;3(1):112–121. doi: 10.1177/104973239300300107. [DOI] [PubMed] [Google Scholar]
- 15.Frank L, Basch E, Selby JV, et al. The PCORI perspective on patient-centered outcomes research. JAMA. 2014;312(15):1513–1514. doi: 10.1001/jama.2014.11100. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
All data generated or analyzed during this study are included in this published article, the Electronic Supplementary Material, as well as on the project website (http://genetics-outcomes.net/AutGO-PhaseI).