Skip to main content
JAMIA Open logoLink to JAMIA Open
. 2019 Dec 3;3(1):2–8. doi: 10.1093/jamiaopen/ooz060

Citizen science to further precision medicine: from vision to implementation

Carolyn Petersen 1,, Robin R Austin 2,, Uba Backonja 3,4, Hugo Campos 5, Arlene E Chung 6, Eric B Hekler 7, Pei-Yun S Hsueh 8, Katherine K Kim 9, Anthony Pho 10, Liz Salmi 11, Anthony Solomonides 12, Rupa S Valdez 13
PMCID: PMC7309265  PMID: 32607481

Abstract

The active involvement of citizen scientists in setting research agendas, partnering with academic investigators to conduct research, analyzing and disseminating results, and implementing learnings from research can improve both processes and outcomes. Adopting a citizen science approach to the practice of precision medicine in clinical care and research will require healthcare providers, researchers, and institutions to address a number of technical, organizational, and citizen scientist collaboration issues. Some changes can be made with relative ease, while others will necessitate cultural shifts, redistribution of power, recommitment to shared goals, and improved communication. This perspective, based on a workshop held at the 2018 AMIA Annual Symposium, identifies current barriers and needed changes to facilitate broad adoption of a citizen science-based approach in healthcare.

Keywords: research methodology, healthcare systems, community participation, consumer involvement, patient acceptance of healthcare, citizen science

INTRODUCTION

Precision medicine has been defined as the development of prevention and treatment strategies that take individual variability into account.1 To date, precision medicine research efforts (eg, All of Us Research Program2) have emphasized largely expert-driven efforts for better identifying different interventions that work for different people at different times. A central promise of precision medicine is prevention and treatment approaches that move beyond “on average” patient responses3 to create more personalized, targeted interventions.4–6 As evidenced by the extended discussion about various “precision” efforts,4–16how to realize the vision of precision medicine remains an open question. A key consideration is how to include patients, citizens, patient innovators, patient informaticians, citizen scientists, and scientific citizens in the process and how informatics infrastructure can incorporate these contributions meaningfully.

Citizen science has been defined as “the general public engagement in scientific research activities when citizens actively contribute to science either with their intellectual effort or surrounding knowledge or their tools and resources.”17 Citizen science initiatives have contributed to academic science and continue to broaden its scope and depth.18 Although much of the work is conducted outside the walls of academia or other organizational settings via novel methods, some patients who take on the role of citizen scientists do use traditional research and/or informatics approaches to answer their questions.19

Citizen science can empower individuals to generate scientific questions and share their data. Citizens can generate answers to population health questions of interest to both patients and the healthcare system,20,21 making citizen science a feasible approach to healthcare research.22–25 Citizen science offers an opportunity to empower marginalized groups, such as sexual and gender minorities, to shape scientific inquiry through participation.26,27 Engagement of citizen scientists enables studies that aren’t accessible without them. Technological advances, changing reimbursement models, innovative informed consent approaches, and other factors are driving a shift in power dynamics within healthcare, affording greater integration of citizen scientists’ work into research and clinical care.28–32

Citizen science has the potential to extend and enhance the practice of precision medicine if healthcare practitioners and citizen scientists work together to develop scientifically sound approaches.33,34 For example, the Personal Genome Project UK uses a citizen science approach in which participants have consented to open access release of personal genome, methylome, and transcriptome data and analyses thought to influence gene function.35 However, the general movement toward citizen science is more mature in fields other than medicine, and this situation creates an opportunity for informaticians to think through what citizen science means—and could mean—in the informatics context. Within fields such as biology, conservation, ecology, and astronomy, previously unimaginable insights into topics such as migration patterns, scope of biodiversity, and asteroid surveillance have been realized through citizen science-based approaches.36 In comparison, citizen science-based efforts within the field of population health, clinical medicine, and consumer health are relatively nascent.21,37–40 This viewpoint delineates the minimum technical, organizational, and citizen engagement requirements needed to facilitate meaningful integration of citizen science into existing healthcare systems. The following implementation recommendations were identified during an expert workshop held at the 2018 AMIA Annual Symposium.

TECHNICAL ISSUES

To support a diverse, open, integrated citizen science ecosystem, several technical issues must be addressed. Though some considerations are specific to citizen science, others relate more broadly to the healthcare environment’s increasingly person-centric approach.41

Platforms that facilitate collaboration among patients and researchers

To accelerate participation, it is critical to have easily accessible, usable technology platforms that facilitate idea generation, prioritization of research questions, community-building between citizens and researchers, and results dissemination. Research networks have demonstrated effective approaches to this challenge although these projects relied on one-off technical tools.42–44 Crowdfunding platforms have emerged as examples of how individuals, scientists, and advocates can reach out to the public for donations and investments.45 Although participants have primarily used these platforms for outreach purposes, these platforms have not enabled collaborative generation of science. Hence, an open-source approach that citizen-scientist communities could leverage and build upon could accelerate patient and researcher matching.

A distributed network

Citizen science at scale requires a network facilitating movement of heterogeneous data sources into a large-scale system in real time. Such a centralized system, however, further necessitates creating a governing body which is still able to preserve and promote citizen scientists’ autonomy within such a framework. In addition, such a network must facilitate compliance with local and national privacy regulations, among others. The growth in infrastructure that facilitates data sharing will support greater implementation of citizen science,46 so resolution of governance and related infrastructure issues is paramount. Platforms designed specifically for citizen science projects support this objective.47 Advancing ethical structures to achieve safety, as used in some patient-led efforts such as the Connected and Open Research Exchange,48 is critical.49,50

Functionality for tracing provenance of data contributions from individual data sources

The ability to potentially track the provenance of common types of data from individuals and organizations is recommended so investigators can explain underlying assumptions and potential sources of bias in studies. These metadata should be collected only when individuals wish to have identifiable data elements shared. Ideally, such functionality would support the remuneration of individuals in exchange for sharing their data. Of course, when participants desire to make personal data open access, as in PGP-UK,35 system functionality should support this approach as well, provided that the shared personal data fall under regulation.

Functionality supporting citizen scientist access to their own health information

Tools that give citizen scientists access to their complete health information (eg, clinical notes, lab results, and radiology reports) are central to advancement of the field. This requirement is important because people who read medical notes report having a better understanding of their conditions, are more likely to take medications as prescribed, and report greater satisfaction and trust of their doctors.51,52 Similarly, access to diverse types of data from electronic health records will allow people to provision access and sharing of their personal data. It is important to acknowledge that individuals may have a range of motivations that lead to their participation as citizen scientists. It will be essential for those engaged in supporting and promoting citizen science efforts in precision medicine to understand and be responsive to this range of motivations, which may span from an interest in improving individual health to contributing to a larger social movement. Technology that provides citizen scientists some control over the management and sharing of data they contributed will also incentivize research engagement.53

A standardized, comprehensive approach to identification and resolution of barriers to data access

Citizen scientists need a standardized process for identifying and addressing local, state, and federal policies that hinder or obstruct access to personal health data. This article was formulated in the context of the US-based healthcare system. Within this context, patients are increasingly expected to assume responsibilities related to managing their own health. Proactive management of personal health benefits from access to data from health providers is currently difficult to obtain due to restrictive policies. Although we expect citizen scientists to play a key role in advocating for revised policies, it will be essential to create structures for sharing the burden of change, particularly in under-resourced settings. With such a process in place, they could then identify ways to facilitate patients’ access to their health data. For example, they could negotiate an understanding among competing health systems to adopt a data access policy providing equivalent access for all patients, thereby reducing the impact of differences in how health systems manage and share data. Such understandings will be critical in the future as greater quantities of genomic data become available and both traditional researchers and citizen scientists seek to use multiple datasets to answer questions of interest.

ORGANIZATIONAL ISSUES

Traditional research institutions must address a number of organizational issues to work effectively with citizen scientists.

Need to honor agency in individuals, families, and communities

For citizen science work to thrive, organizations should avoid treating engaged patients as the exception or as token participants in a provider-centric system. Acknowledging the contributions of the individual, family, and community implies respect and an understanding that everyone comes to the table with different abilities and perspectives. The ideal of honoring agency strives to reduce the authoritative assumption that individuals, families, or communities need to become “engaged,” when in reality they already are, though perhaps outside the priorities of healthcare institutions. It includes recognition of the many forms of expertise that each person (including patients) possesses that facilitate progress toward healthier living, the need to honor the validity of different perspectives,54 and the need to build empathy and acknowledgement of power differentials as a potentially hidden barrier to equitable participation, contribution, and benefit.55

Commitment to ethical behavior

Organizations must develop an ethical framework rooted in community values on which to base citizen science efforts.56 This framework should inform a broad spectrum of issues, including but not limited to privacy, security, data ownership, informed consent, consideration of potential unintended consequences of data sharing, and patient and researcher access. Such a framework would inform how a local citizen science community might create a collaborative, less hierarchical culture based on mutual respect (rather than power) and ethical behavior by all that rewards curiosity and fosters trustworthiness. Substantive work has already been undertaken in this area, as evidenced by the European Citizen Science Association’s “Ten Principles of Citizen Science,” which (among other things) emphasize mutual benefit to professional and citizen scientists, availability of project data and metadata to the public, and acknowledgement of citizen scientists in results and publications.57 Research questions that arise from citizen scientists, rather than researchers, in a community-based participatory research environment, offer another way to honor the commitment to ethical behavior.

New and expanded funding models

Because citizen science is participant-driven, it often lacks access to the traditional sources of funding (eg, government research awards, foundation grants) that support research in universities and academic medical centers. Within the United States, funding institutions (eg, the Robert Wood Johnson Foundation, the National Institutes of Health) have demonstrated growing interests in models of citizen science and those that are closely related, such as community-based participatory research. These funding opportunities, however, follow conventional models of time constraints. As such, while there are growing opportunities to initiate citizen-science-based research efforts, questions of sustainability remain unanswered. Funding structures for citizen science, including financial models that explore contribution of data where participants have substantial involvement, are needed. Patient-powered research networks may be such a model, although their sustainability too remains unproven.

New research models

Partnering institutions must embrace a variety of research models. Health- and self-related citizen science does not necessarily imply a shared experience, but technology can support both scientific endeavors and varied forms of sharing and network-building. Many technology adaptations have been developed as part of the Quantified Self movement (eg, information retrieval skills for evidence gathering, measurement device development, provider alliances based on shared decision-making),58 and these and other approaches need to be developed and implemented more broadly.59 The use of self-monitoring technologies to create persuasive performance feedback that motivates technology users to continue practicing healthy behaviors for further health improvement is one such new model.60 Crowdfunding and crowdsourcing approaches have proven successful in microbiome and metagenomics research,61 and may offer opportunities for researcher-citizen scientist partnerships. Actively engaging patients and caregivers to prevent medical errors has resulted in new technology designs and improved communication between clinicians and hospitalized patients.62,63 Advancing an “agile scientific” model64 that focuses first on helping individuals rather than on producing generalizable knowledge aligns with organizational structures such as learning healthcare systems, but with clearer integration of citizen scientists.65

Authoritative research and engagement guidelines

Patient-researcher partnerships in citizen science range from easily identifiable relationship models (eg, mentorships, co-mentorships, and patients as co-investigators) to nontraditional/creative solutions (eg, patients as principal investigator, patients working independently). Citizen science efforts follow a continuum from researcher-initiated to citizen scientist-initiated, with truly collaborative models residing in the middle of this spectrum. Each participation model leads to different requirements, roles, and expectations for the individuals involved in the effort. All parties must seek to understand how these new models fit within existing academic hierarchies and promote successful collaboration. The growing community of researchers with academic affiliations and research-minded patients and patients doing citizen science requires mutually accepted guidelines for engagement and practice.

Broad outreach to allied groups for shared learning and advancement

The current practice of citizen science, whether consciously described as “citizen science,” as patient-clinician engagement,66 or otherwise, has evolved in many areas without reference to each other. Although groups and programs have shared goals, needs, and challenges, they often lack awareness of and access to each other, and so miss opportunities for shared learning and development. Citizen scientists who receive healthcare in the United States have a very different experience than their counterparts in other regions where universal healthcare, minimal copayment requirements, and relatively smaller health disparities are the norm, and as such, their practice of citizen science naturally has evolved differently. US-based citizen scientists in healthcare may learn from colleagues practicing citizen science elsewhere, wholesale adoption of and alignment with other national models is neither likely nor necessarily beneficial. Citizen science practiced in the United States, in particular, needs infrastructure that facilitates communication and information sharing among organizations, programs, citizen scientists working in other regions, and potential partners (e.g., funders, government agencies).

COLLABORATION ISSUES

In addition to technical and organizational issues, several collaboration barriers must be broken down.

Failure to see patients as collaborators

Many healthcare institutions lack a culture of working with citizen scientists and may require deconstruction of a top-down culture that locates authority in medical professionals. A foundation for culture change arises through recognition of the value of patient-clinician partnerships;67 healthcare organizations must consciously work toward this goal. The historical failure to recognize citizen scientists as qualified interpreters of human genomes and related material must be addressed going forward.68

Targeted approaches for bringing patients into research efforts

Because patients may have little background in science, they may need training in research methodology, ethics, and regulation. Training has already been proven feasible and productive.69,70 When participants can contribute and prioritize research questions, and then see the progression from idea to research study to results, their interest and participation are reinforced.71 Planning research efforts of limited duration may facilitate recruitment of patient-researchers in that minimally ambitious efforts require less time of patients who may already be juggling health needs, job and family responsibilities, and other commitments.72 Similarly, to ensure a truly reciprocal effort, researchers and clinicians should undertake training to develop skills in engaging and collaborating with patients. Movement toward an effective model of citizen science will require adopting principles of mutual respect and learning, in which both patients and researchers are trained in ways to engage constructively with one another. In addition to efforts initiated by researchers to engage citizen scientists, it will be essential to create mechanisms that facilitate citizen scientist’s ability to find and reach out to research partners.

Improved ability to find and include patient partners

Even when researchers wish to co-manage citizen science projects, they may experience difficulty finding individuals with the necessary background, interest, and time. Employing a participatory design process led by a citizen governance council and the research team has proven successful for generating participation and engagement guidelines and for ways to encourage participation throughout the entire research lifecycle in a patient-powered research network.69 For example, Citizen Endo project participants self-track endometriosis symptoms, an activity that researchers are studying as a way to phenotype the condition,73,74 and the “Make the Breast Pump Not Suck” project recruited nursing mothers to help redesign breast pumps.75,76 Other approaches that operate outside traditional foundation and academic structures also are needed, in particular because although citizen scientists may actively seek clinical partners, they typically experience difficulty finding them.

Openness to meeting citizen scientists at varying skill levels

Not all participants come to citizen science with the same set of skills or background. Each situation presents a unique set of challenges that motivate individuals to explore new solutions. Academics learn their trades through a prescribed curriculum, but citizen scientists in healthcare learn out of the need to survive or improve quality of life, sheer curiosity, or a combination of the two. Healthcare organizations and researchers need to build capacity for engaging citizen scientists with variable skill sets and to help them to grow into increasing responsibility within teams as the partnership matures. Large-scale efforts that engage citizen scientists with minimal formal training, such as the American Gut Project, in which self-selected citizen scientists mailed samples through the mail at room temperature, have proven feasible and efficacious.77,78

Trust-building as an essential target and skillset

Although citizen scientists in health are keen to contribute meaningfully to research, they often have had negative and/or unhelpful experiences in healthcare and research environments, which may be particularly acute among marginalized populations.79–87 These experiences may have left them mistrustful of providers, researchers, and others working within health systems and academia. Citizen science will be advanced more quickly and effectively when those in healthcare focus on gauging readiness to participate at various points of the research pipeline and establishing trust with and among patients doing citizen science.88,89 Progress will advance more quickly when researchers use tools and strategies designed to build trust, such as more comprehensive approaches to informed consent that both inform and engage citizen scientists.35,90

CONCLUSION

Collaboration of citizen scientists and professional scientists supports the goals of knowledge generation and support for action. Citizens not only assist scientists, but also work across the spectrum of informatics: data (fact/observation collection), information (data analysis), knowledge (information refinement), and wisdom (knowledge generation for the greater good). Such collaboration in precision medicine is necessary to fully capture the meaningful differences that indicate which intervention to use, for whom, and in what context. Healthcare organizations can, and should, integrate citizen scientists into infrastructures that support evidence generation both for and with citizens. Citizen science has extraordinary potential to extend and enhance the practice of precision medicine if certain technical, organizational, and citizen engagement issues are addressed appropriately.

FUNDING

The authors have no funding sources to declare.

AUTHOR CONTRIBUTIONS

CP, RA, UB, HC, AEC, EBH, P-YSH, KKK, AP, LS, AS, and RSV developed the ideas in this manuscript, wrote and edited the text, and revised the manuscript.

ETHICS STATEMENT

No humans or human materials were used in the production of this work, and no Institutional Review Board approvals were required.

ACKNOWLEDGMENT

The authors gratefully acknowledge Dr. Jennifer Couch for her presentation during the workshop. The authors would like to acknowledge and thank Dr. Don E. Detmer for championing the importance of citizen science within AMIA and for convening many of us interested in this novel area of informatics.

CONFLICT OF INTEREST

The authors have no competing interests.

REFERENCES

  • 1. Collins FS, Varmus H.. A new initiative on precision medicine. N Engl J Med 2015; 372 (9): 793–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.National Institutes of Health. About the All of us Research Program. Washington, DC: National Institutes of Health; 2017. [Google Scholar]
  • 3. Fradkin JE, Hanlon MC, Rodgers GP.. NIH precision medicine initiative: implications for diabetes research. Dia Care 2016; 39 (7): 1080–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Arnett DK, Claas SA.. Precision medicine, genomics, and public health. Dia Care 2016; 39 (11): 1870–3. [DOI] [PubMed] [Google Scholar]
  • 5. Vaithinathan AG, Asokan V.. Public health and precision medicine share a goal. J Evid Based Med 2017; 10 (2): 76–80. [DOI] [PubMed] [Google Scholar]
  • 6. Yanovski SZ, Yanovski JA.. Toward precision approaches for the prevention and treatment of obesity. JAMA 2018; 319 (3): 223–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Akdis CA, Ballas ZK.. Precision medicine and precision health: building blocks to foster a revolutionary health care model. J Allergy Clin Immunol 2016; 137 (5): 1359–61. [DOI] [PubMed] [Google Scholar]
  • 8. Chambers DA, Feero WG, Khoury MJ.. Convergence of implementation science, precision medicine, and the learning health care system: a new model for biomedical research. JAMA 2016; 315 (18): 1941–2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Dolley S. Big data’s role in precision public health. Front Public Health 2018; 6: 68.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Dorsey SG, Resnick BM, Renn CL.. Precision health: use of omics to optimize self-management of chronic pain in aging. Res Gerontol Nurs 2018; 11 (1): 7–13. [DOI] [PubMed] [Google Scholar]
  • 11. Gambhir SS, Ge TJ, Vermesh O, et al. Toward achieving precision health. Sci Transl Med 2018; 10 (430). pii: eaao3612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Gillman MW, Hammond RA.. Precision treatment and precision prevention: integrating below and above the skin. JAMA Pediatr 2016; 170 (1): 9–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Kellogg RA, Dunn J, Snyder MP.. Personal omics for precision health. Circ Res 2018; 122 (9): 1169–71. [DOI] [PubMed] [Google Scholar]
  • 14. Khoury MJ, Galea S.. Will precision medicine improve population health? JAMA 2016; 316 (13): 1357–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Khoury MJ, Iademarco MF, Riley WT.. Precision public health for the era of precision medicine. Am J Prev Med 2016; 50 (3): 398–401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Riveroll A, Thompson K, Robertson K, et al. Precision health: a personalized approach to active health management. Can Med Biol Eng Sci 2018; 41: 1–5. [Google Scholar]
  • 17.European Commission. Green Paper on Citizen Science for Europe: Towards a Society of Empowered Citizens and Enhanced Research. Brussels, Belgium: European Commission; 2014. [Google Scholar]
  • 18. Dick DM. Rethinking the way we do research: the benefits of community-engaged, citizen science approaches and nontraditional collaborators. Alcohol Clin Exp Res 2017; 41 (11): 1849–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [19] Petersen C.Patient informaticians: turning patient voice into patient action. JAMIA Open 2018; 1 (2): 130–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Buyx A, Del Savio L, Prainsack B, et al. Every participant is a PI. Citizen science and participatory governance in population studies. Int J Epidemiol 2017; 46 (2): 377–84. [DOI] [PubMed] [Google Scholar]
  • 21. Den Broeder L, Devilee J, Van Oers H, et al. Citizen science for public health. Health Promot Int 2018; 33 (3): 505–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Armstrong AW, Cheeney S, Wu J, et al. Harnessing the power of crowds. Am J Clin Dermatol 2012; 13 (6): 405–6. [DOI] [PubMed] [Google Scholar]
  • 23. de Weger LA, Hiemstra PS, Op den Buysch E, et al. Spatiotemporal monitoring of allergic rhinitis symptoms in The Netherlands using citizen science. Allergy 2014; 69 (8): 1085–91. [DOI] [PubMed] [Google Scholar]
  • 24. Follett R, Strezov V.. An analysis of citizen science based research: usage and publication patterns. PLoS One 2015; 10 (11): e0143687.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Ranard BL, Ha YP, Meisel ZF, et al. Crowdsourcing–harnessing the masses to advance health and medicine, a systematic review. J Gen Intern Med 2014; 29 (1): 187–203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.University of California San Francisco. The Pride Study. San Francisco, CA: University of California San Francisco; 2015. [Google Scholar]
  • 27.PCORnet PPRN Consortium, Daugherty SE, Wahba S, Fleurence R.. Patient-powered research networks: building capacity for conducting patient-centered clinical outcomes research. J Am Med Inform Assoc 2014; 21 (4): 583–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. King AC, Winter SJ, Sheats JL, et al. Leveraging citizen science and information technology for population physical activity promotion. Transl J Am Coll Sports Med 2016; 1 (4): 30–4. [PMC free article] [PubMed] [Google Scholar]
  • 29. Garrison LP Jr, Towse A.. Value-based pricing and reimbursement in personalised healthcare: introduction to the basic health economics. J Pers Med 2017; 7 (3). pii: E10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30. Welch BM, Marshall E, Qanungo S, et al. Teleconsent: a novel approach to obtain informed consent for research. Contemp Clin Trials Commun 2016; 3: 74–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. Kondylakis H, Koumakis L, Hänold S, et al. Donor's support tool: enabling informed secondary use of patient's biomaterial and personal data. Int J Med Inform 2017; 97: 282–92. [DOI] [PubMed] [Google Scholar]
  • 32. Kaye J, Whitley EA, Lund D, et al. Dynamic consent: a patient interface for twenty-first century research networks. Eur J Hum Genet 2015; 23 (2): 141–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Özdemir V, Badr KF, Dove ES, et al. Crowd-funded micro-grants for genomics and “big data”: an actionable idea connecting small (artisan) science, infrastructure science, and citizen philanthropy. OMICS 2013; 17 (4): 161–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Haga SB, Rosanbalm KD, Boles L, et al. Promoting public awareness and engagement in genome sciences. J Genet Counsel 2013; 22 (4): 508–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.PGP-UK Consortium. Personal Genome Project UK (PGP-UK): a research and citizen science hybrid project in support of personalized medicine. BMC Med Genomics 2018; 11: 108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.National Aeronautics and Space Administration. Target Asteroids! National Aeronautics and Space Administration. Washington, DC; 2019. http://www.asteroidmission.org/get-involved/target-asteroids/ Accessed May 11, 2019. [Google Scholar]
  • 37. Bonney R, Phillips TB, Ballard HL, et al. Can citizen science enhance public understanding of science? Public Underst Sci 2016; 25 (1): 2–16. [DOI] [PubMed] [Google Scholar]
  • 38. Bonney R, Shirk JL, Phillips TB, et al. Citizen science. Next steps for citizen science. Science 2014; 343 (6178): 1436–7. [DOI] [PubMed] [Google Scholar]
  • 39. Riesch H, Potter C.. Citizen science as seen by scientists: methodological, epistemological and ethical dimensions. Public Underst Sci 2014; 23 (1): 107–20. [DOI] [PubMed] [Google Scholar]
  • 40. Sullivan BL, Aycrigg JL, Barry JH, et al. The eBird enterprise: an integrated approach to development and application of citizen science. Biol Conserv 2014; 169: 31–40. [Google Scholar]
  • 41.American Medical Informatics Association. Redefining our Picture of Health: Towards a Person-Centered Integrated Care, Research, Wellness, and Community Ecosystem: A White Paper of the 2017 AMIA Policy Invitational. Washington, DC: American Medical Informatics Association. [Google Scholar]
  • 42. Khodyakov D, Grant S, Meeker D, et al. Comparative analysis of stakeholder experiences with an online approach to prioritizing patient-centered research topics. J Am Med Inform Assoc 2017; 24 (3): 537–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Kim KK, Khodyakov D, Marie K, et al. A novel stakeholder engagement approach for patient-centered outcomes research. Med Care 2018; 56 (Suppl) (10 Suppl 1): S41–S47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Sylvia LG, Hearing CM, Montana RE, et al. MoodNetwork: an innovative approach to patient-centered research. Med Care 2018; 56 (Suppl) (10 Suppl 1): S48–S52. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45. Bassani G, Marinelli N, Vismara S.. Crowdfunding in healthcare. J Technol Transf 2019; 44 (4): 1290–310. [Google Scholar]
  • 46.Stanford Medicine. Stanford Medicine 2018 Health Trends Report: The Democratization of Health Care https://med.stanford.edu/content/dam/sm/school/documents/Health-Trends-Report/HTR-Designed-Version-12.12.18-930am-PT.pdf Accessed December 13, 2018.
  • 47. Wang Y, Kaplan N, Newman G, et al. CitSci.org: a new model for managing, documenting, and sharing citizen science data. PLoS Biol 2015; 13 (10): e1002280.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Nebeker C, Bloss C, Weibel N.. 371. Connected and open research ethics (CORE) initiative: engaging stakeholders to shape ethics in the digital age. Biol Psychiatry 2017; 81(10) (Suppl): S151–S152. [Google Scholar]
  • 49. Aungst H, Fishman JR, McGowan ML.. Participatory genomic research: ethical issues from the bottom up to the top down. Annu Rev Genom Hum Genet 2017; 18 (1): 357–67. [DOI] [PubMed] [Google Scholar]
  • 50. Hekler E. Is it Safe? Opening Pathways. San Diego, CA: http://openingpathways.org/is-it-safe Accessed December 31, 2018. [Google Scholar]
  • 51. Delbanco T, Walker J, Bell SK, et al. Inviting patients to read their doctors' notes: a quasi-experimental study and a look ahead. Ann Intern Med 2012; 157 (7): 461–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52. Bell SK, Gerard M, Fossa A, et al. A patient feedback reporting tool for OpenNotes: implications for patient-clinician safety and quality partnerships. BMJ Qual Saf 2017; 26 (4): 312–22. [DOI] [PubMed] [Google Scholar]
  • 53. Petersen C. The future of patient engagement in the governance of shared data. EGEMS 2016; 4 (2): 1214.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54. Lewis D. Spectrums of Perspective. San Diego, CA: Opening Pathways; http://openingpathways.org/spectrums-of-perspective. [Google Scholar]
  • 55. Sullivan M, Kone A, Senturia KD, et al. Researcher and researched—community perspectives: toward bridging the gap. Health Educ Behav 2001; 28 (2): 130–49. [DOI] [PubMed] [Google Scholar]
  • 56. Khodyakov D, Mikesell L, Schraiber R, et al. On using ethical principles of community-engaged research in translational science. Transl Res 2016; 171: 52–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.European Citizen Science Association. Ten Principles of Citizen Science. Berlin, Germany: European Citizen Science Association; 2018. [Google Scholar]
  • 58. Almalki M, Gray K, Martin-Sanchez FJ.. Refining the concepts of self-quantification needed for health self-management: a thematic literature review. Methods Inf Med 2017; 56 (1): 46–54. [DOI] [PubMed] [Google Scholar]
  • 59. Hinckson E, Schneider M, Winter SJ, et al. Citizen science applied to building healthier community environments: advancing the field through shared construct and measurement development. Int J Behav Nutr Phys Act 2017; 14 (1): 133.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60. Choe EK, Lee B, Munson S, et al. Persuasive performance feedback: the effect of framing on self-efficacy. AMIA Annu Symp Proc 2013; 2013: 825–33. [PMC free article] [PubMed] [Google Scholar]
  • 61. Afshinnekoo E, Ahsanuddin S, Mason CA.. Globalizing and crowdsourcing biomedical research. Br Med Bull 2016; 120 (1): 27–33. [DOI] [PubMed] [Google Scholar]
  • 62. Haldar S, Mishra SR, Khelifi M, et al. Opportunities and design considerations for peer support in a hospital setting. Proc SIGCHI Conf Hum Factor Comput Syst 2017; 2017: 867–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.University of Washington Information School. Patients as Safeguards. Seattle, WA: University of Washington Information School; https://patientsassafeguards.ischool.uw.edu/ Accessed May 11, 2019. [Google Scholar]
  • 64.Agile Science. What is Agile Science? Tempe, AZ: Agile Science; 2015. [Google Scholar]
  • 65. Hekler EB, Klasnja P, Riley WT, et al. Agile science: creating useful products for behavior change in the real world. Behav Med Pract Policy Res 2016; 6 (2): 317–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Patient-Centered Research Outcomes Institute. NAPCRG’s Patient and Clinician Engagement (PaCE) Program 2.0; 2016. https://www.pcori.org/research-results/2016/napcrgs-patient-and-clinician-engagement-pace-program-20 Accessed December 13, 2018.
  • 67. Nickel WK, Weinberger SE, Guze PA; for the Patient Partnership in Healthcare Committee of the American College of Physicians. Principles for patient and family partnership in care: an American College of Physicians position paper. Ann Intern Med 2018; 169 (11): 796–9. [DOI] [PubMed] [Google Scholar]
  • 68. Reardon J. The ‘persons’ and ‘genomics’ of personal genomics. Per Med 2011; 8 (1): 95–107. [DOI] [PubMed] [Google Scholar]
  • 69. Chung AE, Sandler RS, Long MD, et al. Harnessing person-generated health data to accelerate patient-centered outcomes research: the Crohn's and Colitis Foundation of America PCORnet patient powered research network (CCFA Partners). J Am Med Inform Assoc 2016; 23 (3): 485–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70. Chung AE, Vu MB, Myers K, et al. Crohn's and Colitis Foundation of America partners patient-powered research network: patient perspectives on facilitators and barriers to building an impactful patient-powered research network. Med Care 2018; 56 (Suppl 10) (Suppl 1): S33–S40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71. Kim KK, Helfand M.. Engagement in PCORnet research networks. Med Care 2018; 56 (Suppl 10) (Suppl 1): S1–S3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72. Leese J, Macdonald G, Kerr S, et al. Adding another spinning plate to an already busy life’. Benefits and risks in patient-partner-researcher relationships: a qualitative study of patient partners’ experiences in a Canadian health research setting. BMJ Open 2018; 8 (8): e022154. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73. McKillop M, Voigt N, Schnall R, et al. Exploring self-tracking as a participatory research activity among women with endometriosis. J Participatory Med 2016; 8:e17. https://participatorymedicine.org/journal/evidence/research/2016/12/29/exploring-self-tracking-as-a-participatory-research-activity-among-women-with-endometriosis/ Accessed April 18, 2019. [Google Scholar]
  • 74. McKillop M, Mamykina L, Elhadad N.. Designing in the dark: eliciting self-tracking dimensions for understanding enigmatic disease. In: Proceedings of the 36th Annual Association for Computing Machinery Conference on Human Factors in Computing Systems; 2018. http://people.dbmi.columbia.edu/noemie/papers/18chi.pdf Accessed April 18, 2019. [Google Scholar]
  • 75.Make the Breast Pump Not Suck. Make the Breast Pump Not Suck Hackathon. Cambridge, MA: Make the Breast Pump Not Suck; 2018. [Google Scholar]
  • 76. Hope A, D’Ignazio C, Hoy J, et al. Hackathons as participatory design: iterating feminist utopias. In: CHI Conference on Human Factors in Computing Systems Proceedings (CHI 2019); May 4–9, 2019; Glasgow, Scotland, UK. ACM, New York, NY. 2019. https://dl.acm.org/citation.cfm? id=3300291 Accessed May 11, 2019. [Google Scholar]
  • 77. McDonald D, Hornig M, Lozupone C, et al. Towards large-cohort comparative studies to define the factors influencing the gut microbial community structure of ASD patients. Microb Ecol Health Dis 2015; 26: 26555. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78. McDonald D, Hyde E, Debelius JW, et al. American Gut: an open platform for citizen science microbiome research. mSystems 2018; 3 (3). pii: e00031–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79. Whitehead J, Shaver J, Stephenson R.. Outness, stigma, and primary health care utilization among rural LGBT populations. PLoS One 2016; 11 (1): e0146139.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80. LaVaccare S, Diamant AL, Friedman J, et al. Healthcare experiences of underrepresented lesbian and bisexual women: a focus group qualitative study. Health Equity 2018; 2 (1): 131–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 81. Phelan SM, Burgess DJ, Yeazel MW, et al. Impact of weight bias and stigma on quality of care and outcomes for patients with obesity. Obes Rev 2015; 16 (4): 319–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82. Crenshaw K. Demarginalizing the intersection of race and sex: a black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. U Chi Legal F 1989; 1989: Article 8. [Google Scholar]
  • 83. López N, Gadsden VL.. Health inequities, social determinants, and intersectionality. In National Academy of Sciences Discussion Paper Perspectives: Expert Voices in Health and Health Care, Washington, DC, USA; 2016: 1–15. [Google Scholar]
  • 84. Bauer GR. Incorporating intersectionality theory into population health research methodology: challenges and the potential to advance health equity. Soc Sci Med 2014; 110: 10–7. [DOI] [PubMed] [Google Scholar]
  • 85. McCann E, Lee R, Brown M.. The experiences and support needs of people with intellectual disabilities who identify as LGBT: a review of the literature. Res Dev Disabil 2016; 57: 39–53. [DOI] [PubMed] [Google Scholar]
  • 86. Ramasamy V, Rillotta F, Alexander J.. Experiences of adults with intellectual disability who identify as lesbian, gay, bisexual, transgender, queer or questioning, intersex or asexual: a systematic review protocol. JBI Database System Rev Implement Rep 2017; 15 (9): 2234–41. [DOI] [PubMed] [Google Scholar]
  • 87. Mattocks KM, Sullivan JC, Bertrand C, et al. Perceived stigma, discrimination, and disclosure of sexual orientation among a sample of lesbian veterans receiving care in the Department of Veterans Affairs. LGBT Health 2015; 2 (2): 147–53. [DOI] [PubMed] [Google Scholar]
  • 88. Khodyakov D, Mikesell L, Bromley E.. Trust and the ethical conduct of community-engaged research. Eur J Pers Cent Healthc 2017; 5 (4): 522–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 89. Wilkins CH. Effective engagement requires trust and being trustworthy. Med Care 2018; 56 (Suppl 1): S6–S8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 90. Angrist M. Eyes wide open: the personal genome project, citizen science and veracity in informed consent. Per Med 2009; 6 (6): 691–9. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from JAMIA Open are provided here courtesy of Oxford University Press

RESOURCES