Abstract
This qualitative review explores how established citizen science models can inform and support meaningful engagement of public in health research in Australia. In particular, with the growth in participatory health research approaches and increasing consumer participation in contributing to this research through digital technologies, there are gaps in our understanding of best practice in health and biomedical citizen science research to address these paradigm shifts. Notable gaps are how we might more clearly define the parameters of such research and which citizen science models might best support digitally-enabled participation falling within these. Further work in this area is expected to lead to how established citizen science methods may help improve the quality of and the translation of public engagement in health research.
Keywords: citizen science, community based participatory research, crowdsourcing, public health
Introduction
Global growth in participatory research approaches to address complex health challenges is being progressively supported by online platforms, tools and digital sensing devices. The mainstreaming of this form of public engagement is at the forefront of national research and health policy programmes [1,2].
Within Australia, “consumer and community participation” in research is part of a paradigm shift leading to the increasing active involvement of the public (i.e. consumers and community members) and researchers working together to make decisions about health research priorities, policy and practice [3]. In the findings of a 2018 Australian-wide survey (= 868 respondents) on the perceptions of such engagement, the majority of survey respondents (97%) reported that public involvement has meaningful value to all phases of health and medical research. The survey findings also acknowledged that public involvement is complex and differs across the research spectrum, thus requiring models that are flexible and have applicability in diverse research situations [4].
In this context, the authors provide an overview of the applicability of potential models which might be considered by participatory health researchers in Australia. Specifically, public engagement in health and biomedical research is increasingly being influenced by the paradigm of “citizen science”, that is, active participation in research teams by members of the general public with no formal training in the field of research concerned [5-7].
Citizen science as an approach for public engagement in research dates back well-over a century in some fields of research, for example, natural history, where the US Audubon Society’s Christmas Bird Count began in 1900 [5]. Citizen science activity has dramatically increased in the 21st Century, influenced by societal and technological changes and participatory democracy. Critically, it has enabled the large-scale collection and processing of scientific data and widespread dissemination of scientific knowledge and discoveries, notably in environmental sciences, ecology, and astronomy [8-10].
Citizen science traditions in Australia
Australia has long-standing traditions in citizen science and public participation in scientific enquiry. Although the term “citizen science” came into prominence in the mid-1990s [5,6], citizen science in Australia has been established by grassroots activities and bottom-up approaches. Over a century ago, Ferdinand von Mueller (1825-1896), the reputed nineteenth century Australian botanist advertised for the assistance of “lady plant collectors“ across the country in one of the most notable citizen science projects. von Mueller inspired an estimated 1300 volunteers to participate in the project in submitting specimens, which culminated into informing the seven volume Flora Australiensis published between 1863 and 1878 by George Bentham [11].
Australia is also a country holding unique local Indigenous traditional knowledge that spans thousands of years of ecological monitoring and empirical mapping of phenomena that precedes defined citizen science and participatory research approaches [12,13]. Indigenous knowledge and citizen science represent parallel discourses which have emerged in both southern and northern hemispheres [14].
The Australian Citizen Science Association (ACSA)was established in 2014 following national consultations in recognition of the need for a community of practice to support the expanding field of citizen science in Australia, and aligned with international advancements [15]. Around this time, citizen science practices were formalised through communities of practice organisations in Europe with the establishment of the European Citizen Science Association (ECSA) and in the USA, through the Citizen Science Association (CSA). The launch of ACSA in 2015 was particularly bolstered by a White paper Occasional Paper on Citizen Science by the then Chief Scientist of Australia [16]. ACSA has adapted the ECSA Principles of Citizen Science as its working principles that highlight the common good aspects of involvement of citizens in scientific endeavours which, for example, can generate new knowledge or understanding, provide benefits to science and society, support reciprocity, ethical approaches and publicly available results [17].
Australia has fostered an increasing number of citizen science developments, most visibly in the field of biodiversity as exemplified by one of the largest nation-wide platforms of its kind, the Atlas of Living Australia (ALA). Launched in 2010, the ALA is a collaborative, national project that aggregates biodiversity data from multiple sources and currently holds data on over 120,000 individual species in the ALA database [18]. The ALA also hosts the Australian Citizen Science Project Finder online database, with over 600 projects across Australia [19]. Opportunities for large-scale projects in Australia have been made possible as citizen science is associated with increasing funding, infrastructure and support at different levels of government [20].
Citizen science models of participatory health research
Health and biomedical citizen science is not readily defined as a separate practice or associated with a specific framework or schema, although the field is increasingly identified with a number of engagement models [21-24]. Projects involving human health and participants can be viewed as citizen science among various forms of community-based participatory research, action research, patient and public involvement (PPI), self-quantification, crowdsourced health research, among other practices [15,23-26].
The commonalities are associated with an enabled and widening participation, most often supported by ICT and digital and social media which have rapidly increased the range of people who can participate in health and biomedical data collection and, where appropriate, data analysis and other tasks [25,27]. These technological developments, combined with policy initiatives and consumer empowerment, have also given rise to new ways of activation, for example, in health advocacy, health self-management and reporting, and research design.
Generally, citizen science projects have been differentiated according to the extent of responsibilities that participants undertake for research activities, such as collecting and analysing data (contributory) and interpreting and disseminating results (collaborative) [6,8]. In most studies, citizen projects are typically instigated by professionally trained researchers in which participants are supporting tasks in a research process; however, community scaled participatory research projects, for example, can be a cooperative activity. In the latter example, a higher level of ownership may be associated with co-creation in which researchers and members of the public working together across research processes [6,8,28]. A broadening of this framework is citizen-led or extreme citizen science approaches, which aim to provide tools and methods to enable communities to develop participatory research projects to address issues that concern them [28,29].
Meaningful participation, according to Kelty et al [30], engages participants along seven dimensions: for example, receiving an educative dividend; involvement in decision-making and goal-setting; and having control or ownership of the resources produced by participation. There should be a “collective, affective” experience of participating in order for participants to feel they are part of something greater than themselves [30]. In community based participatory research (CBPR), for instance, participants can provide researchers with advice concerning the design of research projects, potentially including the research goals, design of survey instruments, recruitment, and data interpretation and dissemination, among other activities [31-33]. The need to ensure ownership and control over local knowledge is highly relevant in situations of CBPR that can inform wider citizen science practices [32-35].
Such multidimensionality of participatory activities in health research is necessarily expressed in different schemas. Engaging large numbers of the public is often undertaken through crowdsourcing [36] which has been the most visibly applied citizen science method to the fields of health and biomedical research [25,27]. Crowdsourced health research studies have been defined as the nexus of three contemporary trends: “citizen science, crowdsourcing, and Medicine 2.0” [37].
Crowdsourced tasks can be quite wide-ranging but mainly rely on the use of digital tools and platforms for use by the public in undertaking specific tasks [27]. For instance, volunteers are classifying images of the M. tuberculosis bacteria as part of the Bash the Bug project hosted on the Zooniverse citizen science web portal, and have reached over one million classifications [38]. Crowdsourced data processing can further involve both lay-people and those knowledgeable in a discipline, particularly where complex tasks, e.g. forms of annotation, relational tasks or problem-solving are applied. Cochrane Crowd, the citizen science platform of Cochrane, a global network promoting evidence-informed health decision-making by producing high-quality accessible systematic reviews, is comprised of a crowdsourced community of volunteers who undertake supporting tasks such as Randomised Controlled Trial (RCT) identification in research papers [39].
Crowdsourced health research projects can further target health conditions in which participants undertake self-reporting using mobile apps and wearable sensors [40]. Health promotion is increasingly another context in which the engagement of the public through self-reporting and data collection is contributing to forms of public health research and policy [34,35]. Critically, in a public health landscape, citizen science can support localised participatory action research and “participatory epidemiology” leading to the capture of qualitative and quantitative epidemiological information contained within community observations, including traditional oral history [35]. The use of gamification – often termed as “serious games” - is a digitally-enabled method to support participatory action research. For example, SpotLab is an example of a disease surveillance platform that uses gaming and mobile phones turned into low-cost microscopes to provide collective diagnosis of global health diseases, such as malaria diagnosis [40]. Serious games developed by professional scientists, such as the multiple sequence alignment online game, Phylo developed at McGill University (Montreal, Canada), are further instances of contribution by the public in problem-solving tasks as in this example of finding ways of sequence arrangement of DNA and RNA to identify regions of similarity [41,42].
There is a concurrent rise of digitally-engaged and activated citizens, alternatively called “biocitizens” by some in the health research literature and to distinguish from other types of non-health related participation, especially at the individual level [37,43]. The notion of biocitzenship firstly emerged through the concept of biosociality [44] – that is the making of connections between biotechnology and citizenship. Recently the concept of “participatory biocitizen” coined by Melanie Swan [37] refers to an activated individual and as a means to realise personalized medicine by sharing life-logging and self-quantification data through social media platforms. Self-quantifiers, in particular, represent high levels of activation that may motivate these individuals to independently mobilise citizen scientists and/or approaches [43,45,46]. These approaches are often typically outside of the instigation of organised health professionals or scientific organisations, as in the case of biohackers [43]. In the US, “people powered research networks” (PPRN) are leading on the sharing of quantifiable data, exchanging experiences on treatments, and searching for clinical trials on online platforms [45,47]. Examples include the personalized health and research network, PatientsLikeMe [48], and advocacy network iConquerMS which is focused on the multiple sclerosis (MS) community [49].
Commercial and government-led research partnerships provide another means by which public engagement can be supported. For instance, Sage Bionetworks and the University of Rochester partnered with Apple ResearchKit to study the lifestyle of 17,000 Parkinson’s disease patients in the mpower initiative [50]. The nation-wide $215 million Precision Medicine Initiative All of Us in the U.S. aims to build a large-scale research platform between public and private sectors, calling for one million volunteers to contribute their health data [51]. Other opportunities are provided through joint collaborations between the public and healthcare organisations in co-creating knowledge. One example is the Sarroch Bioteca Foundation established in 2012 in pursuit of a “citizenveillance on health” project in Italy [52].
Among the national agencies at the forefront of the development and promotion of public and community involvement in health research include INVOLVE in the UK [53]. INVOLVE is a government funded programme established in 1996 and is part of, and funded by, the National Institute for Health Research, to support active public involvement in the National Health Service (NHS), public health and social care research. Related initiatives are the Patient Centred Outcomes Research Institute (PCORI) of the National Institutes of Health in the U.S. and the Strategy for Patient Oriented Research (SPOR) of the Canadian Institutes of Health Research [1,4].
The Australian context
Aligning with international developments, Australia is implementing various approaches to strengthen public involvement in health research. The National Health and Medical Research Council (NHMRC), for instance, provides national guidelines for responsible research practices and advocates for appropriate public involvement in research [3]. The Consumer and Community Health Research Network based in Western Australia is an example of a practical implementation of a service which is leading programs and resource development to promote awareness and support [54]. The Network is among several agencies comprising the Australian Health Research Alliance (AHRA) which was involved in undertaking a major review project in 2017-2018, including the deployment of an Australia-wide survey to capture the extent and nature of consumer and community involvement across AHRA member organisations. Recommended priorities include the development of guidance on incorporating public involvement across the research life cycle, and associated tools and resources to enable and support partnerships, as well as other capacity building initiatives, such as training [4].
As a complementary direction to these priorities, a review of the application of existing citizen science models to participatory health research has identified several examples for the purposes of this commentary. The selected projects are based or led from within Australia and focus on forms of digitally-enabled public participation in health research which use publicly accessible ‘citizen science’ tools and platforms. Examples draw on the models of contributory (e.g. participation via data collection and processing), collaborative (e.g. public involvement in refining research questions, analysing data, and/or disseminating findings), co-creation (e.g. researchers and members of the public work together across key research processes), and extreme citizen science in which researchers provide tools and methods to enable communities to develop their own participatory research projects [6,8,28]. Projects are thematically clustered using a public health research lens as follows:
Indigenous science and environmental health
In the context of Indigenous participation and environmental health, the Aboriginal concept of health appears as part of a health impact assessment approach by the Australian Indigenous Doctors’ Association [55] and mentioned as an exemplar in a prominent position paper on citizen science and public health [56]. Several Indigenous partnership projects are hosted in the Atlas of Living Australia database (ALA), and discoverable in the ACSA Project Finder. For instance, The Tracks App of the Indigenous Desert Alliance (IDA) is an ecological science mobile app co-created with stakeholders to support the involvement of Aboriginal people and Indigenous Ranger groups in remote areas [57]. The Tracks App supports the collection of information on native animal tracks and other signs, such as scats, diggings, burrows, bones and feathers.
The Commonwealth Scientific and Industrial Research Organisation (CSIRO), an independent Australian federal government agency responsible for scientific research, is supporting different citizen science solutions in a number of environmental health projects. The Eye on Water project is an example of a digitally-enabled participation project using a water quality monitoring app to collect data about water changes around Australia due to climate change phenomena [58]. The project routinely involves school students who assist in making physical and chemical measurements, as well as providing digitally captured data, alongside members of the volunteer public. Researchers are using the datasets to compare water quality data with satellite-derived ocean colour or coastal/inland water feature data [58].
An example of a serious game supporting environmental health education and contributory approaches, QuestaGame was launched in 2014 in Canberra through a successful crowdfunding campaign. The game has players around the world mapping biodiversity sightings and sending photographs through the QuestaGame mobile app for identification. The data feeds are downloaded for research, of which 1.6 million verifications have been identified at the time of this article [59].
Health promotion
Contributory models of approach have largely been applied to public participation in health promotion and public health research in several large-scale campaigns in Australia. For example, the Anti-Cancer Council of the Australian state of Victoria has been running sun protection programmes for several decades: Slip! Slop! Slap! from 1980 to 1988 and SunSmart from 1988 to the present [60]. Sunday Morning is an Australian social media health promotion movement that asks participants to publicly set a personal goal to stop drinking or reduce their alcohol consumption for a set period of time, and to record their reflections and progress on blogs and social networks [61]. The Hello Sunday platform was created in 2010 and thus far participants have shared over 2 million stories according to the website [62]. FoodSwitch is a mobile phone app that allows users to access easy-to-understand information about the nutritional characteristics of packaged foods and, where available, to suggest healthier alternative products [63]. A particular innovation in the app was the incorporation of a crowdsourcing function whereby users are able to contribute information on missing products. If a barcode is scanned but the corresponding Universal Product Code (UPC) is not identified in the database, then the user is prompted to photograph the front of the package, the Nutrition Information Panel (NIP), and the ingredients list. The data are uploaded to the data management center site, and the information is added to a national database. Periodic updates to the database are then made to ensure that the app is supported by complete and up to date product information [64].
Epidemiology and exposure science
Australian researchers have also employed citizen science contributory models to engage the public in specific areas of health condition enquiry. The Big Sleep Survey undertaken in 2010 solicited the contribution of participants who monitored their sleeping habits for one week for the Woolcock Institute and Sydney University [65]. More than 12,000 participants reported on their personal sleep habits in the questionnaire, while over 3,500 people completed the week-long sleep diaries recording the time they went to bed, fell asleep and woke up, for example [66].
DustSafe is a citizen-science initiative based in the Department of Environmental Sciences at Macquarie University in Sydney - a local chapter of the global program: 360 Dust Analysis [67], which encompasses research groups in Australia, Asia, the United Kingdom, and the U.S. Together, these programs are focused on characterising household dust in an effort to understand the potential health exposure hazards residing in that dust. Greater urbanisation means people are spending up to 90% of their time indoors. Consequently, environmental health risks are dominated by fine dust particles from indoor air aerosols and harmful agents that can penetrate deep into lungs and migrate into the blood stream. Knowledge gaps are addressed by engaging the public to submit vacuum dust samples for chemical and biological analysis [68].
An example of a participatory sensing initiative, the Citizen Science Project is a collaboration between RMIT University in Melbourne and the University of New South Wales in Sydney begun in 2018 [69]. The project aims to involve the public in measuring urban heat island effects and local climate change. Public participants perform outdoor microclimatic measurements of temperature, humidity, wind speed, among other measures, using portable handheld sensing devices provided by the project on a selected day at a selected location.
Health data systems
The Garvan Institute of Medical Research in Sydney and the Vodafone Foundation have recently initiated a computing platform that uses the collective processing capacity of smartphones to analyse numbers and compare genetic profiles of tumours through the DreamLab mobile app [70]. The public can contribute their mobile phone’s processing power when a phone is fully charged or charging overnight, for instance, whilst the DreamLab app analyses research data for breast, ovarian, prostate and pancreatic cancers.
An example of a collaborative platform, Crowdsourcing Critical Appraisal of Research Evidence or CrowdCARE is a freely available online tool developed by University of Melbourne researchers that aims to teach critical appraisal and facilitate the sharing of appraisals amongst a global community of clinicians [71]. After completing compulsory training modules, individuals can contribute to, and benefit from, an evolving collection of appraised research evidence generated from an interdisciplinary group, committed to practicing evidence-based practice (EBP). This platform is unique in that it goes beyond the use of crowdsourced judgments by article type (e.g. RCTs), as in CochraneCrowd to an in-depth assessment of the methodological rigour of the articles [72].
On the level of individual self-tracking, which can be a means of contributing personal physiological data to larger research studies, there are numerous communities using self-tracking tools and technologies, many of whom identify with the Quantified Self (QS) movement. QS is defined as “embodying self-knowledge through self-tracking” [73], and one of the early Quantified-Self group setups outside of the U.S. was based in Sydney Australia [46]. Strava, the international social fitness network, has an Australian membership of self-quantifiers who measure physical performance, primarily tracking cycling and running activities using GPS data [74]. Australian researchers have partnered with Strava members in a range of studies, such as understanding the personal data practices of commuting cyclists, for example [75].
Challenges and limitations
Public participation in health research in Australia potentially faces several shared challenges to those experienced world-wide in the citizen science community. The lack of standard definitional boundaries of what constitutes digitally-enabled public participation in health research relates to a diversity of approaches, e.g. citizen science, community-based participatory research (CBPR), action research, patient and public involvement (PPI), and crowdsourcing, among other approaches [15,23,76].
Critically, within this context, there is a clear gap in readily identifying tools and platforms, particularly those which are supporting digitally-enabled participation of the public, their efficacy and support of different levels of participation. For instance, tools supporting local participatory engagement, e.g. CBPR, may differ from those in use by self-quantifying participants or by a broader public in larger crowdsourced projects, for instance. Generally, in public participation in health research, the more closely-associated a project is with the participant (e.g. in the home or the individual person), the greater the potential for legal, ethical, privacy, biosafety and data management and ownership complications to be raised [77,78]. It is, therefore, necessary to consider what shared standards, methodologies and practices might be applicable [2,79].
A related challenge is that stronger public involvement in health research requires improved understanding of research processes [1]. For instance, public involvement in research data processes and ensuring data quality are pervasive issues; primarily due to the fact that a non-expert public is generally considered to be untrained in research data management or research integrity, or may be prone to systematic errors which can impact data quality [80,81]. Data quality can also be highly context dependent (i.e. “fitness for use”) [81-83].
A higher level of quality assurance is often associated with the use of crowdsourcing in which many people carry out the same work or task, such as contributing to peer review [39,84], or replication of an analysis, such as image identification [27]. Such an approach is desirable across the sciences for validation, accuracy, and in reducing bias. The critical research appraisal tool CrowdCARE, for instance, has shown that novices can be trained to appraise the rigor of published systematic reviews and, on average, achieve a high degree of accuracy relative to the experts [72].
Additionally, there are identified gaps in the literature on reporting methods and the extent of public involvement in clinical trials which constitute a critical share of health and biomedical research design. A study of public research involvement in online trials concerning health self-management, for example, found that detailed reporting of such involvement was hindered by role confusion between research volunteers and trial participants [22]. Other related study findings contribute to the literature by documenting researchers’ perspectives and experiences about sharing results with research participants. One such study surveyed health researchers in which the majority of respondents indicated that health research results should always be shared with participants [85]; although the described barriers to results sharing and various reported reasons not to share results suggest difficulties with a “one-size-fits-all” approach to improving results sharing.
Legal jurisdictional areas may differ in terms of the extent of sharing of results with participants. In the U.S., for instance, it is not normative for health research ethics review boards to encourage health researchers to share aggregate study findings in contrast to practices in Australia, for example [85]. A poll conducted by Research Australia in 2019 indicated that a majority of Australians were willing to share their personal health information for research purposes, in order to advance health and medical research (78%), support healthcare providers in improving patient care (68%), and to assist public health officials in tracking diseases, disabilities and their causes (61%) [86]. However, the proportion of consumers in support of sharing health data with government organisations can be significantly lower [86,87].
Indigenous communities are particularly concerned with issues pertaining to handling, treatment, and ownership of tissue as well as knowledge gained from specimen analysis. This stems from a strong and integrated sense of cultural connection to ancestors and traditional lands and Indigenous communities view biological specimens as inseparable from these connections [88].
Conclusion
Australia has long-standing traditions in aspects of citizen science and participatory approaches to scientific enquiry, as can be gleaned from major projects and historical activities outlined in the present paper. However, there are gaps in our understanding of the extent of public participation in health research as part of this paradigm shift, particularly as it may relate to digitally-enabled processes. Progress in this area depends on identifying partnerships across the shared challenges, and extending work on how established citizen science models may help support the quality of and the translation of public engagement in health research in Australia.
Acknowledgements
Ann Borda received an Expert Visit grant in 2019 as part of the EPIC project funded under the EU Horizon 2020 programme (ICT) to explore advancing approaches to citizen science methods, platforms and capabilities in healthcare and medical research.
References
- 1.Todd AL, Nutbeam D. 2018. Involving consumers in health research: what do consumers say? Public Health Res Pract. 28(2), e2821813. 10.17061/phrp2821813 [DOI] [PubMed] [Google Scholar]
- 2.Wiggins A, Wilbanks J. 2019. The Rise of Citizen Science in Health and Biomedical Research. Am J Bioeth. 19(8), 3-14. 10.1080/15265161.2019.1619859 [DOI] [PubMed] [Google Scholar]
- 3.National Health and Medical Research Council. 2016. Statement on consumer and community involvement in health and medical research. Canberra [ACT]: NHMRC. [cited 31 October 2019]; Available at: https://www.nhmrc.gov.au/about-us/publications/statement-consumer-and-community-involvement-health-and-medical-research
- 4.Australian Health Research Alliance. 2018. Consumer and Community Involvement in Health and Medical Research: An Australia-wide Audit. Available from: https://www.slhd.nsw.gov.au/SydneyHealthPartners/pdf/AHRA_CCI_Final_Report.pdf
- 5.Irwin A. Citizen science: a study of people, expertise and sustainable development. London: Routledge; 1995. [Google Scholar]
- 6.Bonney R, et al. 2009. Citizen science: a developing tool for expanding science knowledge and scientific literacy. Bioscience. 59(11), 977-84. 10.1525/bio.2009.59.11.9 [DOI] [Google Scholar]
- 7.Kullenberg C, Kasperowski D. 2016. What is citizen science? – A scientometric meta-analysis. PLoS One. 11(1), e0147152. doi:. 10.1371/journal.pone.0147152 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Shirk JL, Ballard HL, Wilderman CC, Phillips T, Wiggins A, et al. 2012. Public participation in scientific research: a framework for deliberate design. Ecol Soc. 17(2), 29 10.5751/ES-04705-170229 [DOI] [Google Scholar]
- 9.Wiggins A, Crowston K. 2014. Surveying the citizen science landscape. First Monday. 20(1). 10.5210/fm.v20i1.5520 [DOI] [Google Scholar]
- 10.Silvertown J. 2009. A new dawn for citizen science. Trends Ecol Evol. 24(9), 467-71. 10.1016/j.tree.2009.03.017 [DOI] [PubMed] [Google Scholar]
- 11.Olsen P. Collecting Ladies: Ferdinand Von Mueller and Women Botanical Artists. Canberra[ ACT]: National Library Australia; 2013.
- 12.Prober SM, O’Connor MH, Walsh FJ. 2011. Australian Aboriginal peoples’ seasonal knowledge: A potential basis for shared understanding in environmental management. Ecol Soc. 16(2), 12 http://www.ecologyandsociety.org/vol16/iss2/art12/. 10.5751/ES-04023-160212 [DOI] [Google Scholar]
- 13.Hamacher DW. 2018. Observations of red-giant variable stars by Aboriginal Australians. Aust J Anthropol. 29, 89-107. 10.1111/taja.12257 [DOI] [Google Scholar]
- 14.Leach M, Fairhead J. 2002. Manners of Contestation: “Citizen Science” and “Indigenous Knowledge” in West Africa and the Caribbean. Int Soc Sci J. 54, 299-311. 10.1111/1468-2451.00383 [DOI] [Google Scholar]
- 15.Storksdieck M, Shirk JL, Cappadonna JL, Domroese M, Göbel C, et al. 2016. Associations for citizen science: regional knowledge, global collaboration. Citizen Science: Theory and Practice. 1(2),1–10. Available at: https://theoryandpractice.citizenscienceassociation.org/articles/10.5334/cstp.55/
- 16.Pecl G, Gillies C, Sbrocchi C, Roetman P. 2015. Building Australia Through Citizen Science. Office of the Chief Scientist. Canberra (ACT). Available from: http://www.chiefscientist.gov.au/wp-content/uploads/Citizen-science-OP_web.pdf [Google Scholar]
- 17.European Citizen Science Association Ten Principles of Citizen Science Available at: https://ecsa.citizen-science.net/engage-us/10-principles-citizen-science
- 18.Belbin L, Williams KJ. 2016. Towards a national bio-environmental data facility: experiences from the Atlas of Living Australia. Int J Geogr Inf Sci. 30(1), 108-25. 10.1080/13658816.2015.1077962 [DOI] [Google Scholar]
- 19.ACSA. Australian Citizen Science Project Finder. Available at: biocollect.ala.org.au/acsa
- 20.Australian Government. The Department of Industry, Innovation and Science. Citizen science grants. 2018. Available at: https://www.business.gov.au/assistance/inspiring-australia-science-engagement/citizen-science-grants
- 21.Eitzel MV, Cappadonna JL, Santos-Lang C, Duerr RE, Virapongse A, et al. 2017. Citizen science terminology matters: exploring key terms. Citizen Science: Theory and Practice. 2(1), 1. Available at: https://theoryandpractice.citizenscienceassociation.org/articles/10.5334/cstp.96/
- 22.Nunn JS, Tiller J, Fransquet P, Lacaze P. 2019. Public involvement in global genomics research: a scoping review. Front Public Health. 7, 79. 10.3389/fpubh.2019.00079 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Rowbotham S, McKinnon M, Leach J, Lamberts R, Hawe P. 2019. Does citizen science have the capacity to transform population health science? Crit Public Health. 21, 118-28. 10.1080/09581596.2017.1395393 [DOI] [Google Scholar]
- 24.Wiggins A, Wilbanks J. 2019. The rise of citizen science in health and biomedical research. Am J Bioeth. 19(8), 3-14. 10.1080/15265161.2019.1619859 [DOI] [PubMed] [Google Scholar]
- 25.Créquit P, Mansouri G, Benchoufi M, Vivot A, Ravaud P. 2018. Mapping of Crowdsourcing in Health: Systematic Review. J Med Internet Res. 20(5). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Crocker JC, Boylan A-M, Bostock J, Locock L. 2016. Is it worth it? Patient and public views on the impact of their involvement in health research and its assessment: a UK-based qualitative interview study. Health Expect. 20, 519-28. 10.1111/hex.12479 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Wazny K. 2018. Applications of crowdsourcing in health: an overview. J Glob Health. 8(1), 010502. 10.7189/jogh.08.010502 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Haklay M. 2013. Citizen Science and Volunteered Geographic Information – overview and typology of participation. In: Sui, DZ, Elwood, S, and MF Goodchild (eds.). Crowdsourcing Geographic Knowledge. Berlin: Springer, 105-122. [Google Scholar]
- 29.English PB, Richardson MJ, Garzon-Galvis C. 2018. From Crowdsourcing to Extreme Citizen Science: Participatory Research for Environmental Health. Annu Rev Public Health. 39, 335-50. 10.1146/annurev-publhealth-040617-013702 [DOI] [PubMed] [Google Scholar]
- 30.Kelty C, Panofsky A, Currie M, Crooks R, Erickson S, et al. 2015. Seven dimensions of contemporary participation disentangled. J Assoc Inf Sci Technol. 66(3), 474-88. 10.1002/asi.23202 [DOI] [Google Scholar]
- 31.Israel BA, Schulz AJ, Parker EA, Becker AB. 1998. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health. 19, 173-202. 10.1146/annurev.publhealth.19.1.173 [DOI] [PubMed] [Google Scholar]
- 32.Hicks S, Duran B, Wallerstein N, Avila M, et al. 2012. Evaluating Community-Based Participatory Research to Improve Community-Partnered Science and Community Health. Prog Community Health Partnersh. 6(3), 289-99. 10.1353/cpr.2012.0049 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Minkler M. 2014. Enhancing data quality, relevance, and use through community-based participatory research. In: Federal Reserve Bank of San Francisco and Urban Institute(eds). What Counts: Harnessing Data for America’s Communities. San Francisco & Washington, DC., 244-259. Available at: http://www.whatcountsforamerica.org/
- 34.Den Broeder L. 2017. Citizen Science for Health in All Policies. Engaging communities in knowledge development. PhD thesis, VU University Amsterdam, the Netherlands. Available at: http://library.wur.nl/WebQuery/wurpubs/529909
- 35.Rowbotham S, McKinnon M, Leach J, Lamberts R, Hawe P. 2019. Does citizen science have the capacity to transform population health science? Crit Public Health. 29(1), 118-28. 10.1080/09581596.2017.1395393 [DOI] [Google Scholar]
- 36.Dawson R, Bynghall S. 2012. Getting Results From Crowds: The Definitive Guide to Using Crowdsourcing to Grow Your Business. San Francisco: Advanced Human Technologies. [Google Scholar]
- 37.Swan M. 2012. Health 2050: The realization of personalized medicine through crowdsourcing, the quantified self, and the participatory biocitizen. J Pers Med. 2(3), 93-118. 10.3390/jpm2030093 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.BashTheBug [Internet]. Oxford University [cited 2019 October 1]. Available at: http://bashthebug.net/
- 39.Thomas J, Noel-Storr A, Marshall I, Wallace B, McDonald S, et al. 2017. Living Systematic Reviews 2: Combining human and machine effort. J Clin Epidemiol. 91(Nov), 31-37. 10.1016/j.jclinepi.2017.08.011 [DOI] [PubMed] [Google Scholar]
- 40.Ozcan A. 2014. Educational games for malaria diagnosis. Sci Transl Med. 6, 233ed9. 10.1126/scitranslmed.3009172 [DOI] [PubMed] [Google Scholar]
- 41.Curtis V. 2014. Online citizen science games: Opportunities for the biological sciences. Appl Transl Genomics. 3(4), 90-94. 10.1016/j.atg.2014.07.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Johnson D, Deterding S, Kuhn K, Staneva A, Stoyanov S, et al. 2016. Gamification for health and wellbeing: A systematic review of the literature. Internet Interv. 6, 89-106. 10.1016/j.invent.2016.10.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Pauwels E, Denton S. 2018. The rise of the Bio-citizen. 1st ed. Washington, D.C.: The Wilson Centre. Available from: https://www.wilsoncenter.org/sites/default/files/rise_of_biocitizenfinal.pdf [Google Scholar]
- 44.Rabinow P. 1996. Artificiality and enlightenment: from sociobiology to biosociality. In: Essays on the anthropology of reason. Princeton University Press. [Google Scholar]
- 45.Bietz M, Patrick K, Bloss C. 2019. Data Donation as a Model for Citizen Science Health Research. Citizen Science: Theory and Practice. 4(1), 6, 1–11. Available at: https://theoryandpractice.citizenscienceassociation.org/articles/10.5334/cstp.178/
- 46.Almalki M, Gray K, Martin-Sanchez F. 2015. The use of self-quantification systems for personal health information: big data management activities and prospects. Health Inf Sci Syst. 3(Suppl 1), S1. 10.1186/2047-2501-3-S1-S1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Workman TA. Engaging Patients in Information Sharing and Data Collection: The Role of Patient-Powered Registries and Research Networks. AHRQ Community Forum White Paper. AHRQ Publication No. 13-EHC124-EF. Rockville, MD: Agency for Healthcare Research and Quality; September 2013. Available at: https://effectivehealthcare.ahrq.gov/products/stakeholders-engagement-others/white-paper [PubMed]
- 48.PatientsLikeMe [Internet]. PatientsLikeMe Inc. [cited 2019 October 1]. Available at: patientslikeme.com
- 49.iConquerMS [Internet]. iConquerMS.org [cited 2019 October 1]. Available at: www.iconquerms.org
- 50.mpower initiative [Internet]. SageBionetworks.org [cited 2019 October 1]. Available at: parkinsonmpower.org/
- 51.All of Us [Internet]. U.S. Department of Health & Human Services. National Institutes of Health [cited 2019 October 1]. Available at: https://allofus.nih.gov/
- 52.Tallachini M, Boucher P, Nascimento S. 2014. Emerging ICT for Citizens’ Veillance. European Commission JCR Science and Policy Reports. [Google Scholar]
- 53.INVOLVE [Internet]. UK Department of Health and Social Care. National Institute for Health Research [cited 2019 October 1]. Available at: https://www.invo.org.uk/
- 54.Consumer and Community Health Research [Internet]. The University of Western Australia [cited 2019 October 1]. Available at: www.involvingpeopleinresearch.org.au
- 55.Australian Indigenous Doctors’ Association and Centre for Health Equity Training RaEU. 2010. Health Impact Assessment of the Northern Territory Emergency Response. Canberra: Australian Indigenous Doctors’ Association. Available at: https://hiaconnect.edu.au/reports/health-impact-assessment-of-the-northern-territory-emergency-response/
- 56.Den Broeder L, Devilee J, Van Oers H, Schuit AJ, Wagemakers A. 2016. Citizen Science for public health. Health Promot Int. 33(3), 505-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.The Tracks Hub [Internet]. Indigenous Desert Alliance and Atlas of Living Australia (cited 2019 October 1). Available at: https://biocollect.ala.org.au/trackshub/
- 58.Eye on Water [Internet]. EyeonWater.org and CSIRO [cited 2019 October 1]. Available at: www.eyeonwater.org/apps/eyeonwater-australia
- 59.QuestaGame [Internet]. QuestaGame.com [cited 2019 October 1]. Available at: https://questagame.com/
- 60.SunSmart [Internet]. Sun Smart Victoria and Cancer Council Victoria, Melbourne VIC Australia [cited 2019 October 1]. Available at: www.sunsmart.com.au/
- 61.Hello Sunday Morning [Internet]. HelloSundayMorning.org [cited 2019 October 1]. Available at: www.hellosundaymorning.org
- 62.Tait R, Kirkman JJL, Schaub MPA. 2018. Participatory Health Promotion Mobile App Addressing Alcohol Use Problems (The Daybreak Program): Protocol for a Randomized Controlled Trial. JMIR Res Protoc. 7(5), e148. 10.2196/resprot.9982 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.FoodSwitch [Internet]. The George Institute for Global Health, Sydney, NSW Australia [cited 2019 October 1]. Available at: http://www.foodswitch.com.au/
- 64.Dunford E, Trevena H, Goodsell C, Ng KH, Webster J, et al. 2014. FoodSwitch: A Mobile Phone App to Enable Consumers to Make Healthier Food Choices and Crowdsourcing of National Food Composition Data. JMIR Mhealth Uhealth. 2(3), e37. 10.2196/mhealth.3230 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Big Sleep Survey [Internet]. Centre for Integrated Research and Understanding of Sleep, University of Sydney, NSW Australia [cited 2019 October 1]. Available at: www.cirus.org.au/get-involved/sleep-survey/index.php
- 66.Miller CB, Gordon CJ, Toubia L, Bartlett DJ, Grunstein R, et al. 2015. Agreement between simple questions about sleep duration and sleep diaries in a large online survey. Sleep Health. 1(2), 133-37. 10.1016/j.sleh.2015.02.007 [DOI] [PubMed] [Google Scholar]
- 67.360 Dust Analysis [Internet]. Macquarie University, Sydney NSW Australia [cited 2019 October 1]. Available at: www.360dustanalysis.com/
- 68.Filippelli GM, Taylor MP. 2018. Addressing pollution-related global environmental health burdens. GeoHealth. 2, 2-5. 10.1002/2017GH000119 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Citizen Science Project [Internet]. RMIT University, Melbourne, VIC Australia [cited 2019 October 1]. Available at: citizenscienceproject.org.au
- 70.Retka-Tidd M. Pancreatic cancer: Taking aim at the ‘silent killer’ online and overnight [Internet]. Garvan Institute of Medical Research and Vodafone Australia [cited 2019 October 1]. Available at: https://www.vodafone.com.au/red-wire/dreamlab-pancreatic-cancer
- 71.CrowdCARE [Internet]. The University of Melbourne VIC Australia [cited 2019 October 1]. Available at: crowdcare.unimelb.edu.au
- 72.Pianta M, Makrai E, Verspoor KM, Cohn T, Downie L. 2018. Crowdsourcing critical appraisal of research evidence (CrowdCARE) was found to be a valid approach to assessing clinical research quality. J Clin Epidemiol. 104, 8-14. 10.1016/j.jclinepi.2018.07.015 [DOI] [PubMed] [Google Scholar]
- 73.Quantified Self Institute [Internet]. Hanze University of Applied Sciences, Groningen, The Netherlands [cited 2019 October 1]. Available at: https://qsinstitute.com
- 74.Strava [Internet]. Strava.com [cited 2019 October 1]. Available at: www.strava.com
- 75.Lupton D, Pink S, Heyes LaBond C. 2018. Personal Data Contexts, Data Sense, and Self-Tracking Cycling. Int J Commun. 12, 647-65. [Google Scholar]
- 76.Synnot A, Bragge P, Lowe D, et al. 2018. Research priorities in health communication and participation: international survey of consumers and other stakeholders. BMJ Open. 8(5), e019481. https://bmjopen-bmj-com.ezp.lib.unimelb.edu.au/content/8/5/e019481. 10.1136/bmjopen-2017-019481 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Fiske A, Del Savio L, Prainsack B, Buyx A. 2018. Conceptual and Ethical Considerations for Citizen Science in Biomedicine. In: Heyen N, Dickel S, Brüninghaus A(eds). Personal Health Science. Wiesbaden: Springer, 195-217. [Google Scholar]
- 78.Resnik DB. 2019. Citizen Scientists as Human Subjects: Ethical Issues. Citizen Science: Theory and Practice. 4(1), 11. Available at: https://theoryandpractice.citizenscienceassociation.org/articles/10.5334/cstp.150/
- 79.Heigl F, Kieslinger B, Paul KT, Uhlik J, Dörler D. 2019. Opinion: Toward an international definition of citizen science. Proc Natl Acad Sci USA. 116(17), 8089-92. doi:. 10.1073/pnas.1903393116 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80.Riesch H, Potter C. 2014. Citizen science as seen by scientists: Methodological, epistemological and ethical dimensions. Public Underst Sci. 23(1), 107-20. 10.1177/0963662513497324 [DOI] [PubMed] [Google Scholar]
- 81.Gabrys J, Pritchard H, Barratt B. 2016. Just good enough data: Figuring data citizenships through air pollution sensing and data stories. Big Data Soc. 3(2), 1-14. 10.1177/2053951716679677 [DOI] [Google Scholar]
- 82.Kelling S, Fink D, La Sorte FA, Johnston A, Bruns NE, et al. 2015. Taking a ‘Big Data’ approach to data quality in a citizen science project. Ambio. 44(S4), 601-11. 10.1007/s13280-015-0710-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Kosmala M, Wiggins A, Swanson A, Simmons B. 2016. Assessing data quality in citizen science. Front Ecol Environ. 14(10), 551-60. 10.1002/fee.1436 [DOI] [Google Scholar]
- 84.Strang L, Simmons RK. 2018. Crowdsourcing-for-systematic-reviews. Cambridge: The Healthcare Improvement Studies Institute. Available from: https://www.thisinstitute.cam.ac.uk/wp-content/uploads/2018/06/THIS-Institute-Citizen-Science-Crowdsourcing-for-systematic-reviews-978-1-9996539-1-0.pdf [Google Scholar]
- 85.Long CR, Purvis RS, Flood-Grady E, Kimminau KS, Rhyne RL, et al. 2019. Health researchers’ experiences, perceptions and barriers related to sharing study results with participants. Health Res Policy Syst. 17(25). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 86.Research Australia. 2019. Australia Speaks! Research Australia Opinion Polling for Health and Medical research. Available at: https://researchaustralia.org/reports/public-opinion-polling-2/
- 87.Srinivasan U, Rao S, Ramachandan D, Jonas D. 2016. Flying Blind: Australian Consumers and Digital Health. Australian Health Data Series. vol. 1. Sydney: Capital Markets Cooperative Research Centre. Available from: https://flyingblind.cmcrc.com/files/files/Flying-Blind--Australian-Consumers-and-Digital-Health.pdf [Google Scholar]
- 88.Aramoana J, Koea J and on behalf of the CommNETS Collaboration. 2019. An Integrative Review of the Barriers to Indigenous Peoples Participation. Journal of Global Oncology. 5; JGO.18.00156. [DOI] [PMC free article] [PubMed]