Online Community and Citizen Science supports environmental science learning by young people

Abstract

Community and citizen science in online settings could be seen as a means for young people to engage with and contribute to authentic science. Yet, there is a limited understanding of who takes part in citizen science among young people, what they learn, and through which processes, particularly in online settings. In this exploratory study, we analysed 34 in-depth interviews and log files of young people aged 11–19 years old who took part in citizen science projects, hosted on the Zooniverse platform. Data analysis suggested that participation in online citizen science can bring environmental science learning benefits to young people, with some participants reporting evidence of agency with science, highlighted by taking action to do science in another context. Many participating youth exhibited substantial previous science experiences that helped them to take part and learn from citizen science projects. Considering findings from this study, we present a first working framework of how environmental science learning is enabled or hindered by certain types of participation, as a means to guide the design of online citizen science for young people. We recommend that the future project design, publicity and recruitment in online citizen science activities explicitly target the needs and interests of young people with diverse characteristics and competencies to truly open science to all.

1. Introduction

Engagement with authentic science or authentic scientific research has the potential to change science attitudes and cultivate the next generation of scientists (Broder et al., 2018; Puslednik & Brennan, 2020). Such opportunities are often lacking in youth education from school science to university STEM careers as many experiences that could be perceived as authentic, such as traditional laboratory experiments, are artificial in the sense that learners are guided through the scientific process to reach a predefined, known, and “correct” outcome (Crawford, 2015). Community and Citizen Science (CCS), which we define as inclusive of the range of participatory approaches to science including scientist-led citizen science and community-led community science (Ballard, Dixon, & Harris, 2017; Herodotou et al., 2017), is an opportunity for young people to engage with and participate in authentic or real science and experience the scientific process from the perspective of the professional researcher. It can support inquiry learning processes (Herodotou et al., 2017) and make science relevant to young people especially when focused on community-related issues (Jenkins, 2011). CCS may transform learning by enabling youth to alter current ways of thinking about themselves with science, recognising themselves as “valid, competent, and knowledgeable actors” (Ruiz-Mallén et al., 2016, p. 532). It can help break stereotypical conceptions of science as a subject area that is hard to understand and pursue as a career (Hellgren & Lindberg, 2017) and scientists as people who wear white coats and run experiments in the lab (Archer, Dawson, DeWitt, Seakins, & Wong, 2015). An understanding of science and its importance by young people can not only lead to undertaking STEM careers, but also and foremost, enable youth to manage and process the abundance of information in society, structure their thinking processes, and make informed decisions (Royal Science, 2020).

In addition to the unique learning benefits CCS activities can bring to learners, online CCS was shown to facilitate practical science learning from a distance by connecting learners and educators during the Covid-19 pandemic (Van Haeften, Milic, Addison-Smith, Butcher, & Davies, 2021), showcasing its potential to enable sustainable forms of education in the face of adversity. It thus becomes critical to gain an in-depth understanding of how participation in authentic science through CCS activities may influence young people’s learning including who amongst them are likely to enjoy learning benefits and who are likely to be excluded or left behind. Such knowledge would inform future CCS activities in both formal and informal educational settings and determine how these could be designed to scaffold learning and engagement with science of young people with diverse characteristics and competences.

Existing studies show that CCS programmes are mostly accessed and participated by white, middle or upper class, middle-aged, educated individuals (National Academies of Sciences, Engineering, and Medicine, 2018; Blake, Rhanor, & Pajic, 2020; Mac Domhnaill, Lyons, & Nolan, 2020). These volunteers are found to be highly concerned about the environment and believe that they need to act to protect it (Mac Domhnaill et al., 2020). Importantly, we know that CCS is not equally accessible to all, especially those with limited resources and historically underrepresented groups (Fiske, Prainsack, & Buyx, 2019). Younger individuals, and those with more diverse characteristics, are rather underrepresented in CCS (National Academies of Sciences, Engineering, and Medicine, 2018; Blake et al., 2020; Herodotou, Aristeidou, Miller, Ballard, & Robinson, 2020) and little is yet known about them.

While there is a lack of empirical research on youth learning in online CCS settings, the potential of CCS to support adult volunteers and their learning has been showcased in several other studies. Learning benefits are found to vary across field-based and online CCS projects, and include amongst others, increased content knowledge, development of data collection and pattern recognition skills, scientific literacy, and personal development (e.g. Aristeidou & Herodotou, 2020; Jennett et al., 2016; Peter, Diekötter, Höffler, & Kremer, 2021). Learning gains not related to science and the environment have also been reported such as improved health and well-being, connection to people and nature and a sense of satisfaction (Peter et al., 2021). These learning outcomes were shown to relate to, or be influenced by, factors, examined in this paper, such as volunteers’ previous science experiences, frequency of participation, and CCS project design features (Masters et al., 2016; Edwards & Simpson, 2016; Jennett et al., 2016).

In particular, differences in learning were observed between volunteers with higher levels of education as opposed to those with lower levels of education. The less academically qualified volunteers reported greater learning gains than the more qualified ones (Edwards & Simpson, 2016). In one study, particularly active volunteers were found to report increased topic-specific science knowledge, over and above their previous science knowledge (Masters et al., 2016), suggesting that the degree of engagement with CCS can enhance learning outcomes even for those with prior science knowledge and higher levels of education. Also, the design of CCS projects, in particular opportunities to actively engage and communicate about science, were found to support learning; certain types of participation including the completion of tasks, interacting with others, using external resources and project documentation, and sharing personal creations were shown to facilitate learning outcomes such as an understanding of project mechanisms, development of pattern recognition skills and scientific literacy (Jennett et al., 2016). Specifically, social elements were shown to relate to improvements in scientific literacy (Price & Lee, 2013). Yet, there is a recognized knowledge gap in our understanding of how to design CCS activities that are accessible and enable diverse young people to participate (National Academies of Sciences, Engineering, and Medicine, 2018; Blake et al., 2020) and connect personally with the proposed scientific research (Phillips, Porticella, Constas, & Bonney, 2018).

With regards to young people in particular, an emerging yet limited body of research aims to understand whether and how young people participate and learn from CCS programmes. In field-based settings, an examination of coastal and water quality programmes showed evidence of youth developing the learning outcomes of Environmental Science Agency (Ballard et al., 2017), including enhanced understanding of science and inquiry practices, youth identifying with scientific practices and roles, and acting in personally consequential ways with science to improve their own lives, communities, and the natural ecosystem. These learning outcomes resulted from certain activities youth had the opportunity to engage with, including an investigation of authentic socio-ecological systems, participation in rigorous data collection processes, and dissemination of project outcomes to external audiences (Ballard et al., 2017). In a similar study, learning was shown to occur when youth connected their participation to real science or the work of scientists, their own actions and their future identities, yet noting that not all youth reported learning benefits (Harris, Dixon, Bird, & Ballard, 2020). Differences in reported learning outcomes were influenced by several factors such as youth not believing or understanding that their data will be used by scientists, little understanding of the data collected and its potential impact on people and the environment, the framing and facilitation of the CCS task, prior individual experiences and community beliefs, and the degree of engagement with the community and scientists (Harris et al., 2020).

In online settings, few studies showed that young people of different ages take part in CCS projects (Herodotou et al., 2020) and their contributions have the potential to support authentic research in biodiversity (Aristeidou et al., 2021). In Zooniverse mainly 16–19 years old were found to take part in projects, yet sporadically and compared to adults, in limited numbers (Herodotou et al., 2020). Their contributions on Zooniverse, as well as other CCS platforms including iNaturalist, followed the pattern of activity seen in many volunteering projects; the majority of them contributed a few tasks, while only a few of them were particularly active. In terms of the potential of online CCS to support learning, a study with school children in a physics class showed changes in science ideas about, for example, data observation and correlation after engaging with Zooniverse (Straub, 2020). Similarly, a survey of 64 young people on Zooniverse revealed evidence of enhanced understanding of science processes, increased confidence and performance, changes in science perceptions, and development of new roles after taking part in Zooniverse projects (Herodotou, Ismail, et al., 2021). No science attitude or belief changes were reported, nor any plans to take action and support the environment. Learning outcomes were stronger for youth with significant prior science knowledge, experiences, and perceptions such as talking about science out of school, visiting online science platforms and physical places such as museums and going outdoors (ibid).

The rather limited understanding of young people’s learning and participation in CCS warrants further and systematic investigation to identify who are the young people who take part and report learning benefits from online crowdsourcing CCS, what these benefits look like, and how learning is facilitated or inhibited by the design of CCS projects and the particular types of participation in science that youth engage in. Such insights should help educators, practitioners and researchers to reconsider the design and delivery of CCS programmes and identify ways that CCS can promote specific learning outcomes and cater for youth with diverse characteristics, experiences and limited prior science experiences. In this exploratory study, we interviewed 34 self-selected young volunteers (11–19 years old) who participated in CCS projects, and analysed their log file data tracking their participation on the online platform Zooniverse. Data analysis answered the following research questions (RQs):

• RQ1: What are young people’s self-reported science learning outcomes, specifically environmental science agency, from taking part in Zooniverse projects?

• RQ2: How do different types of participation relate to self-reported learning outcomes?

• RQ3: How does participation, as captured by log files, relate to young people’s self-reported learning outcomes?

• RQ4: Who are the young people who are more likely to report learning benefits from participation in Zooniverse?

This study is part of a four-year international research collaboration - LEARN CitSci (Learning & Environmental Science Agency Research Network for Citizen Science) (https://education.ucdavis.edu/ccs-learn-citsci) - between three natural history museums in the UK and the US, and three universities: The Open University, University of Oxford and University of California Davis. The aim of the project was to understand how young people take part and learn from online and field-based CCS programmes, in order to provide recommendations to practitioners, educators and researchers as to how to improve the design of CCS programmes to cater for young people’s needs and enhance youth learning outcomes. This paper reports on findings from young people’s participation in online settings only, and in particular CCS projects hosted on the Zooniverse platform (see section 3).

2. Theoretical frameworks

Applying learning theories to CCS research contributes beyond conceptually guiding a stream of work; it help us unveil the grounding of individual learning within cooperative and exploratory environments (Hajibayova, 2020). To understand and examine young people’s learning in online CCS settings, we adopted the framework of the Environmental Science Agency (ESA) proposed by Ballard et al. (2017). ESA builds on the concept of critical science agency (Basu & Calabrese Barton, 2009; 2010) which postulates that young people draw from content-specific knowledge and their own expertise to bring about change in themselves and their communities. ESA consists of three aspects: (a) understanding of environmental science content and processes associated with that content; referred to here as ESA1, (b) development of roles or recognition of their own expertise within environmental science; referred to here as ESA2, (c) using environmental science knowledge and practices as “a foundation for change”, shown when youth plan to enact, or enacted, changes beyond the project participation; referred to here as ESA3. The ESA framework views learning as change in knowledge, skills, and attitudes or perspectives, with an emphasis on how participation in environmentally-related activities can enable development of identity and agency with science. The ESA framework has been used for CCS programs in both formal and informal learning settings in which young people were found to exhibit different aspects of ESA within different settings or programmes (Ballard et al., 2017; Harris et al., 2020). In this paper, the ESA framework informed the design of the interview protocol, our analysis including an associated codebook, and the organisation and presentation of self-reported learning outcomes.

In addition to the ESA framework that guided the design of the study, the process of data interpretation was also informed by Self-Regulated Learning (SRL) theories, referring to self-generated thoughts, feelings and actions for the implementation of a personal goal (Zimmerman, 2000). Given the absence of a teacher or a formal teaching structure, Zooniverse could be seen as an informal learning context that promotes SRL by allowing volunteers to take charge of their own learning i.e., identifying their own learning needs, setting up personal learning goals and learning strategies, and evaluating their own performance and outcomes (Loyens, Magda, & Rikers, 2008). Different technologies and tools have been designed to support SRL in online settings, some of which provide direct instruction as to how to regulate learning while others aim to promote SRL while completing learning tasks (Broadbent, Panadero, Lodge, & de Barba, 2020). Key to supporting SRL are design characteristics including: a) feedback that can reveal errors, communicate information and recommend strategies, and b) self-assessment tools such as guidelines, prompts, exemplars as well as algorithms that detect learners actions and prompt certain responses, c) support in the form of help, hints and automated feedback, and d) fading of scaffolding and support over time and after learners have been trained by e.g., deactivating feedback, hints (Roscoe, McNicol, Raghav Bhat, & Craig, 2020).

Visualisation and interaction tools such as graphs, progress bars and networks showing progress towards learning objectives were shown to positively influence motivation in online SRL whereas social comparison components impact engagement and time management (Pérez-Álvarez, Maldonado-Mahauad, & Pérez-Sanagustín, 2018). The use of learning analytics has provided innovative insights to understanding online SRL. For example, specific patterns of actions were identified while learners were studying a MOOC, including search for certain information to deepen understanding or answer questions, exploring content and assessment material following the proposed learning design, exploring new content only and discussing it in a forum, and engaging with assessment activities such as quizzes and homework. These patterns of actions were shown to relate to specific groups of learners; for example, a low performing group was most frequently engaging with content-oriented actions, whereas a high performing group with content and assessment actions (Fan, Matcha, Wang, & Gašević, 2021). The extent to which online CCS projects on Zooniverse enable forms of SRL i.e., volunteers take an active role in defining their science learning will be discussed in this study.

The last framework we utilised to understand how different types of participation related to self-reported learning outcomes is the theory of Technology Affordances from the field of Human Computer Interaction (HCI). The theory, originally proposed by Gibson (Groulx, Brisbois, Lemieux, Winegardner, & Fishback, 2017) and elaborated by Norman (1988) and Kaptelinin and Nardi (2012), refers to the actionable possibilities that can be enabled or allowed by a tool or technology. Technology affordances can be ‘canonical’ (Costall & Richards, 2013), for example, the canonical affordance of an online tutorial is to guide the user to completing a task. Canonical affordances can be customised or ‘appropriated’ by users, for example a user is searching for guidance on the web rather than using the proposed tutorial guide. Technology affordances can be adjusted or defined by users’ needs; they can enable or constrain actions, yet user choice or agency is what will determine how a technology will be used and how canonical affordances as defined by technology designers will be modified and repurposed (Tucker et al., 2016). In this paper, the theory of Technology Affordances assisted in conceptualising and analysing how different types of participation relate to specific design features of Zooniverse projects and how participation types enabled certain forms of learning for some participants while inhibiting these for others.

In the above theories, learners are viewed as having an active role in a process of learning that is experiential and contextual. Theories focused on learning through participation, and learning as participation, from socio-cultural learning theories such as situated learning and the concept of Legitimate Peripheral Participation (Lave & Wenger, 1991), and as far back as Vygotsky (1978), highlight the ways in which interest-driven participation and engaging in the legitimate practices of meaningful activity can not only help a person gain new knowledge and skills, but also develop new identities and roles around those practices. In the context of science practices, CCS activities provide a potentially optimal context in which particularly young people can become aware of and engage in real science practices that may foster development of identities with science as well as other forms of science learning (Ballard et al., 2017; Herodotou, Scanlon, & Sharples, 2021). We therefore examined in this study the relationship between different types of participation in CCS by young people and their development of different manifestations of environmental science agency.

3. Methodology

3.1. Research context and participants

Zooniverse is a crowdsourcing CCS platform that hosts “contributory” projects (Shirk et al., 2012); in this type of CCS projects, volunteers process data already collected and shared by scientists. The platform hosts four different types of projects: (a) answering a question after observing an image, (b) adding free text such as keywords to describe an image, (c) transcribing or marking an image, and (d) identifying species within an image from a given list of options (Herodotou et al., 2020). A guide, called tutorial, detailing how to classify or transcribe a series of images is often presented to volunteers before they initiate a task. A forum functionality (named “TALK”) allows volunteers to communicate with others including scientists. The “Build a project” functionality can be used to create a new project and it is available after registering with the platform. As standard on Zooniverse projects, participants are not given immediate feedback on whether the classification they have provided is correct or not, as almost by definition, the correct answer is not known yet. There are a handful of Zooniverse projects that use a small amount of gold standard data to provide this kind of feedback. This has proved popular with participants, who are keen to understand whether they are doing the task correctly or not, but it is difficult for most projects to implement this functionality. Overall, Zooniverse could be seen as an informal science learning environment structured around asynchronous and written interactions and communication and mediated by (a) a number of tools including different tasks, tutorials, and a forum, and (b) people including volunteers and researchers, the latter being those designing online CCS projects using platform tools and communicating with volunteers.

Aligning with the research objectives of the LEARN CitSci project, we identified and interviewed young people (aged 11–19) who took part in Zooniverse projects. To identify participants, we emailed volunteers via the Zooniverse mailing list. Interested parties followed an email weblink, accessed an information sheet (briefing participants about the aim and procedures of the study), and completed an online form giving their consent to take part (should they be over 16 years old), or provide their guardians’ contact details (should they be younger than 16 years old) for researchers to confirm consent over the phone or via email with guardians. The study received ethical clearance from the Open university UK. Out of the 34 participants who took part in the interviews, 20 were female and 14 were male. In terms of age, eight were aged 11–13 years, and 26 were aged 14–19 years.

3.2. Research model and procedure

We followed a convergent parallel design to combine qualitative and quantitative research data, (Edmonds & Kennedy, 2017). This study is underpinned by the interpretative perspective, integrating ontological and epistemological assumptions that frame reality as multiple socially constructed realities (Merriam & Tisdell, 2016). The contrasting ontological and epistemological assumptions of the qualitative and quantitative paradigms may be seen as conflicting under mixed-methods research (Twining, Heller, Nussbaum, & Tsai, 2017). Hence, we use numerical and non-numerical data for an interpretative and complementary understanding of learning and participation. The need to employ both types of data in this study stems from the nature of online CCS projects and how we can access user information in online platforms, which often produce numerical information that is open to interpretation. Furthermore, the reporting of the study followed principles of good qualitative research as described in Twining et al. (2017) evidenced in aligning RQs across the different parts of the research design, explicitly describing the underlying theoretical frameworks we used for data collection and analysis, detailing processes of data collection, critical engagement with data and interpretation, and reflection on limitations.

Qualitative data in the form of in-depth semi-structured interviews were used to answer RQ1 to RQ3, whereas log files extracted from the Zooniverse platform documented participants’ engagement with the platform and were used to answer RQ4. These data sources enabled us to collect complementary datasets and answer all four RQs. Data were collected individually and analysed consecutively. Interviews took place online via Skype and were audio recorded. Parents or guardians of participants younger than 16 years old were asked to listen into the interview, yet avoid any interruptions of the process. The interviews lasted between 30 and 45 min with an average duration of 40 min. Two of the participants requested to share their answers in writing via email, and therefore they completed a written version of the interview protocol. Interviews were conducted by the second and third authors. They followed a predefined semi-structured interview protocol designed and piloted for the purposes of the entire LEARN CitSci project to enable comparisons between online and field-based settings, yet with additional questions addressing the specifics of online settings. Interview questions aimed to address the three aforementioned RQs, in particular to capture information about participants’ own perceptions of learning after taking part in Zooniverse projects (RQ1), how the design of the projects they took part in enabled or inhibited certain forms of participation and learning (RQ2), and detail what the characteristics of young volunteers that took part in Zooniverse projects are likely to be (RQ3). We drew from the theory of the Environmental Science Agency (ESA) to structure and present self-reported learning outcomes (RQ1). Example interview questions include: (a) (RQ1): Do you think by doing activities on Zooniverse, you got better at learning science or doing science? How so? (b) (RQ2): What do you like about doing projects on Zooniverse? Is there anything you do not like about Zooniverse? (c) (RQ3): How did your previous knowledge in your studies help you to learn using Zooniverse?

Log files of interviewees were extracted from the Zooniverse platform capturing data from the date participants registered with the platform to August 2018 or March 2020 (depending on whether participants were recruited in Year 1 or Year 2 of the project). Log files provided information about the actual usage of the platform by interviewees including their number of contributions, their activity ratio and the mean time spent on completing a task on Zooniverse.

3.3. Process of data analysis

To analyse interview data, we followed principles of thematic analysis (Broadbent, Panadero, Lodge, & de Barba, 2020). We adhered to a predefined code-book that was developed to address the aims of the LEARN CitSci project as a whole and capture learning and participation across field-based and online settings. The code-book was the outcome of a process of coding a number of interviews across multiple coders, identifying codes and subcodes (in this case, the three main aspects of ESA and the sub-categories we saw as different manifestations of each aspect) and describing each code to correspond to both the field-based and online settings. To maximise the reliability of coding, we followed two iterations of a blind-coding exercise followed by a focused discussion on the coding differences between analysts, documenting agreements and decisions. Subcodes or secondary codes were applied to specific actions or behaviours, whereas main or primary codes brought together subcodes with similar qualities. For example, the subcodes of “perceptions of how citizen science works” and "youth’s perspective on or experience with data” were considered evidence of two different manifestations of ESA 1 and grouped together under ESA1 to denote instances of one’s understanding of science content. The first round of coding was conducted by Author 2. Example quotes of each sub/code category were reviewed by Author 1 and agreed on. In cases there was a disagreement as to what a quote may indicate, we reviewed the relevant interview transcript and made a decision considereing the case of the individual participant.

The audio interviews were transcribed by an online professional company. We used the online qualitative analysis tool, Dedoose, to code the interview transcripts, identify relationships between different constructs, and get an indication of how prominent (or nor) certain codes were in the data. To identify and represent in a systematic manner patterns between different constructs (e.g., relationships between different types of ESA, relationships between participation and learning), we followed the tabular data representation of Harris et al. (2020). Tables were extracted from Dedoose after all 34 interviews were coded. The numerical frequencies of each code were replaced by colours to denote a general distinction between codes frequently and less frequently observed across the 34 transcripts.

To analyse log files and identify how participation metrics relate to ESA outcomes, we quantified ESA manifestations as captured in the interview data for each participant, by denoting whether a manifestation was present (coded as 1) or absent (coded as 0) in each interview script. We then produced a summary score per ESA aspect (i.e., ESA1, ESA2, ESA 3) and for all three aspects for each participant. Due to the small and non-normal sample distribution, non-parametric tests were applied to identify any relationships between participation metrics and ESA manifestations.

4. Results

4.1. Youth self-reported learning outcomes from taking part in zooniverse (RQ1)

Young participants reported learning outcomes in support of all three main aspects of ESA: understanding of environmental science content and processes associated with that content (ESA1), development of roles or recognition of their own expertise within environmental science (ESA2), and using environmental science knowledge and practices as “a foundation for change”, shown when youth plan to or enact changes beyond the project participation; (ESA3). Yet, manifestations of ESA1 and ESA2 were more frequent in our data than ESA3. Fig. 1 shows the relative frequency of the main aspects and the sub-categories we identified and coded as manifestations of each aspect of ESA across the 34 participants. Our data showed that all of the participants reported evidence for at least one of the ESA manifestations. Fig. 1 shows how some manifestations of ESA were strongly or more frequently found in our data than others, such as sharing knowledge (ESA2), developing science competence and performance (ESA2), understanding how CS works (ESA1), drawing on prior knowledge (ESA1), gaining a new perspective (ESA2), and developing scientific knowledge (ESA1). Less frequently encountered ESA manifestations were a display of scientific thinking (ESA1), youth perspective with data (ESA1), transformation of practice (ESA3), plans to use science in another context (ESA3) and ownership (ESA2). To provide evidence and richness for each of the manifestations of ESA we examined, Table 1 presents a definition for each alongside example quotes.

Fig. 1. Number of young people reporting each ESA manifestation.

Table 1. ESA manifestations, a definition and example quotes for each one.

ESA1:Environmental Science Content and Science Practice, and Norms of science
ESA	Definition	Quote
Norms of science	Participant talks about norms or rules or protocols that need to be followed, whether for collecting or reporting data or location (such as a lab or a place similar to scientific setting	“To my understanding with Penguin Watch, CS it to have a lot of volunteers go over the same images and count the number of penguins in each image, and then they use that data to figure out breeding and population patterns throughout the season.” (Learner 3 - F – 18)
Scientific skills/using the tools	Participants describe their own engagement with any stage of scientific research	“On Zooniverse, I learn about penguins. I had to count penguins and mark adult penguins, baby penguins and other animals on the photo.” (Learner 16- M – 11)
Draws on prior knowledge and science skills	Participant connects prior experience and content knowledge and skills (scientific practice) to the current learning environment	“I’m part of a youth birding club at a park nearby my home, so I think that really helped because I know about these birds in a way, and it’s interesting to learn a little bit more about them.” (Learner 27- M – 12)
Scientific knowledge & Environmental Science Content	Participant talks about learning new content, adding to their science knowledge	“I’m not sure of the name in English but it’s a kind of [Zooniverse project] about birds that was in Iceland and evolved through the years of drought. I just thought it was quite related because that one was a big Iceland data study, and this was as well. I just went with that first then I branched out.” (Learner 19- F – 18)
Perception of how Citizen Science works	When participants explain about how Citizen Science works or how online CS programs like Zooniverse work	“My contributions after I submit them get looked over by scientists” (Learner 38- F – 11)
Youth’s perspective on or experience with DATA	Connecting observation and fieldwork to Data	“I’m sure it’s a bunch of different people viewing those images and putting in their own opinion on what it is that they see. Then that gets filtered through to see which one is accurate. I’m sure it’s a population count because the ones I do, which is for the wildlife, those more relate to population numbers distinguishing their behaviour and knowing how much of certain individuals are in the population like adults and females versus the children.” (Learner 5 - F – 18)
Displays of scientific reasoning	ESA provides context for learners to engage in scientific enquiry toward specific social purposes	“I would say, obviously with the practicing it and with the herbarium specimen that I was getting faster and better with it. I guess noticing the smaller details of certain things and just being able to discern what certain things were saying and reordering things, and stuff like that.” (Learner 11 - M – 19)
ESA2: Identification of roles in the practice of science
ESA	Definition	Quote
Ownership	Youth developed ownership within the scientific process through figuring out how to collect and analyse data to answer their questions	“I ensure that the counts producing were accurate and using that knowledge of identifying which penguin was a chick and which was not. Sometimes you’d have to use the field guide to know the season that the penguins were in and whether that meant that there were chicks that season or whether there were eggs that season. Just having to apply that outside knowledge made it more than just a rote’s identification task, I made it feel more like doing actual science. (Learner 3 - F – 18)
Shares knowledge/ expertise/teaching others	Became a content expert and took on more responsibility for communications to outside audience	“I told school friends that I enjoy using it and what I do on there. They seemed to be quite interested and I think they checked it out.” (Learner 24- M – 12)
Takes on Roles or developing new role	Youth find specialized roles within the project and become an expert or resource to the group for that topic	“ I could intuitively understand which one they’re asking me to look for. It’s also easier for me to pick out which behaviour an animal’s doing because I’ve seen examples of it in the past.” (Learner 21 - F – 16)
Science identity - Performance	Participants say that they got better at something	I feel confident to find new project on Zooniverse.[..] I got better at looking for details. (Learner 4- M – 13)
Science identity - Competence	Understanding the topic and becoming comfortable engaging in	“Taking part in Zooniverse raise questions …. I’ll go and keep track and see what planet they’ve identified as a result of this and after everything … It’s probably outside of Zooniverse that I look at research article, on Google or something.” (Learner 33 - F – 17)
Science identity - Recognition by others	Being recognized by others as a good “scientist”, using tools and hypothesizing solutions/ideas	“I’ve had been pulled up a couple times and friends have asked me questions about it so I have answered them. Yes, I basically told them what the platform was and what the projects I was doing were about.” (Learner 33 - F – 17)
Increase Value or Gaining new value	Participants talk about gaining a new perspective by valuing or taking an interest in science	“With my participation in Zooniverse I learned some new things. This is the most important thing in science, maybe learning. I learned for example, how photometry works with the exoplanets, or how it feels to do research in a practical way, in practice.” (Learner 22 - M – 18)
ESA3: Development of a sense of agency
ESA	Definition	Quote
Desire to become some type of scientist	Participants envisioning themselves to become a scientist and performing in any type of science	Next year I’m going to university and I want to pursue a life science major, so I think science is going to be a part of my life that’s going to stay outside of school …. Zooniverse reinforced that specific idea of how important data and the work of gathering data is to science […]Zooniverse really lets you be a part of that initial data gathering stage, lets you see how those conclusions were reached and just how much work is needed to do that research.” (Learner 3 - F – 18)
Taking initiatives to do/help/ contribute to science	Participant takes to start a new project, make a new observation, use the tools outside of the program or use a platform (i.e. Zooniverse)	“ Zooniverse was useful. I use it in my degree currently in the university. I was learning the different kinds of animals out there. That’s broadening my knowledge of what I can teach to children too.” (Learner 30 - F – 19)
Describes plans to use (scientific) practice in another context	An action or intention a youth demonstrates to use the tools or/and scientific practice learned in the program in another context	I’ve always been interested in some kind of science and Zooniverse really helped me to find out which part of science I was most interested in.[…] Zooniverse is making me searching a bit more about astronomy […] I write a blog about Astronomy (Learner 24-M – 12)
Transformation of practice as a result of agency (generative)	Change of practice within a platform (e.g. level of expertise in using the platform)	“I was doing the animal ones and the science one or the space ones at the same time. I was never a big fan of transcribing because I can’t read handwriting, but then I found notes from nature which most of them are typed out so I could actually transcribe them a lot easier. I’ve just been doing them. [ …. ] the cyclones and the Caribbean projects, [transcriptions] helped from that. Actually went in and looked at all the reports and what they’ve done about it and how they use that information that we provided for it.” (Learner 30 - F – 19)

More than one manifestations of ESA was often identified across most of the participants. Hence, we examined the relationships between the different aspects of ESA (i.e., ESA1, ESA2, ESA3) and the different manifestations of ESA in order to understand how ESA manifestations relate to each other. Our analysis revealed both intra (within) and inter (across) relationships between the different manifestations of ESA. Intra-relationships refer to a participant mentioning more than one manifestation of ESA that belongs to the same aspect (e.g. ESA1) (See Table 2). For example, a description of plans to use (scientific) practice in another context (ESA3) was found to co-occur with evidence showing a transformation of practice as a result of agency (ESA3), whereas the development of scientific skills in using tools (ESA1) was co-present with the development of scientific knowledge and environmental science content (ESA1). These relationships indicate how certain learning outcomes may develop in conjunction when taking part in projects on Zooniverse and that certain ESA manifestations could be analysed and understood together.

Table 2. A matrix showing intra relationships between ESA manifestations.

ESA1	Scientific Knowledge & Environmental Science Content
Draw on prior knowledge and science skills
Scientific skills/using the tools
ESA2	Gaining new value	Science identity - Competence (Self efficacy and confidence)
Science identity - Competence (Self efficacy and confidence)
Science identity - Performance (Gaining expertise and develop skills in using the tools)
ESA3	Taking initiatives to do/help/contribute to science or other areas i.e. environment, local community	Transformation of practice as a result of agency
Describe plans to use (scientific) practice in another context
Taking initiatives to do/help/contribute to science or other areas i.e. environment, local community

^*In green, frequent relationships in the 34 interviews

^*in yellow, less frequent relationships.

Inter-relationships refer to the co-presence of manifestations of ESA across different aspects in a single participant. We found evidence in support of all combinations of aspects, that is, ESA1 and ESA2, ESA1 and ESA3, and ESA2 and ESA3. Our data showed that in some cases learners were able to connect prior experience and content knowledge and skills (scientific practice) to the current learning environment in Zooniverse (ESA1). A young person explains how prior experience has led to increased confidence in understanding the topic and becoming comfortable engaging in science (ESA2). “Having the familiarity with the subject matter [via Zooniverse]. I previously understood a lot of the animal species they were asking me to classify in the Amazon because I had seen documentaries and films with their behaviours and such, so it [Zooniverse] made it a lot easier for me to distinguish between two species of tapir or two species of bird where I could intuitively understand which one they’re asking me to look for. It’s also easier for me to pick out which behaviour an animal’s doing because I’ve seen examples of it in the pas” (Learner 21-16-F).

In the example below, another participant narrates their participation in Zooniverse, as an ongoing learning experience. They recognized their gained content knowledge about lights (ESA 1) and identified themselves as “helper” by activating their knowledge and helping others to understand (ESA2). Taking initiative to use the tools outside of the program to do research and help people to understand better about CS topics manifest how youth has enacted agency (ESA3). “I read up on articles on Zooniverse, I get to actually do a hands-on thing and learn about it through doing it. I’ve gotten better at understanding different light curves for that one project with the Variable Stars. I’m better at seeing where there’s just a mistake in how the computer saw something rather than an actual planet there in the Planet Hunter Project.[…] One day I was for hours just doing all of these classifications on one of them and I went, “Wow, I know this stuff really well.” I was pretty happy because I’m like, what I’m doing here is going to be helping out people understand this stuff. My research is going towards something” (Learner 5-18-F). This participant shows evidence of self-regulated learning (SRL) by setting and implementing her own goal of completing a number of classifications for a project and then evaluating their learning by noting significant improvements in her classification skills of light curves.

The analysis of our data showed in some cases that science knowledge (ESA 1) and confidence (ESA 2) develop in parallel. The following example also showed that academic perceptions about science changed after taking part in Zooniverse: “I did one project that was based on natural sciences, and I felt smart afterwards. I’m not very good at science academically, but I feel like I’ve just learned something, and I built on the very minimal-type knowledge I have already.“ (Learner 32-18-F). Another 19 years old participant explains how increased science knowledge had an influence on their professional practice (ESA1 – science knowledge & ESA3 – influencing practice in another domain): “It was useful. I use it in my degree currently in university. I was learning about the different kinds of animals out there. That’s broadening my knowledge of what I can teach to children, as [Early childhood is my first degree] I’m going, “Hey, there’s not just Australian animals, there’s all these other animals that are actually in the world as well that I have some kind of a knowledge about as well." (Learner 30-19-F). This participant shows evidence of regulating their learning by setting up personally relevant learning goals, that of improving their knowledge of animals, and participating in informal learning activities on Zooniverse to achieve that. The excerpt above also reveals reflection and evaluation of her engagement with Zooniverse, noting improvements in her knowledge of different animals. Another participant who perceived himself as confident in taking part in Zooniverse projects, explains how feeling confident resulted in taking the initiative to produce a script to analyse some of the collected data (ESA 2 – confidence & ESA3 – extending learning out of Zooniverse): “For the most part, I’d say pretty confident [in Zooniverse] I feel like 9 out of 10 on a scale of 10 [ …. ] I actually with the data that they’ve collected with [Zooniverse project name], I decided to write my own little script just to figure out what we could do with this data on just a basic level.” (Learner 18-17-M). These insights suggest that developing knowledge, skills and identities with science (ESA 1 and ESA 2) can enrich if not facilitate the development of agency with environmental science (ESA 3).

4.2. Types of participation and learning (RQ2)

The aforementioned manifestations of ESA were shown to emerge from certain learning processes or result from certain types of participation in Zooniverse. These types of participation took place either in the online context of Zooniverse or in other online and offline contexts. In this paper, participation is viewed as the learning process through which manifestations of ESA can be developed. Drawing from the theory of Technology Affordances (Groulx et al., 2017; Kaptelinin & Nardi, 2012), participation is defined by the design features of Zooniverse, while also “appropriated” by young people’s own actions including engagement with other resources and knowledgeable others. In particular, we observed three types of participation related to specific design features of Zooniverse i.e., doing a task, exploring, and communicating, that were implemented either within or out of Zooniverse, as follows:

• (a)Within Zooniverse participation refers to (i) task-related participation: completing project tasks. Tasks are data analysis activities defined by the design features of the platform. Reviewing the functionality of Zooniverse, a task includes a set of images (presented in a sequence) and a number of associated questions (fields) that should be answered before moving to a next image and repeat the process. Questions can be of four types: they are asking volunteers to: (a) observe an image and answer a question, (b) describe an image adding free text such as keywords, (c) transcribe or mark an image, and (d) identify species on an image from a given list of options. (ii) exploration and discovery: searching for and reading about projects on Zooniverse. Reviewing the Zooniverse functionality, this is enabled by design features including a search projects field, a filtering (e.g. by most active, most help needed, most popular) functionality, grouping of projects by topic (e.g., arts, biology, climate, history), a pop-up guide describing the task, text detailing task objectives, team members, FAQ, project background information, quotes from researchers, and (iii) communication via the Zooniverse forum (Talk) with other volunteers and scientists. Forum topics are organised around specific projects or more generic questions (e.g. project building, troubleshooting) and are accessible through the homepage or via a specific project. Forums posts can be tagged enabling key word search across posts. The within Zooniverse participation was enabled by the functionality of Zooniverse and specific design features including image marking, open-ended and closed question responses, searching, filtering, project grouping, text descriptions, and a Forum.
• (b)Out of Zooniverse participation refers to activities taking place outside the platform and included (i) exploration and discovery including searching for information online or offline related to Zooniverse projects (e.g., searching the internet, reading a book) and (ii) communication with knowledgeable others (e.g., parents, teachers), often to receive help and complete a task on Zooniverse. This form of participation was initiated by young people themselves and, in some cases, was shown to enable the development of ESA. It often co-occurred with other forms of participation within Zooniverse such as task-specific participation. Out of Zooniverse participation is seen as a form of design appropriation initiated and regulated by young people in an effort to scaffold their science learning such as find an answer to a question or get support to complete a project task. Design features enabling this form of participation were online and office resources and other knowledgeable actors.

Fig. 2 maps certain ESA manifestations with types of participation. Table 3 (See below) presents example excerpts showcasing these relationships. Task-related participation was shown to relate to a greater number of ESA manifestations as compared to other types of participation, suggesting that participation in project tasks is a central learning process through which ESA can be developed. Yet, certain manifestations of ESA such as the desire to become a scientist were not associated with task-related participation, suggesting that completing Zooniverse projects may not inspire youth to pursue a career in science. What was shown to facilitate a desire to become a scientist was Exploration inside Zooniverse. Exploration outside Zooniverse enabled, for example, youth to take on new roles and develop scientific knowledge. Communication with others in Zooniverse enabled, for example, enhanced performance and display of scientific reasoning. Communication out of Zooniverse enabled in particular plans to use practice in another context, taking on new roles, enhanced performance and scientific knowledge.

Fig. 2. A working framework of how different forms of participation enable certain manifestations of ESA.

Table 3. Types and context of participation in relation to ESA.

Type and context of participation	Example interview quotes for different types of participation and their relationship to different ESA aspects
Within Zooniverse: Task-related participation	Normally I just sit there and read up on articles on Zooniverse. I get to actually do a hands-on thing and learn about it through doing it.[…]I’ve gotten better at understanding different light curves for that one project with the Variable Stars. I’m better at seeing where there’s just a mistake in how the computer saw something rather than an actual planet there in the Planet Hunter Project."(Learner 26–17-F)(ESA1: science knowledge; ESA2: Science identity – Performance) Zooniverse definitely helped. When I first was starting to get an interest, I went to Zooniverse and it gave me inspiration to look on other websites, do my own research […]I get to see real data and I get to classify that real data and I actually get to be a part of the process."(Learner 24–12-M)(ESA3: Describes plans to use (scientific) practice in another context)
Outside Zooniverse: Exploration and discovery	“Zooniverse encourages me to do more research on the different species. Where it’s like, “Oh, I worked with this species but I don’t know it very well.” So I want to go look it up and see where it lives and what’s its lifespan and cool adaptation and things like that.“(Learner 9–18-F) (ESA2: Performance and gaining experience) “Zooniverse definitely helped. When I first was starting to get an interest, I went to Zooniverse and it gave me inspiration to look on other websites, do my own research […]I get to see real data and I get to classify that real data and I actually get to be a part of the process.“(Learner 24–12-M) (ESA3: Transformation of practice)
Within Zooniverse: Exploration and discovery	“The first thing I do when I first login to Zooniverse is search for all new projects or a project that I like and I do it for 20 min, half an hour, it depends on the time that I have. Every time I go to Zooniverse I try to learn something new or to focus on something that I didn’t know before.” (Learner 22–18-M) (ESA2: Gaining new value/interest) Co-occurring with Task-related participation:“When I’ve read something new within some of these projects, I like showing up with people going, “Hey, this is what this one is,.” or I did a whale one recently where we would classify different kinds of whales. I was going out to people saying, “If you look up this tail, this e kind of whale this is.” Actually all these other people learn different kinds of whale tails with me.“ (Learner 30–19-F) (ESA2: Sharing knowledge; ESA3: Develops a new role that of the “storyteller")
Within Zooniverse: Communication	“I was trying to figure out how to classify another project. I didn’t end up doing it because I just couldn’t understand. When I went to the “Talk” section, people put it more detail, so it was better understood” (Learner 24–12-M) (ESA2: Science identity-Performance) “I get emails from the various Zooniverse projects that I work on and I get to find out if they reached their goal or not or if they still need help and I think that’s really fun because then I get to- sometimes I’ll share their responses or links to their papers that they have submitted and that’s really fun because then I can study the research that I got to help with.” (Learner 9–18-F) (ESA3- Describes plans to use (scientific) practice in another context)
Outside Zooniverse: Communication	Co-occurrence with ‘Exploration and discovery’:“I came up with questions, I posted them on [Zooniverse] forum. I started Googling about it, I asked my physics teacher about them, to try and find out exactly the things I was actually identifying in the pictures there (Zooniverse]” (ESA2: Takes on Roles or developing new role) (Learner 26–17-F)

The development of some manifestations of ESA was enabled by all types of participation, for example, scientific knowledge (ESA1) was enabled by all types of participation (doing a task, exploring, communicating), enhanced performance (ESA2) by all types apart from exploring out of Zooniverse, and taking on new roles (ESA2) by all forms apart from a task-related participation. These insights have implications for the design of CCS activities and how these can enable specific learning outcomes for young people. These implications are presented in the discussion section.

In Table 3, we share example quotes that showcase how different types of participation enabled certain manifestations of ESA for the great majority of our sample. A few participants encountered difficulties when taking on certain types of participation in particular, doing a task and communication via the Forum, that constrained their ESA development. These participants sought support outside the platform by searching for information and communicating with others in order to complete a project task or, as they could not complete a task, they moved to a next one. In relation to task related participation, a participant explains how completing a task on Zooniverse has not led to any learning outcomes, as the participant could not understand the context of the research and the task itself “didn’t feel scientific”: “What we were doing was measuring things, labelling it and [finding] same colours of them. That in itself didn’t feel particularly scientific because it was more looking colour I guess. But that [is] because I don’t have the biological understanding to understand the scientific context of the research so they have that and maybe I show more scientific. We couldn’t understand the scientific context behind it.” (Learner 36-19-F).

Another participant explains the process of engaging with a task on Zooniverse making reference to how the guide tutorial was not as effective as expected: “I didn’t really understand what I was doing when I tried Zooniverse, so I had to go to the tutorial blog and on watching.[…] Probably saying like, you know how on YouTube there are playthroughs of games and such, maybe some like someone doing a classification while being recorded and learning how to do it, because I couldn’t understand what in the world I was supposed to do or what I was supposed to click”. Similar difficulties were raised by another participant who, commenting on the guide tutorial explained how this did not help them to understand how to take part in the project and therefore they had to quit and move to another project: “I don’t feel like the tutorial adequately prepares me for the actual classification, and so if I feel like I don’t have enough information to go off, I usually will not keep going on with that project because I don’t want to mess it up if I feel like I’m doing it wrong” (Learner 23-19-F).

Some participants commented in particular on the process of completing a task and noted difficulties in relation to the lack of feedback about task correctness: “It’s hard to use my own judgment as to whether the penguin was too far away to count because sometimes they’d be in the background really small and I wasn’t sure if I should label or if I should just leave it and say that there were too many to count.” (Learner 3 - F –18). Another participant comments on the same project: “I hope that I have marked every penguin but sometimes there is a full photo of penguins and it’s very hard to mark all of them. […] Sometimes, there is some dark spots or light spots and it’s really hard to understand what is it.” (Learner 10-F-19). Commenting on a different project, a young participant reiterates on being unsure about the correctness of the task they are completing: “Sometimes the handwriting is not legible to me and so those are ones that I’ve clicked out of because I just simply couldn’t read the text. […] I don’t feel like the tutorial adequately prepares me for the actual classification, and so if I feel like I don’t have enough information to go off, I usually will not keep going on with that project because I don’t want to mess it up if I feel like I’m doing it wrong.” (Learner 23 - F –19). Another participant explicitly states the need for feedback as to whether a classification is right or wrong: “For me, it’s mainly to get feedback on whether I classify the galaxy right or wrong, because sometimes I’m not sure about whether it is or it’s not. Then I’m quite afraid that I will get it wrong and then kind of mess up with the results […] I changed for something that’s clearer.” (Learner 29-F-17).

In relation to communication with others in Zooniverse, a participant explained how raising a question via the Zooniverse forum did not enhance their science understanding as the answer given was not understandable: “I asked a scientist on Zooniverse one question: what is the black hole. After that, I got a very big answer. I just understand a little bit but still, it is very nice.” (Inv-12-F). Another participant posted a question on the forum for which they never received a reply: “I have once asked one question [in TALK], but I didn’t get that answer […] I was a little disappointed but after that I saw that my father was just saying the answer, what is it. Still, I have a little bit of doubt on that question” (Learner 1-12-F).

4.3. Participation as captured by log files (RQ3)

Table 4 presents an overview of the participation metrics and number of ESA manifestations for 27 interviewees (log files for seven participants could not be identified). A large spread in the number of contributions is observed with a median value of 499 contributions. Similarly, the time spent on average to complete a contribution ranged considerably, with a median of 1 min and 38 s. The activity ratio suggested that on average participants visited Zooniverse five days every 100 days (since their registration), yet a few visited every day. Aligning with the aforementioned interview analysis, manifestations of ESA1 and ESA2 were more frequent than ESA3. A Spearman’s Rho test showed no significant associations between participation metrics and ESA manifestations, suggesting that the number of contributions (p = .723; NS), activity ratio (p = .694; NS) and mean time devoted on tasks (p = .210; NS) were not related to ESA aspects.

Table 4. Participation metrics extracted from log files.

Metric	Definition	N	Minimum value	Maximum value	Median	SD
Number of contributions	Number of classifications made by young people	27	1	9966	499	2849
Activity ratio	The ratio of days contributing at least one classification to the total days registered with Zooniverse. The closer to one, the more active a user is Average time devoted on completing a task in seconds	27	.01	10	.05	0.32
Mean time devoted	Average time devoted on completing a task in seconds	27	8.8	2699	98	641
ESA1	Aspect of ESA	27	2	6	4	1.12
ESA2	Aspect of ESA	27	3	7	5	0.96
ESA3	Aspect of ESA	27	0	4	1	1.15
ESA_all	Combined ESA aspects	27	7	14	11	2.05

^*It is noted that Zooniverse users can contribute to projects without being logged in the platform and these contributions cannot be captured by log files. Therefore, these numbers may not be accurate for some of the users.

4.4. Who are the young people who reported learning benefits (RQ4)?

A set of interview questions asked young people about their prior science experiences as a means to understand the reasons they participated in Zooniverse. Rather unexpectedly, we identified that the great majority of participants (n= 30) had significant prior science experiences that were influencing participation practices and self-reported learning outcomes. Prior science experiences were expressed in (a) prior science knowledge, (b) prior interest in science, and (c) prior experiences in working or visiting science places such as a science club, museum or institute. Young people who took part in this study were rather oriented towards science before they took part in Zooniverse. Youth’s prior experiences explained (i) participation in Zooniverse, (ii) facilitated completion of certain tasks, and (iii) enabled certain manifestations of ESA. The below examples illustrate these points.

• (a)Prior science knowledge: A participant is explaining how their prior science knowledge worked as “a little stepping stool” for understanding how a space project worked and how to contribute: “I think [previous science experiences] did. I think it kind of gave me a little stepping stool for some of them. Some of the space ones, I had prior knowledge of how they worked and what sort of things that I was already looking for.” (Learner 30-19-F).
• (b)Prior interest in science: A participant explains how their science interest and “love”, shown in reading science articles and an intention to study Zoology enabled certain learning outcomes, including enhanced science performance and topic-specific knowledge (ESA1: Science knowledge, ESA2: Science identify - performance): “Since I was in preschool I’ve loved science […]. When I started using Zooniverse it was Space, […] normally I just sit there and read up on articles on Zooniverse, I get to actually do a hands-on thing and learn about it through doing it.[…] For National Geographic, it’s more about reading the articles that are already made as opposed to Zooniverse where I can take part in the research and I can help and get experience with training my eyes because I do want to go into zoology. For me, this is a really great opportunity to be able to practice those skills” (Learner 5-18-F).
• (c)Prior experiences in working or visiting science places (museums, clubs etc): A young person with previous experiences of working in a research lab and performing similar science tasks explains how tasks on Zooniverse felt familiar and easy to perform: “Last summer, I interned in a research lab where we were studying Drosophila fruit flies, and I did similar rote work in which we had to use computer software to measure the length of the pupae casings. I was used to doing it that, repetitive research tasks. I actually liked it. It was boring, but I also really like the idea that my work was helping to form data which would help my researcher in her research.[ …. ]Coming across this platform, it was similar. Based on my past experience, I could tell it was legit and that this stuff that we’re volunteering was helping in actual research. My past experience helped me to really like this website and the work I was doing, even though sometimes it was boring but it was cool." (Learner 3-18-F)

A few participants, specifically four young people, were found to declare limited prior science experiences, yet as the rest of the participants, they reported learning outcomes after taking part in Zooniverse.

• (1)A 16 years old male participant: This young learner uses Zooniverse due to interest in learning new facts about science and monitor birds, especially those that do not live nearby and are not easy to find. He stated: “I prefer projects about birds (Battling Birds) because I like to monitor birds. Birds in life don’t come to me and see me, I can’t see them often. I prefer projects with video”. This young learner declared limited previous experiences: “I can’t say that I’m very excellent in science, I can understand something [ …]I don’t know anybody (family or friends) who do a scientific-related job”. However, it was easy for him to take part in projects and he faced no difficulties. He declared changes in the understanding of how CCS works (ESA1) and expressed increased confidence (ESA2): “I think my confidence was influenced, but a little”. The learner identified the lack of platform translation to other languages as a barrier (as he is not an English native speaker), yet this did not inhibit participation: “May be it will be more comfortable if there will be Russian language on the site. But it is not very necessary, I like it.”
• (2)A 19 years old female participant: Another participant explained how they took part in projects on Zooniverse without any previous experiences that helped them with that as their skills are in a different: “I don’t have prior knowledge or skills to do the tasks. My skills are all just transcribing. I do a lot of transcribing work for school when I do interviews because I’m a journalist student.” (Learner 32-19-F). Interestingly, the same participant explained that they came across Zooniverse through a podcast they heard and they decided to use it as “a good way to distract” themselves while studying: “I heard about it from a podcast, and I thought it was interesting. Sometimes, I get a little distracted when I’m studying, and I figured that would be a good way to distract myself, instead of watching YouTube” (Learner 32-19-F). This participant took part in Zooniverse project related to her interest about history in particular: “I’m really fascinated about American history. I’ve transcribed some other ones before” (Learner 32-19-F). This young person showed evidence of ESA2: science identity-confidence: “I feel that taking part in Zooniverse activities influenced my confidence in my ability to learn or to do science. I did one project that was based on natural sciences, and I felt smart afterwards. I’m not very good at science academically, but I feel like I’ve just learned something, and I built on the very minimal-type knowledge I have already.” (Learner 32-19-F) This participant declared that they do not engage with projects that have “very long instructions” and tasks that are “hard to read” such as transcribing Shakespeare’s writing, stating: “Science is boring to me, or sometimes it is just too hard”.
• (3)A 15 years old male participant: Another young person explained that they started using Zooniverse after their parents forwarded an email with a link to the platform. When asked about their previous experiences with science, they explained that “I don’t learn biology anymore […]Some sciences, yes. I think I’m kind of good. What I learned in school had mostly nothing to do with what I did in Zooniverse” (Learner 28-15-M). This is the case of a young person who declared changes in their understanding of science after joining Zooniverse: “In my knowledge on animals like different types of species that are on this planet, that has definitely increased.” (ESA2- Identity-performance). Also, this participant shared their new knowledge gained from Zooniverse with friends: “I told my friends about this new project I’ve been using on Zooniverse. Yes. If I’ve seen some interesting facts, I tell them that stuff that I learned about, for example, animals on Zooniverse.” (Learner 28-15-M) (ESA2-Identity-sharing knowledge). This participant found it easy to take part in Zooniverse projects and needed no help to complete tasks. Yet, while he had questions about projects, he felt shy to start a conversation in the Forum: “I didn’t know how to start a conversation”.
• (4)A 8 years old male participant explained that “I don’t use science outside school or other after-school activities” and that “I don’t know if I am good at science. I am good at science because I am doing Zooniverse, it helped me in science at school” (Learner 31-8-M) emphasizing the positive impact of Zooniverse on science understanding and on school activities. This interviewee declared a change in their science identity - performance (ESA2) in the area of astronomy: “I am better at astronomy, by doing projects about astronomy on Zooniverse.” (Learner 31-8-M). He declared difficulties in understanding the language of some projects resulting in him searching the web for their meaning.

  Overall, these four participants presented rather limited prior science experiences compared to other participants. Yet, they declared some evidence of ESA after taking part in Zooniverse, noting the potential of online CCS projects to bring learning benefits to young people with rather limited prior science experiences. Also, participating in Zooniverse projects was straightforward for one of them, while the rest experienced difficulties related to the language used to describe projects and relevant instructions as well as the process of taking part in the Forum.

5. Discussion

In this study, we evidenced through the analysis of 34 interviews and log files (a) the potential of online CCS to support and promote a range of learning outcomes, as described by the ESA framework, for young people aged 11–19 years old, and (b) types of participation that enabled or hindered certain learning processes and outcomes, as informed by the theory of design affordances. This analysis resulted in developing a working framework that maps certain types of participation with the development of certain ESA manifestations. This is because participation was identified as a pathway to developing ESA. We considered participation and learning phenomena under the significant role of previous science experiences in enabling CCS participation and facilitating learning. In the next paragraphs, findings are discussed in relation to the Research Questions (RQs) of this study.

5.1. RQ1: what are young people’s self-reported science learning outcomes, specifically environmental science agency, from taking part in zooniverse projects?

Drawing from the ESA framework, young people reported increased understanding of science content, identified roles for themselves in the practice of science, and to a lesser degree developed a sense of agency for taking action to support themselves and their communities using science. These findings align with studies examining youth learning in field-based settings (Ballard et al., 2017; Harris et al., 2020). The development of science agency as a foundation for changing one’s self and others (ESA3) was tightly connected to an enhanced understanding of science content (ESA1) and/or the development of roles in the practice of science (ESA2), suggesting that young people can become more agentic in science, through opportunities that actively engage them with authentic science, as done by scientists as well as by drawing from previous science experiences and understanding that can help them engage with an understand science. Similarly, drawing from our findings, it is likely that when ESA 3 is deployed, the scientific literacy and identity of a young person (ESA1 and ESA2) can be reinforced by the implications and demands of taking on actions.

Our findings evidenced how participation in CCS helped young people become more “competent and knowledgeable actors” (Ruiz-Mallén et al., 2016). In some cases, it also helped to break stereotypical conceptions of science as presented in formal education (a subject area hard to understand and engage with) (Hellgren & Lindberg, 2017) by enabling youth to participate in tasks usually conducted by professional scientists. These findings contradict formal education implementations of CCS showing improvements in science knowledge yet not in other constructs including science identity and nature connectedness (Williams, Hall, & O’Connell, 2021). It is worth noting the different methods used to capture learning that may explain observed differences; in our study we used self-reports whereas other studies used less subjective methods of data collection such as pre/post test questionnaires.

5.2. RQ2: how do different types of participation relate to self-reported learning outcomes?

Certain forms of participation were shown to enable, yet in some cases inhibit, self-reported learning outcomes. These forms of participation could be seen as the learning processes or the conditions required for the development of ESA in online CCS. These processes were supported by the design features of the Zooniverse platform as well as appropriation actions initiated by participants that took place outside the platform. Within the platform, learning processes aligned with the “canonical” affordances (Tucker et al., 2016) of the platform and included taking part in tasks designed by scientists, that is, transcribing or classifying/annotating an image and identifying species in an image, exploring and discovering new content on the platform, and communicating with others via a forum. Outside the platform, learning processes were the result of technology appropriation by participants, and included searching for content related to a project on Zooniverse online (in other websites) or offline in books, and communicating with knowledgeable others such as guardians. These processes align well with existing studies (Jennett et al., 2016) that showed the importance of task completion, interaction with others and use of external resources to support CCS learning. Moreover, they align with field-based studies of youth agency with science by searching for additional resources to find answers to questions (Calabrese Barton & Tan, 2010). Youths’ identity with science has a role in how they use and bring in their personal resources and interpersonal relationships with respect to science (Ballard, Harris, & Dixon, 2018), hence, appropriation actions evidenced in out of Zooniverse participation can provide insights of youths’ identities with science and the material and social resources through which they display it.

A significant novel contribution of this study is the production of a first working framework of how certain types of participation can enable specific learning outcomes (see Fig. 2). In particular, engagement with a scientific task, exploration and discovery within and outside Zooniverse, and communication are core activities for the development of specific ESA manifestations. As expected, task-based participation on Zooniverse was shown to relate to a great number of ESA manifestations compared to other types of participation, suggesting that hands-on activities and direct engagement with CCS tasks are core learning processes through which ESA can be developed. This finding could be explained by the design of the platform and its emphasis, or call to action, to engage volunteers with transcribing and describing images, rather than, for example, interacting with other volunteers as in CCS platforms such as iNaturalist. Yet, certain manifestations of ESA including the desire to become a scientist, plans to use practice in another context, taking on new roles, recognition by others, ownership, a display of scientific reasoning and understanding of the norms of science, were not associated with task-related participation, suggesting that online CCS projects should consider the inclusion of additional activities, or design features, that are likely to enable these other learning outcomes. For example, taking on or developing new roles in the practice of science was facilitated by exploration and communication practices, suggesting that online CCS projects should be flexibly designed and provide options for choosing projects and tasks, while at the same time design tasks that open-up communication channels with other participants and scientists. These findings confirm current observations showing that the majority of CCS projects, by focusing on data collection practices, target the development of a rather small range of science skills (Stylinski, Peterman, Phillips, Linhart, & Becker-Klein, 2020) and raise the need for considering a broader range of learning outcomes when designing CCS activities or tools.

A few young people reported difficulties in relation to task-based participation and communication via the forum. These difficulties inhibited learning by forcing youth to quit or not take part in specific projects, suggesting that the design of online CCS projects could be improved to accommodate the difficulties some young people may face, including the fact that some participants had to seek help in external resources and more knowledgeable others to complete a CCS task. Drawing from reported difficulties, proposed improvements are: (a) the provision of task-specific feedback showcasing the elements of an image that should be transcribed and how to manage aspects that are not clearly identifiable. This could be achieved either through example analysis, that is the provision of a number of images classified by scientists, or the provision of personalised feedback to participants while they are transcribing the first few images of a project; this would help increase confidence in doing the task and enable sustained participation. (b) the language used to describe the scientific context of projects should be simplified to ensure young people understand the context of the project and the rationale for doing specific tasks, that at times may feel not scientific. (c) the guidance (tutorial) of how to complete a task should be redesigned to model/showcase how a participant completes a task. (d) the Forum should be monitored so questions raised are timely answered while the language of responses considers for the fact that some participants may have limited understanding of scientific phenomena.

5.3. RQ3: how does participation, as captured by log files, relate to young people self-reported science learning outcomes?

Aligning with existing studies (Herodotou et al., 2020), the log files analysis of young volunteers showed that the majority of them were not visiting Zooniverse frequently or systematically. Also, in contrast to existing studies suggesting that the more active volunteers are likely to learn more from taking part in CCS, over their previous science knowledge (Masters et al., 2016), our analysis suggested that this is not the case for the cohort of young people examined in this study. In particular, there was no significant relationship between an increased number of ESA manifestations and higher levels of participation in Zooniverse. While the degree of participation was not related to learning outcomes, we yet observed (see RQ4) that prior science experiences as reported by participants were related to ESA, in most cases supporting ESA development.

5.4. RQ4: who are the young people who are more likely to report learning benefits from participation in Zooniverse?

The great majority of the young people we examined in this study were shown to have considerable prior science experiences as shown in their prior science knowledge, prior interest in science, and prior experiences in working or visiting science places such as a science club or museums. These prior experiences worked as “a little stepping stool” that helped young people to take part and complete tasks, and inform certain learning outcomes. These experiences shaped how participants perceive themselves and what they learn about science (National Academies of Sciences, Engineering, and Medicine, 2018). Aligning with existing studies (Herodotou, Ismail, et al., 2021), young people who were more likely to engage and learn from CCS projects, at least at the point of data collection, were those with an existing interest, experiences and aspirations for science. The role prior interest or “high fascination” about science can play in learning was shown in other studies, relating positively with retaining science knowledge in the long-term as opposed to short-term learning outcomes (Schneiderhan-Opel & Bogner, 2020). In fact, our findings resonate with Archer et al.’s (2015) concept of science capital, which comprises: scientific literacy, scientific-related values, knowledge about transferability of science in the labour market, consumption of science-related media, participation in out-of-school science learning contexts, knowing someone who works in a science-related job, parental science qualification, and talking to others about science outside the classroom. Differences in science capital between families can explain observed inequalities in engagement with science amongst youth, as the children of families with higher science capital are more likely to pursue science trajectories for their future (ibid).

Similarly, in our study, the majority of young participants in the Zooniverse projects had an a priori interest and experiences with science, suggesting that youth with limited prior science experiences are less likely to engage with online CCS and gain some of the aforementioned learning benefits. The few cases of youth with limited prior experiences in our sample reported learning outcomes, emphasizing the significant role online CCS can play in supporting ESA development. One of the major challenges CCS is facing is how to truly make science accessible to all, and in particular how to engage with young people from groups typically under-represented in science. Youth need meaningful opportunities and supportive environments to enact agency (Mundford & Sanders, 2015), this includes accounting for their prior experiences and access to science. This study emphasizes the need to reconsider how online CCS projects are designed and advertised in order to engage with diverse communities that can benefit from taking part in online CCS, while also considering ways to overcome self-selection biases in CCS evaluation studies as the one reported in this paper. We suggest that future studies of online CCS programs should examine the science capital of participants in a more systematic manner, and seek to capture how CCS should be designed to improve youth’s science capital.

6. Conclusions, limitations and future work

This exploratory study should be seen as a starting point for the further exploration of how young people develop environmental science learning outcomes when participating in online CCS projects. Findings suggest that participation in authentic scientific activities can bring several benefits to young people including enhanced knowledge, development of science-related skills and competences, and opportunities for enacting science agency, as the foundation to changing one’s self and others. These findings refer to young people aged 11 to 19, the majority of whom had significant prior experiences or interest in science before joining online CCS activities. Informal science learning environments such as Zooniverse are shown to support self-directed forms of learning for specific young people, evidenced in them taking the initiative to participate in CCS projects, choosing specific learning activities (joining specific projects), identifying best strategies to support their learning, searching for information online, raising questions in a forum or reading relevant tutorials, and evaluating their learning and performance via reflection in project interviews. Self-directed learning took different forms, shown in the types of participation young people exhibited and which, in most of the cases, supported science learning and agency. These forms of participation were enabled by specific design features such as the platform’s functionality to complete scientific tasks and talk to a Forum and/or initiated and regulated by young people themselves such as seeking help from significant others or online and offline resources. Activity on Zooniverse was not mediated by a teacher or a formal instructional strategy but rather defined and implemented by young people themselves.

Despite the fact that all participating youth in this study reported enhanced learning outcomes from taking part in Zooniverse projects, for some of them participation was not straightforward. They reported difficulties while taking part in CCS projects related to a lack of feedback about task correctness, understanding of the scientific context of a study, information in the guide/tutorial to support task completion and interactions in the Forum. Preliminary insights suggested that young people with limited science experiences are amongst those likely to experience additional difficulties when joining Zooniverse, raising the need to design CCS projects in ways that are more inclusive. A key dimension should be the provision of formative feedback, given its significant role in promoting learning and learners’ success (e.g. Stiggins, 2005). In CCS projects, such feedback may be hard to provide given that the correctness of a task is not known beforehand but developing while participants are taking part in a project. A small amount of gold standard data, now used in some of the Zooniverse projects, could be applied more widely across projects to help participants with task completion, and further enhanced to provide personalised feedback while youth are working with a dataset. This should be seen as a training stage prior to asking participants to complete any project tasks. It could support a range of expertise (Fischer & Scharff, 1998) by providing opportunities for gradual engagement with CCS, such as easy or simple tasks followed by scaffolded routes to more complicated ones. Initial findings suggest that an incremental release of small amounts of data in a single Zooniverse project can stimulate sustained volunteer participation (Spiers et al., 2019). This kind of “levelling-up” approach has been pioneered by the Zooniverse on their Gravity Spy project, which guides volunteers through a series of levels of increasing complexity. Volunteers can unlock the next level once they have gained enough experience and understanding on the previous level.

Future studies should expand our understanding of learning by examining children younger than 11 years old and those with limited or no prior science experiences. Such examinations would provide us with valuable insights as to how to engage diverse audiences with online CCS and that science learning benefits are accessible in an inclusive and equitable manner. Considering the young age of children, direct interview questions could be complemented with other methodologies to help children recall and reflect on their experiences with online CCS projects such as vignette methodology (Crafter, O’Dell, Abreu, & Cline, 2009) or videography (Flick, 2018). With regards to demographics, it would be useful to collect additional information about participants (it was not feasible to collect in this study) such as geographic location, race and socio-economic status as it could inform study generalisations and initiatives aiming to enable inclusion and diversity in CCS projects.

We acknowledge that this study has captured a snapshot of learning in time through self-reported data and this approach bears certain limitations that future studies should address. In particular, future work should aim to understand growth in learning over time via long-term or longitudinal examinations. Such studies could be supported by the development of tools that automatically capture, record and visualise participants’ actions and learning behaviours online and over a period of time. The use of learning analytics was shown to effectively capture learning patterns in online learning environments such as MOOCS and associate these to specific groups of learners, mapping successful tactics in self-regulated learning (Fan et al., 2021).

In addition, the examination of learning was structured on the theory of Environmental Science Agency (ESA) and measured via self-reports (interviews). Future studies should seek to examine other aspects of learning not captured by ESA such as motivation to engage with science, trust of science and community-level outcomes (Jordan, Ballard, & Phillips, 2012). Performance-based measurements in the form of pre-post testing (e.g., Schneiderhan-Opel & Bogner, 2020; Stylinski et al., 2020) and structured observations could complement self-reported understandings of learning and provide a multifaceted measurement of the impact of online CCS activities on learning for young people. In the same line of thinking, it would be beneficial to follow up with those young people reporting evidence of agency and identify whether these participants realised their intentions to take action and support science, thus validating interview outcomes.

To enable access to authentic science and break stereotypes of who can take part in it, we need to consider and define the intended learning outcomes of a given CCS project and plan activities that enable, or are likely to develop, these outcomes accordingly. As emerged in this study, direct and hands-on engagement with a scientific task is a key activity that can promote self-directed science learning such as the development of scientific knowledge and enhanced science performance. Yet, in some cases this did not suffice, as the development or taking up of new roles in the practice of science was enabled by other forms of participation, not planned or intended by the CCS activity, including exploration and discovery and communication with others. This study examined a single online CCS platform, Zooniverse. Zooniverse exhibits specific design affordances with a focus on data collection. Future studies should seek to examine other online CCS platforms such as iNaturalist.org and iSpotnature.org focused on biodiversity data collection, or nQuire.org.uk focused on community-led research, to further understand how different design affordances may enable a more diverse set of learning outcomes and help to engage with science those young people with limited prior science experiences.

Our study suggests that prior science experiences were of particular importance to participating youth as they helped them to engage with online CCS and complete science tasks. Yet, it remains the need of how to design (or re-design) CCS projects or platforms that are inclusive and can engage youth with more diverse characteristics including those with limited prior science experiences. To further support technology-enhanced, self-directed forms of learning, online CCS projects should promote collaboration (Fischer & Scharff, 1998) between volunteers and scientists as well as amongst volunteers by enabling communication with regards to the completion and understanding of a task. The design cycle of a CCS project should ideally result from a process of co-design with communities of interest or at least be accompanied by systematic testing with potential end users to ensure it meets the needs of participants and can achieve intended learning requirements (alongside the scientific ones). CCS offers a unique opportunity to youth to engage and understand scientific processes, yet to do this in an inclusive and equitable manner it should reflect “the diversity of publics” and be designed in ways that do not “reinforce existing inequities in science and society” (Soleri, Long, Ramirez-Andreotta, Eitemiller, & Pandya, 2016). Significant consideration should be given to how young volunteers are recruited in order to reach communities that are less likely to have access to technologies and networks, such as communities of Colour and low-income ones. While interested in science (National Academies of Sciences, Engineering, and Medicine, 2021), such communities face obstacles to participation in informal learning activities, exacerbated during the Covid-19 pandemic, and the need for fast devices and internet connection to complete any learning activity. Building bridges between informal CCS programmes and formal education could enable engagement of young people in formal settings with informal science experiences such as CCS projects.