MYHealth Impact Projects: A Mixed Methods Evaluation of an Online Research Training Program for Adolescents

Abstract

Biomedical research is essential to healthcare and medicine. However, challenges in training and decreasing persistence among trainees have contributed to the U.S. ongoing healthcare crisis. MYHealth Impact Projects is a 9-month online research training program for adolescents where twenty-two participants used a nationwide text message poll to complete a team-based adolescent health research project. We conducted a mixed methods evaluation to assess engagement and changes in participants’ researcher identity and self-efficacy, science motivation, and science literacy, including: 1) pre-, mid- and post-surveys, 2) program participation, and 3) semi-structured interviews. We used Kruskal-Wallis rank tests to compare changes over time. We conducted inductive thematic analysis on qualitative data and integrated final themes with the quantitative results for mixed methods inferences. Participants completed an average of 17.8 hours in bi-weekly sessions (28.75 hours max). There were statistically significant increases in researcher identity and science literacy. Salient themes included: 1) dispelling preconceived notions of being a researcher, 2) building research literacy through hands-on experience, 3) developing self-efficacy through an environment that supports autonomy, and 4) emerging interest in research careers through near-future goals. These findings emphasize the importance of hands-on and authentic learning in research training.

Introduction

Biomedical research serves as the foundation for ongoing progress in healthcare and medicine. However, decreasing persistence among trainees in recent decades has further contributed to the ongoing healthcare crisis (Nabel, 2009). The number of Science, Technology, Engineering, and Mathematics (STEM) degrees awarded at all levels nearly doubled between 2000 and 2019 (National Science Foundation [NSF], 2022a, b), but there is still considerable dropout along the training pathway and into biomedical research careers. Less than 30% of undergraduate STEM degree recipients work in the field post-graduation (Okrent & Burke, 2021). In addition, completion of PhD programs and post-doctoral fellowships has fallen by more than 10% (NSF, 2024), and of the 14% of PhD awardees that pursue a tenure-track research faculty position, 90% discontinue research within the first ten years (Gayen et al., 2025). At the center of the current biomedical research crisis are limited and poor quality mentorship coupled with a lack of trainee preparedness for varied career opportunities, leading to dropouts rather than pivoting within the field (Kim et al., 2013; Vapiwala et al., 2019). Without intervention, the risks of the current biomedical research crisis include the “evaporation of the scientific discovery pipeline,” the loss of a generation of scientific leaders and mentors, and unimaginable disruptions to the practice of evidence-based healthcare (Daniels, 2015).

While significant structural change is needed to reduce barriers to STEM and research careers, interventions that provide supportive training and mentoring opportunities have shown promise in building the psychological resources, social capital, and formative experiences needed to support students’ commitment to pursuing STEM research (Salazar et al., 2024). Moreover, early intervention can broaden participation, nurture interest, and promote persistence in STEM fields (Bottia et al., 2015; Maltese et al., 2014; Sahin et al., 2018). In that regard, out-of-school programs present a unique opportunity as a practical option to cultivate early interest and persistence in STEM and promote biomedical research careers (Allen et al., 2019; Castilho et al., 2023; Chan et al., 2020; Witzel et al., 2020; Young et al., 2017). Out-of-school programs that provide engagement with STEM through real-world and project-based learning have increased youth interest and self-efficacy in STEM and future STEM careers (Lewalter et al., 2021). Authentic and applied experiences in STEM are especially important in the development of science identity, interest, self-efficacy, and science literacy for students from historically marginalized communities who often have less exposure to research in both classroom and out-of-classroom settings (Cutucache et al., 2016; Rookwood et al., 2022) and can benefit from broad mentoring and support networks to gain upward educational and professional mobility (Hammerness et al., 2024). Furthermore, STEM out-of-school programs that require in-person participation can be limited by space and time, so programs that offer virtual opportunities can reduce barriers to participation, such as transportation, and increase the reach and access to participation across different geographic locations (Villatoro & Moseley, 2021).

Youth Participatory Action Research (YPAR), a research inquiry approach closely aligned with community-based participatory research, positions youth as primary investigators on problems that affect them and their communities to create change (Ozer, 2016; Rodriguez & Brown, 2009). YPAR projects, which involve youth in all aspects of the research process from study design to dissemination, allow not only the exploration of topics such as mental health but also address educational injustices and inequities impacting youth with the goal of improving local knowledge about disparities in particular (Fisher et al., 2024; Godinez et al., 2025). When YPAR approaches are used in the context of education, they allow students' voices to be centered and facilitate more personal connections to learning, which both have positive effects on academic motivation, achievement, and engagement, especially for students marginalized based on race and socioeconomic status (Voight & Velez, 2018). YPAR has been applied successfully in both in- and out-of-school programs to increase student motivation and confidence in research and bolster chances of college and career success (Anyon et al., 2018; Ozer & Douglas, 2013;).

Using a YPAR approach, MYHealth seeks to engage high school aged participants from Southeast Michigan in a hands-on, relevant research training and mentorship experience to promote interest and persistence in research careers. The program takes place online over three phases: 1) Summer Launch, which is a 10-day introduction to health-related research and careers, 2) Impact Projects, which is a 9-month training that helps participants learn about and conduct research, and 3) Peer Leaders, where returning participants get additional training to support the next cohort (Chuisano et al., 2023; DeJonckheere et al., 2023). The complete structure and evaluation methods of MYHealth have been detailed in the program’s protocol (Chuisano et al., 2023). In this article, we present results from a mixed methods evaluation of the first cohort of the Impact Projects phase where we examine participants’ engagement in an applied research experience and how youth-led projects can influence participants’ researcher identity, research self-efficacy, science literacy, and motivation.

Methods

In this study, we conducted a mixed methods evaluation of the first year of the Impact Projects phase of MYHealth, which occurs over nine months during the academic year and teaches participants the full research cycle as they complete an applied research project from design to dissemination. For our evaluation, we integrated quantitative and qualitative approaches to understand how the program impacts participants’ researcher identity, research self-efficacy, science literacy, and motivation. Specifically, we employed a convergent mixed methods design (Figure 1) to compare quantitative outcomes (participant attendance, change in constructs measured through surveys at three points in time) to qualitative themes (captured in post-program semi-structured interviews and open-ended survey questions).

Figure 1.

Convergent mixed methods design of MYHealth Impact Projects program evaluation

Participants and Recruitment

For the overall MYHealth program, participants include current and recent high school students who were recruited via high school counselors, youth-serving organizations, and other community partners. Recruitment focuses on partners who serve large populations of students from groups that have historically been excluded from research. All participants who completed the Summer Launch phase were invited to participate and received additional information about Impact Projects and how to enroll. Finally, snowball sampling allowed study participants to share recruitment materials with others they thought would be interested in the MYHealth program.

Potentially eligible adolescents were sent information about MYHealth via email and indicated their interest through an online survey. To be eligible, adolescents had to be current high school students (9th - 12th grade) in Southeast Michigan and be able to join the virtual program. Participants aged 18 and older provided written consent, while those under the age of 18 provided written assent and their parent/guardian provided written consent. Participants without computer access were provided with a laptop for the duration of MYHealth. This study was approved by the University of Michigan Institutional Review Board (HUM00214694).

Curriculum and Program Components

The objective of the MYHealth Impact Projects phase is to engage adolescents in an out-of-school research training to increase their interest in pursuing a research career in the future. Participants are trained as co-researchers and work in small teams to design a qualitative study that explores youth perspectives on a health-related topic of their choice using MyVoice, a nationwide text message poll of 14–24-year-olds in the U.S. (DeJonckheere et al., 2017). This approach is aligned with YPAR as youth are trained and support to conduct research on other youth. In the present study, research topics were selected through a Group Level Assessment (Vaughn et al., 2022) where teams identified adolescent health issues of interest and importance to them. Following a qualitative team-based approach in line with the existing MyVoice study's methods (DeJonckheere et al., 2017), teams developed and revised open-ended survey questions through iterative feedback from MYHealth faculty, staff and facilitators, as well as MyVoice team members with expertise in adolescent health. Surveys were sent to over 800 MyVoice participants via text message in Fall 2022 and collected responses were shared with MYHealth teams in a Google Sheets format for ease of collaborative review and data analysis. Teams worked together to complete an inductive thematic analysis of responses by 1) getting familiar with data, 2) developing a codebook and codes, 3) completing line-by-line coding of responses, 4) identifying major patterns and themes in the data, and 5) narrowing in on final themes. Each step of this process was guided by ongoing training in qualitative research methods and group-based activities facilitated by faculty and staff.

By working on an applied research project, participants develop foundational knowledge in research, research ethics, and primarily qualitative research methods. The program also aims to continue building students’ interest in STEM and research, their ability to see themselves as researchers (i.e., researcher identity), and their motivation towards science and research. The session agendas and the different activities that took place throughout the program are outlined in Table 1. Teams consisted of three smaller groups of 7–8 Impact Projects participants, session leaders (two program faculty members, a program manager, graduate and undergraduate research assistants), and an external evaluator. In some sessions, teams went into Zoom breakout rooms in smaller groups of 3–4 participants to additional tasks facilitate more focused work and provide each additional tasks participant with additional learning support. Meetings included interactive activities to learn about and apply research concepts and methods in a team setting. Participants were asked to complete additional tasks between sessions when needed.

Table 1:

Sample Impact Projects Agenda

September	October	November
Session 1: MYHealth Overview and Introductions	Session 2: Identifying Priority Areas and Research Questions	Session 4: Refining Research Questions as a Group
	Session 3: Developing and Evaluating Research Questions	Session 5: Learning to Integrate Feedback
		Session 6: Finalizing Research Questions
December	January	February
Session 7: Ethics and Informed Consent	Session 8: Developing a Research Plan	Session 10: Developing the Codebook
	Session 9: Mid-way Survey and Introduction to Memoing	Session 11: Introduction to Coding
March	April	May
Session 12: More Coding and Introduction to Theme Development	Session 14: Revisit Themes and Develop Findings	Session 16: Finalize Presentation for AHI Conference
Session 13: Reviewing Questions and Theme Development	Session 15: Create Presentation for AHI Conference	Session 17: End-of-Year Celebration and Future Steps

The three research studies by the MYHealth Impact Projects participants in 2022–2023 focused youth’s view on COVID-19 and its effects on their future, youth’s perspectives on healthcare accessibility in America, and youth’s sentiments health education in schools and how it can be improved. Impact Projects participant teams presented findings from their research studies in a joint presentation at the 2023 Conference on Adolescent Health (Ahmed et al. 2023). In addition, our team has completed a confirmatory analysis on two of the datasets. The research resulted in additional conference presentations and manuscript submissions to academic journals (Durden et al., 2025; Pestian et al., 2025).

Data Collection

At baseline, mid-way through, and upon completing the program, participants completed an online survey via Qualtrics. Surveys were emailed to students to be completed independently outside of session time. Survey items focused on science interest and motivation, scientific literacy, research self-efficacy, and researcher identity. The baseline collected demographic information and science experiences. Throughout MYHealth Impact Projects, data on participation was collected. In addition, all participants were invited to complete semi-structured interviews. More details about each of these methods are provided below.

Program Participation.

Attendance was evaluated for each session (roughly every 30 to 60 minutes) by recording who was logged onto the Zoom meeting. In addition, the completion of online independent activities was tracked. Depending on the independent task's complexity, 30 or 60 minutes of time were awarded for completion of the activity. The total number of hours of potential activities was calculated at the end of the program.

Demographics, Science Participation, and Aspirations.

Validated multiple-choice questions were used to collect information on gender, racial and ethnic identities, disability status, geography location, and grade. To measure socioeconomic status, participants completed a two-item version of the MacArthur Scale of Subjective Social Status, which has been used and validated with adolescents and young adults from various socioeconomic backgrounds (Goodman et al., 2001). Each question was presented as a ladder, and participants were asked to indicate where they perceived themselves within the larger U.S. society and their own community. Each rung of the ladder corresponded with numbers from 1 (i.e., bottom rung associated with “People who are worst off”) through 10 (i.e., top rung associated with “People who are best off”). Additionally, previous science participation (i.e., “Have you ever attended a science camp or special science program?”) and STEM interests were measured through self-report items from the “Is Science Me?” Questionnaire (Aschbacher et al., 2010).

Researcher Identity.

To measure Researcher Identity, participants completed a 14-question Guttman-style scale. This scale has been previously validated and used with high school students to measure their aspirations, interest, self-efficacy, agency, community, and how they perceived themselves as they engaged in a scientific investigation (Koo et al., 2021; Wilson et al., 2022). For each question, participants selected from five text options, which are assigned a numerical value that ranges from 1 (lowest level) to 5 (highest level). For example, one of the questions is structured as: “Which statement below best captures your identity as a researcher? (1) Being a researcher is not part of who I am, (2) I am not sure if being a researcher is part of my identity, (3) Being a researcher might be a small part of my identity, (4) Being a researcher is a part of my identity, and (5) Being a researcher is a big part of my identity.

Science Motivation.

To assess science motivation, we used the Science Motivation Questionnaire-II (SMQ-II), which measures four components of motivation to learn science: intrinsic motivation, self-efficacy, self-determination, grade motivation, and career motivation (Glynn et al., 2011). This 25-item questionnaire has been validated in different settings with youth (Bryan et al., 2011). Each question is on a 5-point scale from never (1) to always (5) and includes items like “I believe I can master science knowledge and skills” or “My career will involve science”.

Research Self-Efficacy.

Research self-efficacy was measured using four items from the Research and Action Self-Efficacy scale developed by Ozer and Schotland (2011). Participants rated their agreement with each statement on a 4-point scale. For example, items include “I can explain guidelines for conducting good research (i.e., ethical and with a rigorous method) to another person.” and “I can teach other students to develop their own research projects on issues of concern for them.”. Research self-efficacy was collected only at the midpoint and end-of-program surveys.

Science Process Skills Inventory.

On a defined 4-point Likert scale from never (1) to always (4), participants responded to 11 items evaluating their ability to apply scientific concepts in a full research cycle. The Science Process Skills Inventory (SPSI) has been used with youth ages 12 and over and is not intended to measure science content knowledge (Arnold et al., 2012). Items include “I can use scientific knowledge to form questions” and “I can communicate a scientific procedure to others”.

Open-ended Survey Questions.

The final survey included the following three open-ended questions, placed at the beginning in that order: “What do you think research is?”, “Reflecting back on last year, have your thoughts changed about what researchers look like and do?” and “How so, and How has your vision of yourself as a researcher changed?”.

Semi-structured Interviews.

After completion of MYHealth Impact Projects, semi-structured interviews were conducted to gain a better understanding of program elements that impacted outcomes of interest (i.e., research self-efficacy, science motivation, science literacy, and researcher identity) and participants’ overall experiences in the program. All participants were invited to complete an interview. Program evaluators completed interviews using video conferencing software. Interviews were audio-video recorded and transcribed.

Data Analysis

Quantitative Analysis.

Participants’ sociodemographic and background characteristics, and program participation were summarized using descriptive statistics. For each scale (and subscale for Science Motivation), each participant’s mean score was calculated to generate an overall score. Additionally, Kruskal-Wallis rank sum tests and Dunn’s post-hoc tests with Bonferroni correction were used to evaluate differences in overall scores for researcher identity, science motivation, scientific literacy, and research self-efficacy scores before, during, and after the program. However, responses from participants were not matched across surveys due to lack of participant response at all data collection timepoints (n=2) or program dropout (n=4).

Qualitative Analysis.

Interviews and open-ended survey questions were analyzed using an inductive thematic approach (Braun & Clarke, 2006) where the team (JL, HK, RM, AR) familiarized themselves with the data, developed a codebook, and then applied codes to transcripts. Each team member developed preliminary themes representative of the patterns across multiple codes, and the open-ended survey responses were triangulated with the preliminary themes generated from the interviews. The themes were finalized in group discussions with the full research team and captured participants’ nuanced experiences in the program.

Mixed Methods Integration.

In this mixed methods evaluation, the quantitative and qualitative approaches were integrated during data collection, analysis, and interpretation. Data collection tools were selected and developed to evaluate some study constructs from both a quantitative and qualitative perspective. For example, researcher identity and STEM interest were measured quantitatively in the survey using validated scales and explored qualitatively through semi-structured interviews. Following data analysis, survey results were compared to themes. We constructed a joint display that merged the findings from both approaches (Guetterman et al., 2015). Mixed methods inferences, including agreement between the quantitative results and qualitative findings, are presented in the joint display and described in the results.

Results

Quantitative

Twenty-two 9^th-12^th grade students enrolled in the MYHealth Impact Projects during the 2022–2023 academic year. In total, 17 program sessions were held every other week from late September 2022 through May 2023. The average session length was 1 hour and 41 minutes ± 23 minutes. Participants spent an average of 17.8 hours in bi-weekly sessions, which is almost two-thirds of the total possible in-session time (28.75 hours). Participants also spent an average of 7.6 hours working independently on online post-session activities, which is over half of the total post-session time (13 hours). One MYHealth Impact Projects group had two sessions (a total of 4 hours of programming) canceled due to low attendance. Surveys were collected from 21 students at baseline, 15 at midpoint, and 17 at program completion. Demographics and other background information were available for 20 participants at baseline (Table 2). Most participants identified as female (95%). Participants primarily identified as Arab/Arab American (n = 10, 50%), African American/Black (n = 4, 20%), and South Asian/South Asian American (n = 4, 20%). Most participants placed themselves around the middle of the subjective social status in the MacArthur scale, with a mean ladder score of 6.30.

Table 2:

Baseline Sociodemographic and Program Participation Descriptive Statistics

Variable	N1 = 20
Gender
Female	19 (95%)
Male	1 (5.0%)
Race/Ethnicity
African American/Black	4 (20%)
Arab/Arab American	10 (50%)
Middle Eastern	1 (5.0%)
South Asian (Indian subcontinent)/South Asian American	4 (20%)
White/Caucasian/European/European American (German, Italian, Irish, Canadian, etc.),	1 (5.0%)
Japanese/Japanese American
Grade
10th grade	4 (20%)
11th grade	9 (45%)
12th grade	7 (35%)
Macarthur Social Scale Ladder
Mean (SD)2	6.30 (1.84)
Program Participation
In-Session Hours2	17.8 (9.4)
Post-Session Hours2	7.6 (4.4)

¹n (%),

²mean (SD),

SD: Standard Deviation

The Kruskal-Wallis rank sum tests indicated no statistical significance for the differences in medians for participants’ science motivation (for all 5 sub-scales) and researcher self-efficacy. Statistically significant differences were reported for researcher identity and science process skills inventory (Table 3).

Table 3:

Pre-, Mid-, and Post-Program Survey Comparisons

Variable	Pre, n = 21	Mid, n = 15	Post, n = 17	P-value1
Researcher Identity
Median	3.42	3.75	4.08	0.009
(IQR)	(3.08, 3.67)	(3.33, 4.00)	(3.67, 4.33)
SMQ-II Intrinsic Motivation
Median	4.00	3.80	4.40	0.4
(IQR)	(3.60, 4.60)	(3.50, 4.50)	(4.00, 4.80)
SMQ-II Self-Efficacy
Median	4.00	4.20	4.20	0.8
(IQR)	(3.80, 4.60)	(3.90, 4.50)	(4.00, 4.60)
SMQ-II Self- Determination
Median	3.60	4.20	4.00	0.9
(IQR)	(3.40, 4.60)	(3.20, 4.50)	(4.00, 4.40)
SMQ-II Grade Motivation
Median	4.80	4.80	4.80	>0.9
(IQR)	(4.40, 5.00	(4.60, 5.00)	(4.40, 5.00)
SMQ-II Career Motivation
Median	5.00	4.80	5.00	0.9
(IQR)	(4.00, 5.00)	(4.20, 5.00)	(3.80, 5.00)
Science Process Skills Inventory
Median	3.00	3.36	3.36	0.035
(IQR)	(2.45, 3.34)	(3.18, 3.59)	(3.09, 3.82)
(Missing)	1	0	0
Research Self-Efficacy
Median	N/A	3.25	3.50	0.5
(IQR)	(N/A, N/A)	(3.12, 3.50)	(3.25, 3.75)
(Missing)	21	0	0

¹Kruskal-Wallis Rank Sum Test,

IQR = Interquartile Range, N/A = Not Applicable

Post-hoc Dunn’s tests with Bonferroni correction indicated that Researcher Identity was significantly higher at Post than Pre (adjusted p = 0.01). Dunn’s post-hoc tests with Bonferroni correction indicated that Research Identity (adjusted p = 0.006) and Research Self-Efficacy (adjusted p = 0.04) were both significantly higher at Post compared with Pre, while differences between Pre and Mid and between Mid and Post were not significant.

Qualitative

Interviews were conducted with participants (n = 10) to better understand the impact of the MYHealth Impact Projects on their relationship with research. Thematic analysis resulted in four overarching themes: 1) Dispelling preconceived notions of being a researcher, 2) Building research literacy through hands-on experience, 3) Developing self-efficacy through an environment that supports autonomy, and 4) Emerging interest in research careers through near-future goals. The themes presented below include data from both interviews and open-ended survey responses.

Theme 1: Dispelling preconceived notions of being a researcher.

Participants described how being part of MYHealth Impact Projects challenged misconceptions they had about being a researcher, which allowed them to see opportunities for themselves to be researchers in the future. Many participants reflected on preconceived stereotypes they harbored about who can be researchers based on race, gender, and age. In addition, participants described previously having limited ideas on what jobs, skills, and settings were considered “research”. For example, two participants explained:

I used to imagine researchers as white people in white coats in a sterile white lab surrounded by chemicals, beakers, and tubes. Now, I know researchers come from all communities, and asking questions and getting answers is also a form of research.

I've realized that anyone can do research no matter your age and that was very valuable to me, because at first…when I pictured like a researcher, I would think of like an old man like a lab coat. It was like a very stereotypical image.

Many participants maintained that they learned that “research isn’t limited to anybody” and that you can engage in research at different levels. During the interviews, one participant expressed:

Over the course of [Impact Projects], I've learned that everyone is a researcher in their own way. Whether it's just you looking up a question that you have, or even like the small things versus the big things like people and careers.

Participants described how personal characteristics, including curiosity, passion, motivation, and patience, play essential roles in developing research skills and pursuing a research career. Two participants explained their perspectives on what it means to be a researcher in the interviews:

I think being a researcher really just has to do with your level of curiosity and the passion you have to kind of find answers to those questions that you have, and I think like I said, before anyone can be a researcher…being a researcher is just having that will or passion to find answers and to you know, … get more findings to whatever you are not sure about or have questions about.

So, no matter how much you make mistakes, you just need to be patient and just keep working until you discover what you're trying to do.

Furthermore, many participants reflected on the implications of changes in their understanding of what it meant to be a researcher on their own researcher identities. According to participants, getting a broader understanding of the vast range of researcher skills, roles, and fields helped them see themselves as researchers now and in the future. Two participants described:

My vision as a researcher has significantly changed because before going through the Impact Projects process, I really didn't know what it takes to be a researcher and what the job really entails. However, by going through the entire process and working on different steps of the research process throughout the year, I was exposed to more of the reality of a researcher and found myself more confident with the title of a researcher as well.

Being able to participate in [MYHealth] program, I feel like really opened up, like I would say different, a new pathway for me, and like a lot of my friends that I know in the group.

Theme 2: Building research literacy through hands-on experience.

Participants reported gaining new knowledge and experiences while carrying out the youth-led projects, including practical research skills (e.g., research question development, qualitative analysis, presentation) and interpersonal skills. Participants described how, before the program, they had primarily been exposed to quantitative approaches to data collection and analysis. In contrast, MYHealth Impact Projects participants learned how to conduct qualitative research on adolescent health perspectives and opinions and reported that exposure to a different type of research broadened their perspective on what counts as research. Two participants explained:

So, the difference between qualitative and quantitative data was really helpful…and I also found out that I really love the qualitative approach of research.

I didn't realize, of course, stuff like this, where you survey people and ask them questions and have to use […] methods of sorting data to find conclusions was also a form of research.

Participants highlighted some of the specific technical qualitative research experiences they gained during the program. For example, one participant recounted: “I didn't really have that much experience with especially such large data sets before. So, getting to see a unique set of data and being able to analyze it was a really good experience for me.” Another participant explained the experience of learning about and applying line-by-line coding when using a qualitative approach:

The coding part was definitely a challenge, mostly because it was hard to actually come up with the codes themselves and decide what's what are the patterns we see in the data. And it was also difficult to decide how a specific response should be coded because of the fact that many of them fit into multiple codes. So that's definitely one of the things that I remember.”

As a culmination of their work, participants shared their findings at a national conference, and many reported learning about the importance of dissemination in research. One participant explained: “…after we're done with our research, we have to share that with others so that they can benefit from it too. Not just us.” Moreover, many appreciated having the opportunity to present in front of research experts for the first time:

I haven't had other opportunities to present research, but that opportunity was very fun because it was my first time ever presenting research. It's kind of like a special thing. It opened my mind up to the idea of sharing research.”

The same participant continued to explain that listening to other researchers’ presentations was “interesting because I got to see other people's research, and actually understand their process that they went through and all that was fun.”

In addition to technical research experiences, participants described developing non-technical skills like communication and teamwork. Almost all participants shared their realization of the importance of communication in research, particularly when working in teams (e.g., “If you don't have good communication skills and you're just kind of keep to yourself, you're not really helping the research. So, you need to be able to help your team and communicate with them”). Many participants described how they became more comfortable with communication over the course of the program. When it came to the role of teamwork in the research process, participants noted how working in teams allowed for research to be more efficient and finding solutions when feeling “stuck” easier. For example, two participants explained:

I learned more about the methods and how teamwork can really help in research, too, just because there are so many responses [in the data]. And it would take a lot of time for one person to individually, do everything. So, I learned, how teams can work together to not just split up the work…and then encompass everything we can look at not only our part, but what our teammates also did.

We would discuss it and then we will come back and ask the other researchers and they would help guide us and sometimes we would have to like tweak the code definitions, so it can fit more participants answers…It was really helpful to have other people give their opinion when you were stuck.

Theme 3: Developing self-efficacy through an environment that supports autonomy.

Participants described the MYHealth Impact Projects environment as a space that allowed them to build confidence in their research knowledge and skills. This environment, being virtual, presented an opportunity to learn and collaborate with others that would not have been accessible to many if done in a different format. One participant explained:

I will say working together, being able to communicate from on different places, even though we all in different cities. But we all were in the same room about the meeting. So, that was the thing that I valued the most. That was what made me, you know, join the session every Saturday to see these people, even though I don't know them. But I just they were just a part of my life during that time.

In this environment, participants also learned by doing, which made their experience more impactful (e.g., “When you do something yourself, it makes it easier for it to stick in your brain and helps you gain confidence”). In addition, participants described how they were treated as equal members of the research team. One participant explained:

I think it's really cool how you know we get treated as like part of the team…Of course we were learning from the staff. And it was really cool to learn from them. But at the same time, we got treated as co-researchers. You know, … we were actually part of the team. We got to actually do stuff. We were very included in all of this and that really like that really set out to me it was really cool how we were treated as like on the same level.

Many participants expressed being pleasantly surprised by how participant-driven the program was and how valuable having autonomy was to their overall experience. When referring to the freedom students had to select their research topics, one participant explained:

Obviously, there were guidelines, but it wasn't just everything wasn't set, and we catered it to what our research question is and what our interests were, and we made it what we wanted to make it. And I think that's what stood out to me…how flexible it was, and how we were able to input our own individual ideas into the program.

Many reported that the inclusive and supportive environment helped them get comfortable participating. One participant described how they were given a choice of different ways to share their opinions based on their preference and comfort level:

[I like] how everything was inclusive, and how even if you were kind of shy to talk, you were allowed to have your own space. You were allowed to type in the chat. If you didn't want to speak verbally, you can contribute in any way that you want…everybody was included, and they want the way they wanted to be included.”

The continued support from the instructional team gave participants a sense of ease and assurance knowing that help was always there if needed. For example, one participant explained:

[Support from the training team] makes it a lot easier than going through the research process on your own, because we always had people supporting us, like everything that we did. We could always go back to the group. It was just everybody was always there to help out.

Participants also felt supported by their peers and enjoyed the collaborative environment. As some described, “we all gave each other advice”, “the people who I was working with definitely made the experience better” and “it felt like everybody was just a group of friends meeting to get something done.”

Theme 4: Emerging interest in research careers through their near-future goals.

After acquiring the initial research skills through the program, participants expressed wanting to continue developing their skills through further research-oriented extracurriculars (including staying involved in MYHealth) and potentially research careers. One participant explained:

I am thinking about going to the next step, the [MYHealth] Peer Leaders program, to better help my understanding of research. And you know, actually help out others doing the research program. Because I've already been through it and I think it's like very helpful.

Others described wanting to look for more opportunities while in high school or college to gain more independent research experience. For example, one participant shared:

I’ll be a junior next year, so I'll have some more time to have experience in high school, and then hopefully, in college. I can talk with like my college counselor, and I'd like to pursue some type of research classes or individual independent research studies in college, to go along with my primary education.

For some, participating in additional research opportunities would help them consider whether a research career is something they want to pursue in the future. One participant described:

I'm [going to] do other research programs and hopefully, find out what exactly I want to do from then on. I think maybe I'll take a research class, maybe AP research, maybe start out taking college classes like to see if I like research, if I want to like pursue as a job.

Participants often acknowledged that they still had a lot to learn but thought that they had gained valuable skills through MYHealth Impact Projects that will help them in their future research endeavors. One participant described:

I am aware that I have much to learn about research and that I still have a lot to learn, but I have made great progress in understanding how to conduct research and I feel somewhat prepared to conduct my own research project using the skills I have acquired via this program.

Furthermore, most participants reported interest in pursuing a medical career before joining the MYHealth Impact Projects, and many described being interested in research as well in the post-program interviews. One participant explained:

I want to continue with research to as far as I can, you know, including all the way it'll be along. But I also want to do research for my future career, which I hopefully plan to be an anesthesiologist, which will take a long time. But I want research to be a part of that as well.

Many explicitly stated that their research interest remained high or grew during the program, and expressed their desire to pursue a research-oriented career. For example, a participant who was already interested in a research career shared: “I kind of always have wanted to work in a research-related field. And even after this, I still do want to, in the future, eventually, be working as a researcher.”. Another participant shared how their interest grew during the Impact Projects: “I think my interest has definitely increased in the research, and I'm definitely thinking about pursuing more like of a research-integrated career in the future.”.

Mixed Methods Results

We integrated quantitative results and qualitative themes by creating a joint display for comparison (Table 4). The display includes the quantitative result for each construct, corresponding qualitative findings (experiences described in the interviews), and the level of agreement between approaches. Overall, the qualitative findings corroborated some of the quantitative results, while also expanding understanding of how the program influences researcher identity, science literacy, research self-efficacy, and science motivation.

Table 4:

Joint Display of Quantitative, Qualitative Results and Meta-inferences

Scale	Quantitative Change in pre and post program medians, P-value	Qualitative Participants’ experiences - post-program	Mixed Methods Integration
Researcher Identity	3.42 to 4.08, 0.009	Gained a broader understanding of who can participate in research and in what ways (Theme 1) Viewed themselves as researchers (Theme 1) Gained confidence in their research skills (Theme 3) Reported a sense of belonging to the community they built with peers through the course of the program (Theme 3)	Agreement, the qual supports the quant increase in researcher identity.
Science Literacy	3.00 to 3.36, 0.035	Gained meaningful experience with the research process (Theme 2) Highlighted different aspects of the research process (Theme 2) Gained non-technical skills (e.g. communication, teamwork) (Theme 2)	Expansion, qual supports the quant increase in science literacy. Qual findings also indicate that participants gain non-technical skills.
Research Self-Efficacy	3.25 to 3.50, 0.5*	Gained confidence in their research skills due to being treated as an autonomous beings (Theme 3) Gained confidence due to a supportive learning environment (Theme 3) Presented findings to experts in the field boasted confidence (Theme 2)	Expansion, while the change in research self-efficacy was not statistically significant, participants described increased confidence in their research skills through completion of applied research projects.
Science Motivation		Enjoyed making independent decisions about research designs (Theme 3) Made plans to further research knowledge (Theme 4) Increased interest in research pursuits/careers (Theme 4)	Expansion, while not statistically significant changes in aspects of science motivation, participants reported increased motivation related to research and future research opportunities.
Intrinsic Motivation	4.00 to 4.40, 0.4
Self-Efficacy	4.00 to 4.20, 0.8
Self-Determination	3.60 to 4.00, 0.9
Grade Motivation	4.80 to 4.80, >0.9
Career Motivation	5.00 to 5.00, 0.9

^*mid- to post- medians since the Research Self-Efficacy scale was not collected at baseline.

The increase in researcher identity from pre- to post-program was supported by themes described by participants in semi-structured interviews and open-ended survey questions, including how they gained additional understanding of what research and researchers look like (Theme 1), practiced applying research skills (Theme 2) and felt a sense of belonging to a supportive learning community (Theme 3). Still, participants were unsure in their ability to conduct research independently (Theme 4).

The increase in science literacy from pre- to post-program was supported by qualitative findings about the development of research knowledge and skills during the program (Theme 2). In addition to the technical research skills gained, participants reported acquiring non-technical skills such as teamwork and communication, which were essential to their research progress.

While no changes of statistical significance were reported for participants’ intrinsic motivation, self-efficacy, self-determination, grade motivation, career motivation, and researcher self-efficacy, evidence in the qualitative findings provided context to the dynamics (or lack thereof) of those constructs throughout the program. For example, participants reported increased confidence in applying research skills and communicating about their research projects (Theme 2) despite the lack of measurable change in self-efficacy.

Discussion

In this mixed methods evaluation, we sought to understand how an out-of-school online research training program, MYHealth Impact Projects, contributed to participants’ motivation and persistence toward STEM research careers and their development of researcher identity, research self-efficacy, and science literacy. The findings of this study showed a statistically significant increase in researcher identity and scientific literacy during the program, which was supported by qualitative evidence from participants’ perspectives that indicated a strong relationship between the development of researcher identity and meaningful engagement with research. Our inquiry adds to the growing body of research focused on the positive impacts of youth involvement in research to address gaps in biomedical research training and disparities in STEM-related careers.

According to the results of the current evaluation, the MYHealth Impact Projects program provided high school participants with a meaningful research experience in an online group setting. The program’s sessions, which took place in small groups, allowed participants to interact and build rapport with faculty, staff, undergraduate and post-graduate facilitators, and other high school participants from across the region as they learned about research. Espinosa (2011) found that among undergraduate students, women of color who persisted in STEM often reported belonging to a community of peers and others engaged in STEM. While we did not measure change in community or belonging as part of the quantitative evaluation, interviews with participants described how the supportive environment created by both mentors (i.e., facilitators, staff, faculty) and peers in the program positively impacted engagement and contributed to their sense of belonging. Such positive mentoring experience can improve motivation, interests, and career outcomes among marginalized groups (Hansen et al., 2024; Hernandez et al., 2017).

Outreach programs, mentorship, and community increase commitment toward STEM careers through the development of science identity and self-efficacy (Chemers et al., 2011, Syed et al., 2019). Science identity — self-perceptions of oneself related to science and a science community (Aschbacher et al., 2010) — plays a vital role in interest and persistence toward STEM-related degrees and careers by supporting individuals to feel connected to and capable of science (Chemers et al., 2011; Stets et al., 2017; Vincent-Ruz & Schunn, 2018). For example, a recent quantitative study on the participation of adolescent girls in different informal science programs reported reciprocal associations between STEM identity and self-efficacy with implications on science career interests (Zhao et al., 2024). Recently, researcher identity has been explored as a means to increase commitment toward research careers, which can occur in outreach programs like MYHealth Impact Projects (Koo et al., 2021). Our study demonstrated statistically significant growth in researcher identity during the program alongside qualitative descriptions of increased confidence in research knowledge and skills. In interviews, participants in our study described how applied research activities helped them to build competency in skills similar to those used by experts in the field. Several participants described beginning to see themselves as researchers.

Traditional science classroom settings that are heavily lecture and test-based, where students are expected to learn largely by memorization, are not conducive to developing scientific thinking and understanding (Lujan & DiCarlo, 2005). During MYHealth Impact Projects, participants learned to think, investigate, and communicate like researchers by engaging in scientific inquiry to answer real-world questions relevant to people their age. These are disciplinary literacy practices essential in understanding and building knowledge in a given discipline (Watts-Taffe et al., 2021). In interviews, participants described examples of how they increased their research knowledge, specifically in their understanding of the research cycle and qualitative research methods, which further supported their confidence in current and future research opportunities. Our evaluation found a significant increase in science literacy from pre- to post-program, which is contextualized by students’ evaluations of their own skills, where many know they are still at a novice level and need to conduct more research independently. Part of their development of researcher identity and research self-efficacy is being able to self-evaluate and have a potential plan in place to get to skill mastery (Blackmore et al., 2021; Ramdass & Zimmerman, 2008).

Our study supports that an online outreach program can support development of researcher identity through authentic research experiences. The YPAR approach underlying MYHealth provided opportunities for participants to actively engage in real-world research processes as members of our research team. Despite intending to incorporate youth voice in research, many YPAR programs have engaged with youth in a more passive manner where the youth participants were not involved in all aspects of the research process (Jacquez et al., 2013). As a result, youth in YPAR programs often do not have the opportunity to learn about or practice research skills, which may limit the impact on their research interest, identity, and self-efficacy. Traditional research frameworks prioritizing academic expertise over lived experience can create barriers to participation and reinforce negative beliefs about belonging in research (Bradbury-Jones et al., 2018; Bradbury-Jones & Taylor, 2015). Our qualitative findings indicate that acquiring actual research skills and being treated as knowledgeable, capable, and active members of the research team can foster a sense of belonging with mentors and peers that can positively contribute to researcher identity, research self-efficacy, motivation, and engagement. Moreover, YPAR programs may help youth connect with science as a tool for generating social and community change (Worker et al., 2023). Attention to how YPAR programs are actively engaging youth in the development of research skills, decision-making, power sharing, and communication is needed to ensure that youth are given the opportunity to meaningfully participate in research to build research self-efficacy and identity.

Out-of-school programs’ attendance is not well studied, and variations in attendance can differ drastically between programs (Roth et al., 2010) and negatively impact how programs are evaluated (Leos-Urbel, 2015). All aspects of MYHealth Impact Projects took place virtually, and despite expected remote fatigue (Bullock et al., 2022) following the switch to online learning during the COVID-19 pandemic, students remained engaged, attending an average of 84% of program sessions across 34 hours of programming. This has significant implications for YPAR implementation in remote/virtual formats where technology can be leveraged to scale research training access to youth, particularly those from marginalized backgrounds.

Limitations

This study has a few limitations that must be considered. We selected the Science Motivation Questionnaire to measure diverse aspects of motivation toward STEM, including intrinsic and extrinsic motivation. However, several of the scales are focused on school-based science learning and the scale has been validated primarily in school contexts. Participant interviews revealed that while their understanding of “research” broadened over the course of the program, some maintained a distinction between qualitative research and “science.” Similarly, a study on science identity in middle school-aged youth reported significant changes in results when the term “science” was explicitly used in survey items (Hill et al., 2018). As such, additional research is needed to explore the nuances in these terms and validate a scale of research motivation in this population.

High school students who enroll in a research training program may be different from their peers. For example, most participants enrolled in MYHealth Impact Projects indicated an interest in future biomedical careers at baseline. While previous interest in or experience with STEM is not required to participate, students with an aptitude for and interest in research may be more likely to sign up. As a result, we actively worked to include students with no outside research or STEM experiences in our program. In addition, while all participants were invited to participate in the interviews, only 10 completed them. Those who were not interested in participating in the interviews may have had a different experience with the program that is not represented in the findings. Future iterations of our program hope to maintain engagement with participants to understand variations in program participation and potential program challenges that are not fully described in this data.

Lastly, participants interviewed were asked to reflect on their experiences over the course of their participation in the MYHealth Impact Projects, which lasted 9 months. Many mostly remembered the most salient parts of their experiences, such as the enjoyable and challenging parts of the research process. Future iterations could benefit from incorporating more instances of recorded self-reflection from students throughout the program.

Conclusion

During the MYHealth Impact Projects, high school participants primarily from historically underrepresented backgrounds conducted their own adolescent health research during the academic year. Our mixed methods evaluation indicated increases in participants’ researcher identity and scientific literacy (with statistical significance) and no changes of statistical significance in research self-efficacy, and science motivation. These quantitative findings were supported by meaningful engagement in research, a supportive learning environment, a broader view of research, and future interest in research opportunities participants reported in the qualitative data. Our findings add to the growing body of literature that describes how STEM outreach programs that provide authentic and positive research experiences to youth can be a supportive path for growing interest and persistence in STEM among high school students.