Skip to main content
NCBI home page
PLOS One logoLink to PLOS One
. 2022 Dec 1;17(12):e0278453. doi: 10.1371/journal.pone.0278453

A systematic review of the 60 year literature: Effects of outreach programs in supporting historically marginalized and first-generation, low-income students in healthcare education

Eric Jenkins 1,#, Jocelyn Elizabeth Nardo 1,*,#, Shima Salehi 1
Editor: Nabeel Al-Yateem2
PMCID: PMC9714932  PMID: 36454878

Abstract

We have reviewed over 60 years of studies on healthcare education outreach programs that are aimed to support first-generation, low-income, as well as underrepresented racial and ethnic minority groups (historically marginalized students) to pursue pre-health professions. As a systematic literature review, we present the challenges studies on healthcare education outreach programs had as three main categories: 1) Design, 2) Evaluation, and 3) Analysis. 1) Designs of studies on healthcare education outreach programs often lacked theoretical foundations whereby a) the interventions did not present theories underlying a causal mechanism of inequity in health professions; and/or 1b) the defined outcome measures were not clearly aligned with the problem the intervention tried to address. 2) Evaluations of studies on healthcare education outreach programs were not always conducted effectively whereby: 2a) controlled groups were commonly absent for comparison with the intervention group; and/or 2b) post measures were solely used without pre-measures. 3) Analyses of studies on healthcare education outreach programs were not adequate whereby: 3a) the response rates and effect size were commonly low; and/or 3b) qualitative results commonly did not supplement quantitative results. Overall, our findings reveal studies on healthcare education outreach programs have common challenges that hinder the reliability of their effects supporting historically marginalized students in pursuing pre-health professions. To address such challenges with studies on healthcare education outreach programs aimed at supporting historically marginalized students, we created a decision flow chart for researchers to ask themselves: 1) how is the design guided by theoretical goals; 2) how are measurements used to evaluate success; and 3) how does the analysis lead to reliable results?

Introduction

Originating from long-standing social inequity, healthcare disparity in the United States for first-generation Americans, persons from low socioeconomic backgrounds, and underrepresented racial and ethnic minority groups is growing especially in light of the COVID-19 pandemic [1]. Healthcare disparity is impacted by healthcare professionals’ ability to empathize with patients, which can be limited by compassion burnout [2] and cultural competency [3]. However, healthcare professionals who are first-generation Americans, persons from low socioeconomic backgrounds, and Black, Indigenous, People of Color (BIPOC) have greatly contributed to how medical conditions such as strokes, heart attacks, and cancer impact subpopulations differently, which reflects how compassion and cultural competency often originate at the intersection of lived experience and medical training [4]. Thus, researchers have hypothesized healthcare disparity can be addressed by supporting students who are first-generation Americans, persons from low socioeconomic backgrounds, and underrepresented racial and ethnic minorities (hereafter, historically marginalized students) to become healthcare professionals [4] since historically marginalized students are often multilingual [5], have financial knowledge from navigating the healthcare system from the position of a person from a low socioeconomic background [6] and have the necessary compassion and cultural competency [7]. Unfortunately, a massive education disparity also exists for historically marginalized students as compared to their historically non-marginalized counterparts to pursue healthcare professions [8].

Many private philanthropic organizations, academic systems, and federal or state governments have allocated money and resources for recruitment of historically marginalized students into medical, dental, pharmaceutical, and other post-secondary programs to mitigate the education disparity [9, 10]. Examples of recruitment strategies for healthcare programs consist of soliciting and recruiting students from diverse areas of the United States to pursue healthcare professions by waiving application fees [11] or lowering qualifying exam scores [12]. The argument for recruitment-based strategies derives from examining the pipeline of healthcare professionals [13]. Essentially, the pipeline metaphor posits that creating a more diverse healthcare, and in general STEM, workforce necessitates plugging the metaphorical “leaks.” The leaks refer to the loss of historically marginalized students interested in these professions throughout their education pathways [14]. Yet, nearly 60 years of active recruitment of historically marginalized groups into post-secondary programs to address the education disparity have still not addressed the growing underrepresentation of such groups within healthcare professions, resulting in the current healthcare disparity [15, 16].

Therefore, the pipeline metaphor for creating a more diverse STEM workforce should be critically elaborated by considering how institutional, societal, and cultural factors shape the trajectories of historically marginalized students [14, 17–20]. Moving away from the pipeline metaphor that focuses mostly on recruitment [14], research has suggested providing resources, networks, and opportunities early on through healthcare education outreach programs for the intrinsic and societal benefits of historically marginalized students [21–23]. Healthcare education outreach programs have been shown to facilitate the educational pathways of historically marginalized students to pursue STEM majors, leading to long-term careers in healthcare professions [23]. For example, Stray and colleagues found that the ongoing (1992-present) collaboration between faculty mentors at the University of Mississippi Medical Center (UMMC) and local high school teachers and students increased high school graduation rates, college application submissions, enrollment in STEM baccalaureate majors, and continued STEM enrollment for master’s degrees as well as degrees in healthcare professions [22]. This was due to UMMC’s support of the local community, mentorship of science teachers, and inquiry-based classroom activities that were funded by UMMC. The study at UMMC drew from the Mississippi Science Framework that theorized program components such as student research mentorship, teacher professional development, disciplinary content knowledge enrichment, and practical experiences, which has promoted sustainability of the program for 28 years [22].

Hence, the structure of effective healthcare education outreach programs is an emerging direction for researchers to pursue in addressing the education and resulting healthcare disparity in the United States. To understand how healthcare education outreach programs are researched, we employed a systematic literature review [24] to examine studies on healthcare education outreach programs across six decades. We examined how studies on healthcare education outreach programs were designed, evaluated, and analyzed to support historically marginalized students reliably. The review suggests that although the emergent direction can be productive for supporting historically marginalized students to pursue and persist in healthcare professions, there is a notable paucity of evaluated research that can provide sufficient evidence to guide the structure of healthcare education outreach programs [24]. We hope by providing a systematic literature review, we can support researchers structuring and facilitating healthcare education outreach programs that can reliably determine efficacious outcomes for historically marginalized students to pursue future STEM degrees, leading eventually to healthcare professions and dismantling healthcare disparity in the United States. Our systematic literature review seeks to answer:

  • How are studies on healthcare education outreach programs designed by drawing from theory?

  • How are studies on healthcare education outreach programs evaluated using quantitative, qualitative, and mixed-methods?

  • How are studies on healthcare education outreach programs analyzed to generate reliable and generalizable results?

Overview of current review methods

The following literature review is considered a systematic literature review [25], which contains these important features to search and evaluate the literature: eligibility criteria; inclusion criterion; search strategy; literature identification; selection process; data collection; screening for inclusion; data items; quality and eligibility assessment; study risk of bias assessment; reporting bias assessment; effect measures; synthesis methods; and certainty assessment [26]. Table 1 groups alike features together and describes how the feature works in the context of our systematic literature review.

Table 1. Features of a systematic literature review and context.

Features Definition of Feature Context
Eligibility Criteria The breadth of the literature that the authors choose to review. Our systematic literature review draws from science education, health-related journals, and cultural studies in education that are concerned with healthcare education outreach programs. To be considered, the studies selected must be peer-reviewed and written in English.
Inclusion Criterion
Search Strategy The databases used to search the literature that the authors choose to review. We searched within ERIC, American Educational Research Journal, Education Research Complete, MedEdPORTAL, SCOPUS, Web of Science, Academic Search Premier within Stanford’s Lane Library combined database.
Literature Identification
Selection Process The keywords for consideration that the authors choose to review. We searched used the databases using the keywords: mentor* AND (STEMM or STEM or healthcare OR “healthcare” OR medic* OR “health science*”) AND (“high school” OR “secondary school”) AND (minority* OR black OR latin* OR “first gen*” OR “first-gen*” OR underrepresented) AND “outreach” OR “pipeline program.”
Data Collection
Screening for Inclusion
Data Items The process for ensuring that the articles selected from the literature meet the inclusion criterion, literature identification, and screening for inclusion. All three authors were involved in reviewing the abstracts to ensure selected studies contained either qualitative, quantitative, or mixed qualitative and quantitative data as well as reported findings. Discrepancies were discussed together until reaching consensus.
Quality and Eligibility Assessment
Study Risk of Bias Assessment The process for reducing bias in sampling. After the abstracts were reviewed, the first author coded the studies for inclusion and all three authors worked collaboratively to verify the inclusion of each study for review. We examined the articles for their quality of 1) design, 2) evaluation, and 3) analysis and documented features of each article in tables (see S1 File). To examine bias within the articles reviewed, we documented if authors declared any conflicts of interest.
Reporting Bias Assessment
Effect Measures The process of synthesizing and determining themes. The first author coded the data, examining study challenges, theoretical construct, program components, evaluative components, evidence based success, evaluative improvement needed, and study bias. All three authors worked collaboratively to verify the quality of 1) design, 2) evaluation, and analysis) for each study by noting study features, theory, methods, and findings. Discrepancies were discussed together until reaching consensus.
Synthesis Methods
Certainty Assessment
Open in a new tab

Study search, retrieval, eligibility, and selection

All three authors initially searched the following databases: ERIC, American Educational Research Journal, Education Research Complete, MedEdPORTAL, SCOPUS, Web of Science, and Academic Search Premier within Stanford’s Lane Library combined database. Within each database, we searched within the last ten years and used the following search terms: mentor* AND (STEMM or STEM or healthcare OR “healthcare” OR medic* OR “health science*”) AND (“high school” OR “secondary school”) AND (minority* OR black OR latin* OR “first gen*” OR “first-gen*” OR underrepresented) AND “outreach” OR “pipeline program.” However, only 19 articles were generated within the last ten years. Due to the small initial turn-out of articles related to the keywords, the search was expanded to include all articles regardless of publication date. Our second search returned 55 relevant articles from the journals of Academic Medicine, Medical Education, Medical Teacher, and BMC Medical Education of which 18 were excluded due to not being classified by the journal as a research study containing reported findings with qualitative, quantitative, or mixed qualitative and qualitative data (see Fig 1). To be eligible for review, the articles needed to be research studies, which have explicit methods, data, and findings sections. All three authors were involved in the screening for inclusion and quality and eligibility assessment. The majority of the research studies (N = 19) are from the past ten years (see Fig 2) and the majority of research studies (N = 12) from the Journal of Academic Medicine (see Fig 3).

Fig 1. Illustrates the criteria for inclusion of articles.

Fig 1

Open in a new tab

Fig 2. Bar graph showing the range of publication dates for the selected articles in the literature review (N = 37).

Fig 2

Open in a new tab

There is a 10-year range per each article, with most of the articles (n = 19) being within the last ten years.

Fig 3. Chart illustrating the number of articles per publication journal for the literature review (N = 37).

Fig 3

Open in a new tab

Most of the articles are from the Journal of Academic Medicine (n = 12).

Eligibility criteria for quality

Establishing the effectiveness for studies on healthcare education outreach programs required empirical evidence [27], which needed to include quantitative measures like midterm exam scores, final exam scores, concept inventory scores, etc.; and/or qualitative measures like interview data revealing student behaviors, affective factors, intention to pursue healthcare professions, etc. [28]. We categorized the report of each study on the effectiveness of the reported program based on examination criteria listed in Fig 4. For more in-depth analysis of each study on healthcare education outreach programs, see S1A and S1B Table in S1 File [27–31].

Fig 4. Illustrates the criteria for the coding and analytical strategy of the articles reviewed.

Fig 4

Open in a new tab

Coding and analytical strategy

After the sample of (N = 37) was established, the first author coded the articles by noting the barriers, theoretical construct, program components, duration, evaluative components, evidence-based success, evaluative improvement needed and bias of study (See S1A and S1B Table in S1 File). The examination criteria was then organized into 1) Design, 2) Evaluation, and 3) Analysis to assess the adequacy of each study on healthcare education outreach programs. Authors worked collaboratively to determine the studies included in the systematic review had qualities for 1) design, 2) evaluation, and 3) analysis. The three criteria are crucial for any study design because they are interdependent, meaning the effects of design will carry over into evaluation and the effects of evaluation will carry over into analysis. 1) Design refers to how studies on healthcare education outreach programs used theoretical considerations from the literature to support access for historically marginalized students. 2) Evaluation refers to how the studies on healthcare education outreach programs collected data for measurement. Finally, 3) Analysis refers to how studies on healthcare education outreach programs verified the effectiveness for historically marginalized students pursuing future healthcare professions. The studies were determined to be adequate if the studies answered the questions in Fig 4.

Results

Overall, the results of the systematic literature review suggest that there is a need for studies on healthcare education outreach programs to integrate empirical data with practitioner expertise to understand the unique barriers of historically marginalized students pursuing healthcare professions. Moving away from the pipeline metaphor and towards sustainable progress, studies on healthcare education outreach programs should 1) understand the main barriers to historically marginalized students entering, persisting, and succeeding in healthcare professions; 2) the protective factors that are correlated with increased entry, success, and persistence into healthcare professions; and finally, 3) the components that can successfully mitigate barriers or facilitate protective factors [18–22]. Table 2 Illustrates the percentage of studies that satisfied each category of the examination criteria. Of the N = 37 studies from the literature review, 54.2% (n = 20) met the design criteria, meaning that the studies on healthcare education outreach programs had a theoretical framework that guided design. Similarly, 54.2% (n = 20) of the studies on healthcare education outreach programs met the design and evaluation criteria, meaning studies had accurate baseline comparison (e.g., pre-measurements and post-measurements) and had an appropriate comparison group. Finally, only 4.2% (n = 2) of the studies on healthcare education programs met the design, evaluation, and analysis criteria together, meaning studies had a proper response rate and presented quantitative and/or qualitative results effectively. All articles in the review had no conflicts of interest to report, indicating low risk for bias. For full study characteristics of each article presented in the literature review, see S1A and S1B Table in S1 File. For each of the criteria, we will present the challenges, sub-challenges, and examples of selected studies on healthcare education outreach programs from the systematic literature review.

Table 2. Percentage of reviewed articles that met the selected criteria (N = 37).

Percentage of Programs Meeting Design Criteria Percentage of Programs Meeting Design and Evaluation Criteria Percentage of Programs Meeting Design Evaluation, and Analysis Criteria
54.2% (n = 20) 54.2% (n = 20) 4.2% (n = 2)
Open in a new tab

1. Challenges in design

We considered studies on healthcare education outreach programs to be adequately designed if there was an explicit theoretical framework section and the study goals could be supported by the measured outcomes. Complete theoretical framework sections contained hypotheses/research questions which drew from either literature, prior research performed, or an existing model. We considered studies on healthcare education outreach programs to be inadequately designed if there was no theoretical framework section or the study goals could not be supported by the measured outcomes. Incomplete theoretical framework sections did not contain hypotheses/research questions that drew from either literature, prior research performed, or an existing model. From the inadequately designed studies on healthcare education outreach programs, we determined two main challenges, which we have labeled as a) incomplete or missing theoretical framework; and b) misaligned study goals.

  • a) Incomplete or missing theoretical framework. From the systematic literature review, 54.2% (n = 20) of the studies on healthcare education outreach programs contained an explicitly labeled theoretical framework, hypotheses/research questions, and supported study goals. On the other hand, 45.8% (n = 17) of the studies on healthcare education outreach programs either did not have an explicit theoretical framework section, had missing hypotheses/research questions, or unsupported study goals (see Fig 5). For example, a study that tracked gifted historically marginalized students who participated in the Student Educational Enrichment Program (SEEP) through a questionnaire had no theoretical framework section to define what was meant by gifted [32]. Accordingly, the outcomes of the study explained that gifted historically marginalized students were more likely than their non-gifted counterparts to pursue healthcare professions; however, determination was predicated only on interest rather than enrollment or graduation numbers. Overall, the studies on healthcare education outreach programs without theoretical frameworks were usually from before 2010 (n = 16), indicating a trend that recently published research recognizes the value of theoretical frameworks in design (see Fig 2).

Fig 5. Shows that 54.2% of programs studied specifically detailed a theoretical framework for the design of their intervention, 16.7% of programs studied mentioned but did not specifically detail a theoretical framework for the design of their intervention, and 29.1% of programs studied did not reference any theoretical framework.

Fig 5

Open in a new tab

  • b) Misaligned study goals. Of the 54.2% studies on healthcare education outreach programs with detailed theoretical frameworks, 54.2% (n = 11) had study goals that aligned with the theoretical framework while 45.8% did not have aligned study goals (see Fig 6). The studies on healthcare education outreach programs with detailed theoretical frameworks had missions to provide academic and disciplinary content knowledge enrichment and mentorship through shadowing or interning with a healthcare professional (for examples see [33–36]). The structure of these healthcare education outreach programs had sustainable components such as: scholarly presentations, college immersion experiences, mentorship greater than 1-year, medical content exposure, and/or college admissions/application counseling. In contrast, unaligned studies on healthcare education outreach programs provided less sustainable components like no mentorship or had mentorship that was less than one year (See Fig 7) [37, 38] despite these studies mentioning in their literature reviews and discussions that long-term mentorship was necessary to retain historically marginalized students in STEM degrees, leading to healthcare professions [39–41].

Fig 6. Shows that 54.2% of programs studied aligned their evaluation design with their specifically detailed theoretical framework and 45.8% of programs studied either did not align their evaluation with their theoretical framework or did not reference any theoretical framework.

Fig 6

Open in a new tab

Fig 7. Shows that 45.0% of programs studied did not include a mentorship component.

Fig 7

Open in a new tab

Of the programs with a mentorship program, 35.0% implementing mentorship programs of less than a year’s duration, 20.0% implementing mentorship programs greater than one year’s duration.

2. Challenges in evaluation

We considered studies on healthcare education outreach programs to have adequate evaluation if there was an appropriate comparison group along with pre-measurements and post-measurements. The most common evaluative measurement tools among studies on healthcare education outreach programs were surveys that captured interest, awareness of healthcare careers, college enrollment, and college graduation. Appropriate comparison groups were those that closely matched key characteristics (e.g., demographic, socioeconomic, and first-generation backgrounds) of the experimental group while appropriate pre-measurements and post-measurements established a baseline to examine the efficacy of the healthcare education outreach programs. We considered studies on healthcare education outreach programs to have inadequate evaluation if there were no comparison groups, an inappropriate comparison group (i.e., key characteristics did not match) or if there were no pre-measurements and post-measurements. From the inadequately evaluated studies on healthcare education outreach programs, we determined two main challenges, which we have labeled as a) inappropriate comparison groups; and b) no pre-baseline measurements.

  • a) Inappropriate comparison groups. From our systematic literature review, 15.0% (n = 6) of studies on healthcare education outreach programs had appropriately matched comparison groups, which were generated by comparing outcomes of historically marginalized students enrolled in the healthcare education outreach programs with the outcomes of counterparts from alike schools, states, demographic backgrounds, levels of household income, and/or parent’s generational status [33]. Conversely, 55.0% (n = 20) of studies on healthcare education outreach programs did not have any comparison groups while 30.0% had inappropriately matched comparison groups (see Fig 8). Studies with no comparison groups had sparse data, typically using recruitment numbers into the healthcare education outreach program and enrollment into undergraduate education as the sole measure for efficacy [42, 43]). Moreover, studies with inappropriate comparison groups often 1) compared historically marginalized students to their historically non-marginalized counterparts; 2) compared historically marginalized students without matching key characteristics; or 3) compared historically marginalized students with no other group. These studies did not compare outcomes of historically marginalized students with counterparts not enrolled in the healthcare education outreach programs and often screened based on GPA, prior coursework, and academic record (for examples see [44, 45]). Unfortunately, 45.8% (n = 17) of studies also generated comparison groups using a competitive application process to select historically marginalized students who have already demonstrated evidence of science interest, efficacy, and academic success to participate in their healthcare education outreach program (see Fig 9).

Fig 8. Of the N = 20, only 15.0% of programs studied correctly used a matched comparison group in their study design, 30.0% of programs studied erroneously used an inappropriate comparison group in their study design, and surprisingly 55.0% of programs studied neglected to use any comparison group in their study design.

Fig 8

Open in a new tab

Fig 9. Shows that 45.8% of programs studied used a competitive application process when selecting which URM and FGLI students would be allowed to participate in their healthcare pipeline program and 54.2% did not.

Fig 9

Open in a new tab

  • b) No pre-baseline measurements. As shown in Fig 10, 22.7% (n = 8) of studies on healthcare education outreach programs used pre-measurements and post-measurements to evaluate efficacy. Pre-measurements were typically data collected before or within the first two weeks the healthcare education outreach program and used to create appropriate comparison groups [33]. Examples of pre-baseline measurements include data measuring content knowledge, interest, belonging, identity, or self-efficacy in pursuing healthcare professions [46, 47]. However, 68.2% (n = 25) of studies did not generate a baseline of historically marginalized students’ current academic performance in and motivation of pursuing healthcare professions. Accordingly, studies on healthcare education outreach programs that only collected post-measurements neglected the opportunity to compare historically marginalized students to their pre-measurement selves. Thus, there was no way to reliably evaluate how these studies on the healthcare education outreach programs supported historically marginalized students to pursue healthcare professions [37, 47, 48].

Fig 10. 22.7% of programs studied correctly used both pre-program and post-program measures, 9.1% used post-only measurements, and surprisingly 68.2% neglected to use either pre or post-program measures.

Fig 10

Open in a new tab

3. Challenges in analysis

We considered studies on healthcare education outreach programs to have adequate analysis if there was a substantial response rate and quantitative and/or qualitative analysis. Substantial response rate should be above 80% with a large enough sample size for quantitative and/or qualitative analysis [49]. We considered studies on healthcare education outreach programs to have inadequate analysis if the response rate was below 80% and if qualitative data was not used to supplement for small sample size. From the inadequately analyzed studies on healthcare education outreach programs, we determined two main challenges, which we have labeled as a) low response rate and effect size; and b) underutilization of qualitative methods.

  • a) Low response rate and effect size. From our systematic literature review, 37.5% (n = 14) had a reported response rate above 80%, meaning that of the data collected, at least 80% of participants’ data was included in the analysis (example [50]). Studies with high response rates disaggregated by historically marginalized group and had generalizable outcomes such as the mean performance of historically marginalized students that afforded enrollment into undergraduate education and eventually graduation. In contrast, 40.0% (n = 15) of studies had response rates of less than 60.0% and tended to have competitive application processes [51]. These studies had small sample sizes that could not determine effect size, statistically significant components, or generalizable results (example [33, 37, 51]). Finally, 29.2% (n = 11) of the data did not report any response rate (see Fig 11).

Fig 11. Shows that 37.5% of programs studied had response rates above 80%; 20.8% of programs had response rates between 60% and 80%; 20.8% had a response rate below 60% and 29.2% had no reported response rate.

Fig 11

Open in a new tab

  • b) Underutilization of qualitative methods. Fig 12 shows 24.0% (n = 9) of studies on healthcare education outreach programs utilized qualitative data in the analysis. Examples of qualitative data included student interviews [36, 45, 52]) and open-response text from surveys (example [53]). The role of the qualitative data was to triangulate with the quantitative findings (example [36]) to supplement for small sample size. In our systematic literature review, there were no studies on healthcare education outreach programs that used only qualitative data. However, 76.0% (n = 28) of studies on healthcare education outreach programs utilized only quantitative data analysis, which included a variety of parametric and non-parametric tests.

Fig 12. Shows that 24.0% of the studies on outreach programs utilized a combination of qualitative and quantitative data together, whereas the majority at 76.0% only utilized quantitative methods.

Fig 12

Open in a new tab

Limitations, discussion, and implications

The systematic literature review contains the following limitations that impact how these results should be interpreted: 1) the number of reviewed articles (N = 37) was too small for an in-depth quantitative analysis; 2) we only included research with explicit methods, data, and findings sections, which may have obscured the number of studies that contained only descriptions of research efforts; 3) we used the declaration of conflicts of interest in each article to measure bias, which is not an encompassing approach [54]. Nevertheless, we believe despite these limitations, the findings generate important discussions surrounding the efficacy of healthcare programs for historically marginalized students. The following section will elaborate on why design, evaluation, and analysis are integral features to study efficacy of outreach programs by drawing on related literature in cognitive psychology, science education, and discipline-based education research. To elaborate why these features are effective, we will also provide examples on how to improve the quality of research. Fig 14 below illustrates study features and highlight components for effective studies on healthcare education outreach programs that researchers can use to support their practice.

Fig 14. Illustrates the connection among design, evaluation, and analysis features.

Fig 14

Open in a new tab

Beginning with design, most of the studies on healthcare education outreach programs did not have theoretical frameworks. Lack of theory in study design is a major challenge because theoretical frameworks help researchers guide hypotheses [55]. Through theory-driven hypotheses, causal mechanisms can be generated that operationalize variables of interest with the expected outcomes, which would have helped some studies on healthcare education outreach programs to better structure program components like mentorship and content learning [8]. Without theoretically-driven hypotheses, studies on healthcare education outreach programs are less likely to have an effective design that can be properly tested by selecting previously validated instruments and/or methods from the literature. The focus on theoretical frameworks and resulting hypotheses is also crucial for replicating structures across studies on healthcare education outreach programs. Explicitly using and naming theoretical frameworks has been shown to develop interdisciplinary research [56], which can address complex problems like healthcare disparity in the United States through reliable studies on healthcare education outreach programs that support historically marginalized students to pursue healthcare professions.

Within our literature review, studies on healthcare education outreach programs that missed the design criteria often lacked a mentorship component (e.g., college preparation counseling curriculum or advising), which has been shown to be a protective factor in supporting historically marginalized students pursuing STEM degrees and eventually healthcare professions [55–57]. Mentorship often provides historically marginalized students with social capital, which is a sociological term from Pierre Bourdieu that shows how access to knowledges, resources, and experiences can better facilitate social navigation [57]. Essentially, mentorship provides historically marginalized students access to knowledge about selecting a college, financial aid information/scholarships, and knowing how to enroll when accepted, which is more challenging for students who are first-generation scholars or a person from a low socioeconomic background without existing family members in academia and in healthcare professions [56, 57]. In contrast, studies on healthcare education programs that met the design criteria not only had a mentorship component for historically marginalized students, but also for the families [39]. For example, college-advising workshops and parent-support workshops were shown to help historically marginalized students and their families understand expected coursework, financial aid opportunities for tuition and supplies, as well as on-campus tutoring and learning centers [39].

Analogously, studies on healthcare education outreach programs that met the design criteria also heavily emphasized teaching historically marginalized students the disciplinary content knowledge needed to be successful in STEM majors and therefore in healthcare professions. Our systematic literature reveals 25.0% (n = 9) of studies on healthcare education outreach programs did not have disciplinary content knowledge enrichment, while 75% (n = 28) had disciplinary content knowledge enrichment. Of the studies with disciplinary content knowledge enrichment, 66.7% (n = 19) of studies had biology enrichment, 41.7% (n = 12) had physics enrichment, 33.3% (n = 9) had math enrichment, and 4.2% (n = 1) had chemistry enrichment (see Fig 13); studies on healthcare education programs had multiple enrichments, meaning percentages will be over 100%. Previous works have shown that preparation gaps exist between historically marginalized and historically non-marginalized students [58] and that introductory chemistry serves as a major gatekeeper for historically marginalized students to pursue STEM degrees and eventually healthcare professions [58]. The documented preparation gaps in mathematics and science courses like biology, chemistry, and physics further highlight the education disparity for historically marginalized students compared to their counterparts. Thus, studies on healthcare education outreach programs that emphasized disciplinary content knowledge enrichment, especially for chemistry, illustrate how the use of theory can guide structures that best serve historically marginalized students pursuing healthcare professions.

Fig 13. Shows that 66.7% of programs had biology enrichment, 41.7% had physics enrichment, 33.3% had math enrichment, and 4.2% had chemistry enrichment.

Fig 13

Open in a new tab

25.0% of studies had no content enrichment.

In terms of the evaluation criteria, most studies on healthcare education outreach programs had challenges evaluating differences between historically marginalized and historically non-marginalized students. As mentioned previously, historically marginalized students face disproportionate education disparity when compared to their counterparts [59]; consequently, academic preparedness is a salient feature to consider for structuring healthcare education outreach programs. However, academic preparedness in studies on healthcare education outreach programs often used academic preparedness to discriminate applicants, favoring historically marginalized students who have already demonstrated academic proficiency and an inclination towards pursuing STEM degrees [60]. Instead, some studies on healthcare education outreach programs have accepted students into the program if historically marginalize students had low academic preparedness (i.e., GPA around 3.0) and had a neutral to slightly positive inclination towards pursuing healthcare professions [38, 50]. To address challenges in evaluating differences between historically marginalized and historically non-marginalized students, some studies on healthcare education outreach programs have suggested matching applicants based on one or two similar criteria such as underrepresented minority status, household income level, parental highest education achievement, and baseline GPA rather than evaluating program success in aggregate [32, 50].

Finally, the analysis criteria revealed most studies on healthcare education outreach programs needed either larger sample sizes or longitudinal data to verify claims that the healthcare education outreach program had long-term effects on historically marginalized students pursuing STEM majors and healthcare professions. When analyzing self-reported measurements, response rate is critical because small sample sizes can significantly decrease validity and generalizability of findings [60]. In quantitative analysis, the study outcomes begin to lose validity when response rates fall below 60% due to the uncertainty that the sample of respondents accurately reflects elements of the complete population of interest [61]. To address low response rates, researchers should ask whose responses are missing and what nonresponse bias means for the study goals. For historically marginalized students, nonresponse bias can reveal that an aspect of the healthcare education outreach program is inaccessible (e.g., requiring high academic preparation) [38, 50]. Researchers should also provide information about the sampling process employed, response rates calculations, as well as utilize qualitative research methods to strengthen the evidence being reported, particularly when it challenges the sample size required for quantitative analyses.

Furthermore, studies on healthcare education outreach programs were understandably constrained financially since they were self-supported through grant or university allocation funding. Thus, studies lacked the funding to achieve long-term follow up on whether historically marginalized students in the healthcare education outreach programs went on to pursue healthcare professions. To address challenges with acquiring long-term follow up with historically marginalized students, studies have used the completion of college pre-med freshman and sophomore science coursework to serve as sufficient predictors of persistence and future STEM major graduation [62]. Multiple modes of contact, follow-up appearance, incentives, personalization, and longitudinal mentorship were also used, which had significant impacts on survey response rates after the healthcare outreach program was completed [49]. Researchers can also combine quantitative pre- and post-program measures with qualitative research methods using an appropriate theoretical framework to support the reliability of the analysis [63].

Overall, we strongly suggest researchers invested in reducing disparities of racial and ethnic groups in the healthcare field utilize a guiding theoretical framework to design studies on healthcare education outreach programs and constructs of interest, which will then support proper evaluation and analyses. For proper evaluation, it is important to make sure there is an appropriate comparative group and pre-and post-measures to measure the efficacy of the program. To analyze the efficacy, researchers should make sure their results are reliable and generalizable by ensuring representative sample size for quantitative analysis and/or triangulating quantitative analysis with in-depth qualitative analysis. Fig 14 shows a detailed flow chart of decisions researchers should consider when structuring healthcare education outreach programs.

Conclusion

To support historically marginalized students’ pursuit of and persistence in healthcare professions, studies on healthcare education outreach programs should structure components that foster sustainability through mentorship, financial support, and disciplinary content knowledge enrichment. It is crucial that early efforts to structure healthcare education outreach programs deliberately address equity barriers that historically marginalized students disproportionately face in pursuing healthcare professions through the education system. Deliberately addressing equity barriers for historically marginalized students includes doing studies with 1) a strong theoretical foundation with aligned study goals; 2) an appropriate comparison group with measurements taken before, during, and after the healthcare education outreach program; and 3) a reliable method to ensure the outcomes are replicable and generalizable. Ultimately, addressing challenges in design, evaluation, and analysis can better support future researchers interested in studying healthcare education outreach programs to structure them successfully for historically marginalized students to pursue STEM degrees, leading them eventually to pursue high-paying careers in healthcare professions. The advancement of historically marginalized students is a critical step towards achieving equity and social justice for historically marginalized communities and as healthcare professionals, historically marginalized students would therefore be important frontier agents to mitigate healthcare disparity in the United States.

Supporting information

S1 Checklist

(DOCX)

Click here for additional data file. (20.5KB, docx)
S1 File

S1A Table: Barriers, assumptions, components, evaluative methods, conclusions and evidence-based success. S1B Table: Recommendations, reasoning, and conclusions of non-program high school healthcare education pipeline research.

(DOCX)

Click here for additional data file. (443.8KB, docx)

Data Availability

All relevant data are within the paper and its Supporting Information files.

Funding Statement

The author(s) received no specific funding for this work.

References

  • 1.Mackey K. et al., “Racial and ethnic disparities in COVID-19–related infections, hospitalizations, and deaths: a systematic review,” Annals of internal medicine, vol. 174, no. 3, pp. 362–373, 2021. doi: 10.7326/M20-6306 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Smart D. et al. , “Compassion fatigue and satisfaction: A cross‐sectional survey among US healthcare workers,” Nursing & health sciences, vol. 16, no. 1, pp. 3–10, 2014. doi: 10.1111/nhs.12068 [DOI] [PubMed] [Google Scholar]
  • 3.Ward A., Mandrusiak A., and Levett-Jones T., “Cultural empathy in physiotherapy students: a pre-test post-test study utilising virtual simulation,” Physiotherapy, vol. 104, no. 4, pp. 453–461, 2018. doi: 10.1016/j.physio.2018.07.011 [DOI] [PubMed] [Google Scholar]
  • 4.Skolarus L. E., Sharrief A., Gardener H., Jenkins C., and Boden-Albala B., “Considerations in addressing social determinants of health to reduce racial/ethnic disparities in stroke outcomes in the United States,” Stroke, vol. 51, no. 11, pp. 3433–3439, 2020. doi: 10.1161/STROKEAHA.120.030426 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Horváth Á. and Molnár P., “A review of patient safety communication in multicultural and multilingual healthcare settings with special attention to the US and Canada,” Developments in Health Sciences, 2021. [Google Scholar]
  • 6.Edward J., Morris S., Mataoui F., Granberry P., Williams M. V., and Torres I., “The impact of health and health insurance literacy on access to care for Hispanic/Latino communities,” Public Health Nursing, vol. 35, no. 3, pp. 176–183, 2018. doi: 10.1111/phn.12385 [DOI] [PubMed] [Google Scholar]
  • 7.Marrast L. M., Zallman L., Woolhandler S., Bor D. H., and McCormick D., “Minority Physicians’ Role in the Care of Underserved Patients: Diversifying the Physician Workforce May Be Key in Addressing Health Disparities,” JAMA Internal Medicine, vol. 174, no. 2, pp. 289–291, Feb. 2014, doi: 10.1001/jamainternmed.2013.12756 [DOI] [PubMed] [Google Scholar]
  • 8.Camacho A., Zangaro G., and White K. M., “Diversifying the health-care workforce begins at the pipeline: a 5-year synthesis of processes and outputs of the Scholarships for Disadvantaged Students program,” Evaluation & the health professions, vol. 40, no. 2, pp. 127–150, 2017. doi: 10.1177/0163278715617809 [DOI] [PubMed] [Google Scholar]
  • 9.Naylor K. B., Tootoo J., Yakusheva O., Shipman S. A., Bynum J. P., and Davis M. A., “Geographic variation in spatial accessibility of US healthcare providers,” Plos one, vol. 14, no. 4, p. e0215016, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Cyr M. E., Etchin A. G., Guthrie B. J., and Benneyan J. C., “Access to specialty healthcare in urban versus rural US populations: a systematic literature review,” BMC health services research, vol. 19, no. 1, pp. 1–17, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Gonzaga A. M. R., Appiah-Pippim J., Onumah C. M., and Yialamas M. A., “A framework for inclusive graduate medical education recruitment strategies: meeting the ACGME standard for a diverse and inclusive workforce,” Academic Medicine, vol. 95, no. 5, pp. 710–716, 2020. doi: 10.1097/ACM.0000000000003073 [DOI] [PubMed] [Google Scholar]
  • 12.Williams M. et al. , “The impact of United States Medical Licensing Exam (USMLE) step 1 cutoff scores on recruitment of underrepresented minorities in medicine: A retrospective cross‐sectional study,” Health science reports, vol. 3, no. 2, p. e2161, 2020. doi: 10.1002/hsr2.161 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Bouye K. E., McCleary K. J., and Williams K. B., “Increasing diversity in the health professions: reflections on student pipeline programs,” Journal of healthcare, science and the humanities, vol. 6, no. 1, p. 67, 2016. doi: 10.5888/pcd10.130186 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Cannady M. A., Greenwald E., and Harris K. N., “Problematizing the STEM Pipeline Metaphor: Is the STEM Pipeline Metaphor Serving Our Students and the STEM Workforce?,” Science Education, vol. 98, no. 3, pp. 443–460, 2014, doi: 10.1002/sce.21108 [DOI] [Google Scholar]
  • 15.Toretsky C., Mutha S., and Coffman J., “Breaking barriers for underrepresented minorities in the health professions,” Retrieved from Healthforce Center at UCSF website https://healthforce.ucsf.edu/publications/breaking-barriers-underrepresented-minorities-health-professions, 2018. [Google Scholar]
  • 16.Crowe R. P., Krebs W., Cash R. E., Rivard M. K., Lincoln E. W., and Panchal A. R., “Females and minority racial/ethnic groups remain underrepresented in emergency medical services: a ten-year assessment, 2008–2017,” Prehospital Emergency Care, vol. 24, no. 2, pp. 180–187, 2020. doi: 10.1080/10903127.2019.1634167 [DOI] [PubMed] [Google Scholar]
  • 17.Lucena J. C., “MAKING WOMEN AND MINORITIES IN SCIENCE AND ENGINEERING FOR NATIONAL PURPOSES IN THE UNITED STATES,” Journal of women and minorities in science and engineering, vol. 6, no. 1, p. 32, 2000, doi: 10.1615/JWomenMinorScienEng.v6.i1.10 [DOI] [Google Scholar]
  • 18.Lykkegaard E. and Ulriksen L., “In and out of the STEM pipeline–a longitudinal study of a misleading metaphor,” International Journal of Science Education, vol. 41, no. 12, pp. 1600–1625, Aug. 2019, doi: 10.1080/09500693.2019.1622054 [DOI] [Google Scholar]
  • 19.Metcalf H., “Stuck in the pipeline: A critical review of STEM workforce literature,” InterActions: UCLA Journal of Education and Information Studies, vol. 6, no. 2, 2010. [Google Scholar]
  • 20.Metcalf H. E., “Disrupting the pipeline: Critical analyses of student pathways through postsecondary STEM education,” New Directions for Institutional Research, vol. 2013, no. 158, pp. 77–93, 2014. [Google Scholar]
  • 21.Gallant C., Bork P., Carpenter-Cleland C., and Good D., “Examining the impact of a 2-day scientific conference on high school students’ interest in STEM and confidence in attending university,” Canadian Journal of Science, Mathematics and Technology Education, vol. 20, no. 2, pp. 376–387, 2020. [Google Scholar]
  • 22.Stray S. et al. , “Base Pair: 28 years of Sustained High School Biomedical Research Mentorship Driving Health Sciences Career Progression.,” Journal of STEM Outreach, vol. 3, no. 3, pp. 1–10, 2020. [Google Scholar]
  • 23.RAFANAN R. J. and De Guzman C. Y., “Pursuing stem careers: Perspectives of senior high school students,” Participatory Educational Research, vol. 7, no. 3, pp. 38–58, 2020. [Google Scholar]
  • 24.Gilissen J. et al. , “How to achieve the desired outcomes of advance care planning in nursing homes: a theory of change,” BMC geriatrics, vol. 18, no. 1, pp. 1–14, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Okoli C. and Schabram K., “A guide to conducting a systematic literature review of information systems research,” 2010. [Google Scholar]
  • 26.Xiao Y. and Watson M., “Guidance on conducting a systematic literature review,” Journal of Planning Education and Research, vol. 39, no. 1, pp. 93–112, 2019. [Google Scholar]
  • 27.N. A. of Engineering and National Academies of Sciences, Engineering, and Medicine, Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students' Diverse Pathways. Washington, DC: The National Academies Press, 2016. doi: 10.17226/21739 [DOI] [PubMed] [Google Scholar]
  • 28.Salehi S., Ballen C. J., Trujillo G., and Wieman C., “Inclusive Instructional Practices: Course Design, Implementation, and Discourse,” Frontiers in Education, vol. 6, p. 395, 2021, doi: 10.3389/feduc.2021.602639 [DOI] [Google Scholar]
  • 29.Casper A. M., Eddy S. L., and Freeman S., “True grit: Passion and persistence make an innovative course design work,” PLoS biology, vol. 17, no. 7, p. e3000359, 2019. doi: 10.1371/journal.pbio.3000359 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Ballen C. J., Wieman C., Salehi S., Searle J. B., and Zamudio K. R., “Enhancing Diversity in Undergraduate Science: Self-Efficacy Drives Performance Gains with Active Learning,” LSE, vol. 16, no. 4, p. ar56, Dec. 2017, doi: 10.1187/cbe.16-12-0344 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Theobald E. J. et al. , “Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math,” Proceedings of the National Academy of Sciences, vol. 117, no. 12, pp. 6476–6483, 2020. doi: 10.1073/pnas.1916903117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Thurmond V. B. and Cregler L. L., “Why students drop out of the pipeline to health professions careers: a follow-up of gifted minority high school students.,” Academic medicine: journal of the Association of American Medical Colleges, vol. 74, no. 4, pp. 448–451, 1999. doi: 10.1097/00001888-199904000-00044 [DOI] [PubMed] [Google Scholar]
  • 33.Crump C., Ned J., and Winkleby M. A., “The Stanford Medical youth science program: educational and science-related outcomes,” Advances in Health Sciences Education, vol. 20, no. 2, pp. 457–466, 2015. doi: 10.1007/s10459-014-9540-6 [DOI] [PubMed] [Google Scholar]
  • 34.Holden L., Berger W., Zingarelli R., and Siegel E., “After-School Program for urban youth: Evaluation of a health careers course in New York City high schools,” Information services & use, vol. 35, no. 1–2, pp. 141–160, 2015. doi: 10.3233/ISU-150773 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Katz J. R., Barbosa-Leiker C., and Benavides-Vaello S., “Measuring the Success of a Pipeline Program to Increase Nursing Workforce Diversity,” J Prof Nurs, vol. 32, no. 1, pp. 6–14, 2016, doi: 10.1016/j.profnurs.2015.05.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Habig B., Gupta P., Levine B., and Adams J., “An Informal Science Education Program’s Impact on STEM Major and STEM Career Outcomes,” Research in Science Education, vol. 50, no. 3, pp. 1051–1074, Jun. 2020, doi: 10.1007/s11165-018-9722-y [DOI] [Google Scholar]
  • 37.Fernandez-Repollet E., Locatis C., De Jesus-Monge W. E., Maisiak R., and Liu W.-L., “Effects of summer internship and follow-up distance mentoring programs on middle and high school student perceptions and interest in health careers,” BMC medical education, vol. 18, no. 1, pp. 1–6, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Wallace E. W., Perry J. C., Ferguson R. L., and Jackson D. K., “The Careers in Health and Medical Professions Program (CHAMPS): An Impact Study of a University-Based STEM+H Outreach Program,” Journal of Science Education and Technology, vol. 24, no. 4, pp. 484–495, Aug. 2015, doi: 10.1007/s10956-014-9536-2 [DOI] [Google Scholar]
  • 39.Winkleby M. A., “The Stanford Medical Youth Science Program: 18 years of a biomedical program for low-income high school students,” Academic Medicine, vol. 82, no. 2, pp. 139–145, 2007. doi: 10.1097/ACM.0b013e31802d8de6 [DOI] [PubMed] [Google Scholar]
  • 40.Winkleby M. A., Ned J., Ahn D., Koehler A., Fagliano K., and Crump C., “A controlled evaluation of a high school biomedical pipeline program: Design and methods,” Journal of science education and technology, vol. 23, no. 1, pp. 138–144, 2014. doi: 10.1007/s10956-013-9458-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Winkleby M. A., Ned J., Ahn D., Koehler A., and Kennedy J. D., “Increasing diversity in science and health professions: A 21-year longitudinal study documenting college and career success,” Journal of science education and technology, vol. 18, no. 6, pp. 535–545, 2009. [Google Scholar]
  • 42.Butler W. T., Thomson W. A., Morrissey C. T., Miller L. M., and Smith Q. W., “Baylor’s program to attract minority students and others to science and medicine.,” Academic medicine: journal of the Association of American Medical Colleges, vol. 66, no. 6, pp. 305–311, 1991. doi: 10.1097/00001888-199106000-00001 [DOI] [PubMed] [Google Scholar]
  • 43.Jones F. and Flowers J. C., “New York’s statewide approach to increase the number of minority applicants to medical school.,” Academic medicine: journal of the Association of American Medical Colleges, vol. 65, no. 11, pp. 671–674, 1990. doi: 10.1097/00001888-199011000-00002 [DOI] [PubMed] [Google Scholar]
  • 44.Slater M. and Iler E., “A program to prepare minority students for careers in medicine, science, and other high-level professions,” Academic Medicine, vol. 66, no. 4, 1991, [Online]. Available: https://journals.lww.com/academicmedicine/Fulltext/1991/04000/A_program_to_prepare_minority_students_for_careers.11.aspx doi: 10.1097/00001888-199104000-00011 [DOI] [PubMed] [Google Scholar]
  • 45.MCKENDALL S. B., SIMOYI P., CHESTER A. L., and RYE J. A., “The Health Sciences and Technology Academy: Utilizing pre-college enrichment programming to minimize post-secondary education barriers for underserved youth,” Academic Medicine, vol. 75, no. 10, pp. S121–S123, 2000. doi: 10.1097/00001888-200010001-00039 [DOI] [PubMed] [Google Scholar]
  • 46.Beck P., Githens J. H., Clinkscales D., Yamamoto D., Riley C. M., and Ward H. P., “Recruitment and retention program for minority and disadvantaged students.,” Journal of Medical Education, vol. 53, no. 8, pp. 651–657, 1978. doi: 10.1097/00001888-197808000-00004 [DOI] [PubMed] [Google Scholar]
  • 47.Gefter L., Spahr J., Gruber J., Ross S., Watson L., and Mann B., “Addressing Health Disparities with School-Based Outreach: the Health Career Academy Program,” Journal of Racial and Ethnic Health Disparities, vol. 5, no. 4, pp. 700–711, Aug. 2018, doi: 10.1007/s40615-017-0414-5 [DOI] [PubMed] [Google Scholar]
  • 48.Davis J. A. and Davidson C. P., “The Med-COR study: preparing high school students for health careers,” Academic Medicine, vol. 57, no. 7, 1982, [Online]. Available: https://journals.lww.com/academicmedicine/Fulltext/1982/07000/The_Med_COR_study__preparing_high_school_students.4.aspx [PubMed] [Google Scholar]
  • 49.Draugalis J. R. and Plaza C. M., “Best practices for survey research reports revisited: implications of target population, probability sampling, and response rate,” American journal of pharmaceutical education, vol. 73, no. 8, 2009. doi: 10.5688/aj7308142 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Bidwell S. S. et al. , “Development and Implementation of a Hands-on Surgical Pipeline Program for Low-Income High School Students,” JAMA Network Open, vol. 2, no. 8, pp. e199991–e199991, Aug. 2019, doi: 10.1001/jamanetworkopen.2019.9991 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Philips B. U., Mahan J. M., and Perry R. R., “Minority recruitment to the health professions: a matched comparison six-year follow-up.,” Journal of medical education, vol. 56, no. 9 Pt 1, pp. 742–747, 1981. [PubMed] [Google Scholar]
  • 52.Derck J., Zahn K., Finks J. F., Mand S., and Sandhu G., “Doctors of tomorrow: an innovative curriculum connecting underrepresented minority high school students to medical school,” Education for Health, vol. 29, no. 3, p. 259, 2016. doi: 10.4103/1357-6283.204219 [DOI] [PubMed] [Google Scholar]
  • 53.Enriquez A. G., Lipe C. B., and Price B., “Enhancing the success of minority STEM students by providing financial, academic, social, and cultural capital,” in 2014 ASEE Annual Conference & Exposition, 2014, p. 24.529. 1–24.529. 15. [Google Scholar]
  • 54.Graulich N., Lewis S. E., Kahveci A., Nyachwaya J. M., and Lawrie G. A., “Writing a review article: what to do with my literature review,” Chem. Educ. Res. Pract., vol. 22, no. 3, pp. 561–564, 2021, doi: 10.1039/D1RP90006D [DOI] [Google Scholar]
  • 55.Varpio L., Paradis E., Uijtdehaage S., and Young M., “The distinctions between theory, theoretical framework, and conceptual framework,” Academic Medicine, vol. 95, no. 7, pp. 989–994, 2020. doi: 10.1097/ACM.0000000000003075 [DOI] [PubMed] [Google Scholar]
  • 56.CohenMiller A. and Pate P. E., “A model for developing interdisciplinary research theoretical frameworks,” The Qualitative Report, vol. 24, no. 6, pp. 1211–1226, 2019. [Google Scholar]
  • 57.Julien C., “Bourdieu, social capital and online interaction,” Sociology, vol. 49, no. 2, pp. 356–373, 2015. [Google Scholar]
  • 58.Nardo J. E., Chapman N. C., Shi E. Y., Wieman C., and Salehi S., “Perspectives on Active Learning: Challenges for Equitable Active Learning Implementation,” J. Chem. Educ., Mar. 2022, doi: 10.1021/acs.jchemed.1c01233 [DOI] [Google Scholar]
  • 59.Ladson-Billings G., “From the Achievement Gap to the Education Debt: Understanding Achievement in US Schools,” Educational researcher, vol. 35, no. 7, p. 3, 2006. [Google Scholar]
  • 60.Onwuegbuzie A. J. and Johnson R. B., “The validity issue in mixed research,” Research in the Schools, vol. 13, no. 1, pp. 48–63, 2006. [Google Scholar]
  • 61.Mailu S. K., Adem A., Mbugua D. K., Gathuka P., and Mwogoi T., “Response rate, incentives and timing of online surveys: A study of agriculture researchers in Kenya,” Tanzania Journal of Agricultural Sciences, vol. 20, no. 1, pp. 82–93, 2021. [Google Scholar]
  • 62.Ardura D. and Prez-Bitrin A., “The effect of motivation on the choice of chemistry in secondary schools: adaptation and validation of the Science Motivation Questionnaire II to Spanish students,” Chemistry Education Research and Practice, vol. 19, no. 3, pp. 905–918, 2018. [Google Scholar]
  • 63.Leong E., Mercer A., Danczak S. M., Kyne S. H., and Thompson C. D., “The transition to first year chemistry: student, secondary and tertiary educator’s perceptions of student preparedness,” Chem. Educ. Res. Pract., vol. 22, no. 4, pp. 923–947, 2021, doi: 10.1039/D1RP00068C [DOI] [Google Scholar]

Decision Letter 0

Xiaodan Tang

Transfer Alert

This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present.

14 Sep 2022

PONE-D-22-13001A Systematic Review of the 60 Year Literature: Effects of Outreach Programs in Supporting Historically Marginalized and First-Generation, Low-Income Students in Healthcare EducationPLOS ONE

Dear Dr. Nardo,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

I agree with the two reviewers that your paper focuses on an important topic in the field. However, I also found valuable insights from reviewers about the issues of this paper including the language and organization of this paper and details about the study design. First, I think the organization of this paper is more like a report rather than a publishable manuscript. Please follow PLOS ONE submission guidelines. Also, please polish and proofread the language. Second, provide more details of the included papers for each evaluation criteria. For programs that have theoretical frameworks, what are their frameworks? For misaligned study goals, how are they misaligned? What about those having aligned study goals? In other words, please provide more details than descriptive summaries. Third, how and why the coding strategy was developed? Is it following any theoretical framework? 

Please submit your revised manuscript by Oct 29 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Xiaodan Tang

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following in your Competing Interests section: 

“NO”

Please complete your Competing Interests on the online submission form to state any Competing Interests. If you have no competing interests, please state "The authors have declared that no competing interests exist.", as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now

 This information should be included in your cover letter; we will change the online submission form on your behalf.

3. Please ensure that you refer to Figure 1-3 in your text as, if accepted, production will need this reference to link the reader to the figure.

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This manuscript addresses the important issue of high school outreach programs in healthcare education. The authors identify a number of issues related to the design, analysis, and evaluation of such studies that are currently hindering progress in understanding the efficacy of such programs.

My primary concern is with the written English of the manuscript. There are many places throughout the manuscript where the sentence structure, subject-verb agreement and use of prepositions confuse the meaning of the sentences. Overall, this has a substantial impact on the points the authors wish to communicate.

Some examples:

Page 1, "non-historically marginalized" should be historically non-marginalized

Page 3, "If studies were funding/declared interest"

Page 15, "Without a theoretical driven hypothesis, first, outreach programs...."

Page 16 + 17 there are several sentences which should be carefully reviewed.

There are some additional points that I think need to be addressed which I will put in chronological order.

1. The authors do not adequately draw on the literature to establish the link between underrepresentation in healthcare and gaps in quality care outcomes for marginalized groups. I think more needs to be done to explain this.

2. The UMMC study should be described in greater detail because the primary purpose of the manuscript is to critically evaluate the existing literature. The reader needs to be convinced that the results of this study are believable given all of the shortcomings that are identified in the body of the manuscript.

3. Table 1: Study Risk of Bias Assessment: the explanation of how this was done is not at all clear to me. I cannot distinguish between this explanation and the explanation for data items, quality, and eligibility assessment.

4. (Major) I think more needs to be said about the fact that "lack of study results" was a criterion for excluding articles from analysis. Do you mean quantitative results? I think it is an important oversight that you potentially missed qualitative results in a number of the articles that the search returned. There are only 120 total articles, so I think some review should be done to ensure that you did not miss qualitative results in the excluded articles.

5. Results section, first sentence. I think you need to justify this claim with data or literature.

6. (Minor) Some figures are reported to three decimal places and others to two. Please be consistent.

7. You never explain the concept of social capital, but draw on it several times in the manuscript to justify claims. Please provide some context.

8. Page 16 "Trends in the performance outcomes" -- if these are gaps in outcomes, please say so. It is not clear form this sentence that the outcomes are undesirable.

9. Page 18: what does it mean for qualitative results to "challenges the power" required for quantitative analyses?

Reviewer #2: -Summary. The manuscript involved synthesizing extant literature related to outreach efforts targeted at supporting historically marginalized groups in pursuing healthcare fields. Overall, the systematic review was organized and well-written, with a major strength of the paper being the transparency related to clearly outlining how the data were collected. The primary suggestions for improvement are related to the discussion of data analysis and the reporting of the findings. I have some additional comments and suggestions below.

-Abstract. This provides a nice summary of the themes, previewing the organization of the paper. A minor suggestion would be to include the final sample of papers in the abstract.

-Introduction. The introduction demonstrates a strong grounding in prior literature and familiarity with current trends, including moving away from the pipeline metaphor to addressing how we can support students (as opposed to simply sending more people through the pipeline). Although there is variation in systematic reviews regarding their inclusion, a guiding research question (or questions) could be helpful to more clearly articulate the focus of the review. Based on the analysis and framing, it is clear the authors have a specific goal with reviewing the literature (i.e., evaluating overall quality of research design), which could be communicated through research question(s).

-Methods. I appreciated the transparency related to how the papers were selected and collected, as well as the table outlining how this paper meets the criteria for a systematic review paper (including the PRIMSA checklist in the supplemental). One area for improvement would be the ways in which the data analysis is discussed. Data analysis is briefly discussed in the paper, with a supplemental document providing more insight into the nature of the analysis (this supplemental paper was very helpful). After reading the paper and the supplemental, it is not clear what the coding process looked like. Would you consider each acronym in the supplemental to be a code (e.g., NO, ASP, UBP, HSC, etc.) or were the codes larger in grain size (bullet points under the criteria in Figure 4)? There are also other themes that might suggest additional codes (e.g., Figure 5 – “specifically detailed”, “not specifically detailed” – but this wording is different from the wording in the supplemental under “Theoretical Underpinning Key”, so it is unclear if they are distinct or if they map onto one another; other examples would be the other descriptive trends in the other figures like Figure 9, etc.). Also, was it one author coding or multiple authors coding and did they calculate any measures related to reliability for the code assignments?

-Findings. Broadly, the Findings are organized and have a logical flow, reflecting the previously discussed emphasis of design, evaluation, analysis. The authors mention the paper is descriptive, which is not necessarily an issue; this is common among literature reviews because you are compiling a list that summarizes the research. However, I have some suggestions about the presentation of the descriptive findings. Due to the small sample size, I would suggest moving away from quantifying the findings, particularly, reconsider reporting percentages. The percentages can be misleading because they distance the findings from the context. This is particularly the case when there is discussion about percentages involving a subset of the sample (e.g., “Of the 54.2% of studies with detailed theoretical frameworks, 54.2% had study goals that aligned with the framework”). For transparency and clarity, provide raw values instead of percentages (“n=20 of the studies had a detailed framework, with only n=11 of these studies having goals that aligned with the framework”). This suggestion applies across the findings, including reporting values in figures, tables, etc.

-Conclusion. The claims made and conclusions are appropriate and thoughtful and provide great suggestions for future work in this area. The suggestions would improve future work and afford more substantiative claims with more robust study designs and theoretical grounding.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Jon-Marc G. Rodriguez

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2022 Dec 1;17(12):e0278453. doi: 10.1371/journal.pone.0278453.r002

Author response to Decision Letter 0

24 Oct 2022

Dear editor and reviewers,

We greatly appreciate the careful consideration taken to improve the presentation and content of the manuscript. We believe the editor and reviewers’ comments were essential to strengthen the overall presentation and scholarship of the work.

Reviewer 1 (see uploaded letter for full table of responses)

My primary concern is with the written English of the manuscript. There are many places throughout the manuscript where the sentence structure, subject-verb agreement and use of prepositions confuse the meaning of the sentences. Overall, this has a substantial impact on the points the authors wish to communicate.

-->We agree and sincerely apologize for the written English of the manuscript upon submission. We have significantly re-written every section for clarity. We have also consistently used the same verbiage (e.g., studies on healthcare education outreach programs) when explaining the same phenomenon in the text.

The authors do not adequately draw on the literature to establish the link between underrepresentation in healthcare and gaps in quality care outcomes for marginalized groups. I think more needs to be done to explain this.

-->We agree and the change has been made in the main document (Page 2):

The UMMC study should be described in greater detail because the primary purpose of the manuscript is to critically evaluate the existing literature. The reader needs to be convinced that the results of this study are believable given all of the shortcomings that are identified in the body of the manuscript.

--->“The study at UMMC drew from the Mississippi Science Framework that theorized program components such as student research mentorship, teacher professional development, disciplinary content knowledge enrichment, and practical experiences, which has promoted sustainability of the program for 28 years(22).”

Study Risk of Bias Assessment: the explanation of how this was done is not at all clear to me. I cannot distinguish between this explanation and the explanation for data items, quality, and eligibility assessment.

-->We agree and the change has been made in the main document (Page 4). We also cite (Page 16) how the review of bias by seeing if studies had reported any conflicts of interest is a limitation of the work because it is not an encompassing approach:

“After the abstracts were reviewed, the first author coded the studies for inclusion and all three authors worked collaboratively to verify the inclusion of each study for review. We examined the articles for their quality of 1) design, 2) evaluation, and 3) analysis and documented features of each article in tables (see supplemental information). To examine bias within the articles reviewed, we documented if authors declared any conflicts of interest.”

I think more needs to be said about the fact that "lack of study results" was a criterion for excluding articles from analysis. Do you mean quantitative results? I think it is an important oversight that you potentially missed qualitative results in a number of the articles that the search returned. There are only 120 total articles, so I think some review should be done to ensure that you did not miss qualitative results in the excluded articles.

-->We agree and for clarity, the studies needed to be classified as research studies by the journal (e.g., not editorials or tutorial papers) with defined methods, data, and findings sections. Studies needed to define the data collected and how the data was analyzed rather than a summary of what the components of the healthcare education outreach program did, which was the case for the papers excluded. We also cite this is a limitation of the work (Page 16). The change has been made in the main document (Page 4-5):

“Our second search returned 55 relevant articles from the journals of Academic Medicine, Medical Education, Medical Teacher, and BMC Medical Education of which 18 were excluded due not being classified by the journal as a research article containing reported findings with qualitative, quantitative, or mixed qualitative and qualitative data (see Fig 1). To be eligible for review, the articles needed to be research studies, which have explicit methods, data, and findings sections.”

You never explain the concept of social capital, but draw on it several times in the manuscript to justify claims. Please provide some context.

-->We agree and the change has been made in the main document (Page 17):

“Mentorship often provides historically marginalized students with social capital, which is a sociological term from Pierre Bourdieu that shows how access to knowledges, resources, and experiences can better facilitate social navigation(57).”

"Trends in the performance outcomes" -- if these are gaps in outcomes, please say so. It is not clear form this sentence that the outcomes are undesirable.

-->We agree and have removed the phrasing from the main document.

what does it mean for qualitative results to "challenges the power" required for quantitative analyses?

-->We agree and have removed the phrasing from the main document.

Reviewer 2 (see uploaded letter for full table of responses)

Although there is variation in systematic reviews regarding their inclusion, a guiding research question (or questions) could be helpful to more clearly articulate the focus of the review. Based on the analysis and framing, it is clear the authors have a specific goal with reviewing the literature (i.e., evaluating overall quality of research design), which could be communicated through research question(s).

-->We agree and the change has been made in the main document (Page 3):

Our systematic literature review seeks to answer:

How are studies on healthcare education outreach programs designed by drawing from theory?

How are studies on healthcare education outreach programs evaluated using quantitative, qualitative, and mixed-methods?

How are studies on healthcare education outreach programs analyzed to generate reliable and generalizable results?

One area for improvement would be the ways in which the data analysis is discussed. Data analysis is briefly discussed in the paper, with a supplemental document providing more insight into the nature of the analysis (this supplemental paper was very helpful). After reading the paper and the supplemental, it is not clear what the coding process looked like. Would you consider each acronym in the supplemental to be a code (e.g., NO, ASP, UBP, HSC, etc.) or were the codes larger in grain size (bullet points under the criteria in Figure 4)? There are also other themes that might suggest additional codes (e.g., Figure 5 – “specifically detailed”, “not specifically detailed” – but this wording is different from the wording in the supplemental under “Theoretical Underpinning Key”, so it is unclear if they are distinct or if they map onto one another; other examples would be the other descriptive trends in the other figures like Figure 9, etc.).

-->We agree and for clarity, we do consider each acronym to be a sub-code and they map onto the larger scheme for design, evaluation, and analysis. For the main text, we will only include the larger code category and how it aligns with the design, evaluation, and analysis to streamline the paper. We also changed the figure heading to match better what is in the Supplemental and the paper. The changes have been made in Figure 4 where we have the research question, coding features, and adequate quality criteria.

Also, was it one author coding or multiple authors coding and did they calculate any measures related to reliability for the code assignments?

-->We agree and for clarity, the first author listed coded through the literature and the second and third author listed verified the codes with the first author. Disagreements were discussed together until reaching consensus. The change has been made in the main document (Page 3-5) in Table 1 and Study Search, Retrieval, Eligibility, and Selection.

Due to the small sample size, I would suggest moving away from quantifying the findings, particularly, reconsider reporting percentages. The percentages can be misleading because they distance the findings from the context. This is particularly the case when there is discussion about percentages involving a subset of the sample (e.g., “Of the 54.2% of studies with detailed theoretical frameworks, 54.2% had study goals that aligned with the framework”). For transparency and clarity, provide raw values instead of percentages (“n=20 of the studies had a detailed framework, with only n=11 of these studies having goals that aligned with the framework”). This suggestion applies across the findings, including reporting values in figures, tables, etc.

-->We agree and have also decided to include the number of articles in each of the titles so that it is transparent for the reader to calculate.

Attachment

Submitted filename: Response Letter(1).docx

Click here for additional data file. (29KB, docx)

Decision Letter 1

Nabeel Al-Yateem

17 Nov 2022

A Systematic Review of the 60 Year Literature: Effects of Outreach Programs in Supporting Historically Marginalized and First-Generation, Low-Income Students in Healthcare Education

PONE-D-22-13001R1

Dear Dr. Nardo,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Nabeel Al-Yateem, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for addressing my concerns. I believe that the revised manuscript is much stronger and more clearly written.

Reviewer #2: The authors present a strong contribution to the literature. Comments I raised such as having research questions and modifying the framing of the data analysis/findings were addressed. I especially liked how the research questions mapped onto the three qualities (design, evaluation, analysis) which was also reflected in the discussion of the coding/analysis.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

Acceptance letter

Nabeel Al-Yateem

21 Nov 2022

PONE-D-22-13001R1

A Systematic Review of the 60 Year Literature: Effects of Outreach Programs in Supporting Historically Marginalized and First-Generation, Low-Income Students in Healthcare Education

Dear Dr. Nardo:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Nabeel Al-Yateem

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Checklist

    (DOCX)

    Click here for additional data file. (20.5KB, docx)
    S1 File

    S1A Table: Barriers, assumptions, components, evaluative methods, conclusions and evidence-based success. S1B Table: Recommendations, reasoning, and conclusions of non-program high school healthcare education pipeline research.

    (DOCX)

    Click here for additional data file. (443.8KB, docx)
    Attachment

    Submitted filename: Response Letter(1).docx

    Click here for additional data file. (29KB, docx)

    Data Availability Statement

    All relevant data are within the paper and its Supporting Information files.

    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES