The development and validation of an academic, work health sciences, and community intentions scale for out-of-school-time (OST) participants

Abstract

This study applies Exploratory Factor Analysis (EFA) to examine the internal structure and reliability of an academic, work, and community intentions scale for a cohort of out-of-school-time (OST) academic/STEM enrichment program participants (N = 533). This study utilizes the SPSS and SAS statistical software packages for comparative analysis. Both provide evidence of a three-factor model for intentions–Academic, Work/Health Science, and Community (i.e., the AWHSCI). The ordinal Cronbach’s alpha reliability coefficients were excellent or good. Non-parametric tools were employed to determine differences in participants’ academic, work/health science, and community intentions by race and gender.

1. Introduction

In the 1960s, the United States (US) implemented federal regulation supporting out-of-school-time (OST) academic enrichment programs to serve underprivileged and at-risk youth. Known as the TRIO programs, the purpose was to provide a structured environment for disadvantaged (e.g., racial, socio-economic, etc.) students to progress through the academic pipeline (U.S. Department of Education, 2020). Through the years, various organizations have developed OST programs for underprivileged youth emphasizing a health sciences/STEM-based curriculum (Rivers et al., 2020; Verizon, 2018; Verizon, 2014; Wilson, Kauffman, & Purdy, 2011; Le, Mariano, & Faxon-Mills, 2015; McKendall et al., 2014; Carter, Mandell, & Maton, 2009; Winkleby, 2007; Fincher et al., 2002; McKendall et al., 2000; Soto-Greene et al., 1999; Carline et al., 1998; Huttlinger & Drevdahl, 1994). Many of these programs target underserved students; however, the Health Sciences and Technology Academy (HSTA) is a unique partnership between West Virginia University (WVU) and rural Appalachian communities. This partnership has a grassroots philosophy of “home-growing” and educating students with the intent of creating a health sciences and STEM workforce in the local communities.

Developed in the early 90’s, this partnership was in response to the Association of American Medical Colleges initiated Project 3000 by 2000. The goal was to increase the numbers of under-represented minority (URM), specifically students of color entering medical schools (Tung, 2021). At the time, West Virginia’s (WV) population was 96% Caucasian; nevertheless, the WVU School of Medicine responded in 1994 by creating and implementing HSTA, a community-based organization (CBO) and OST program (US Department of Commerce, 1992). Since there were not enough medical practitioners to serve WV, WVU also renewed the endeavor to address the state’s health care/education crisis—a large medically underserved population and an undereducated workforce (Digest of Education Statistics, 1998; Thomson et al., 2003; Hammond & Hammond, 2003). This was important since research has shown that medical practitioners from rural settings are more likely to practice in those areas (Rosenblatt & Hart, 2000).

An important historical moment for CBO and higher education partnerships in the US was the 1997 Presidents’ Summit for America’s Future (National Archives and Records Administration, 1997). America’s leaders examined the landscape and surmised that the Nation faced serious social problems threatening “our children’s lives and our future” (America’s Promise, 2004). The Summit produced five key promises to the Nation’s youth: “a caring adult role model, a safe place to learn and grow; a healthy start and future; an opportunity to learn marketable skills through education; and opportunities to give back through community service” (State of West Virginia, 2022). Today, CBO and university partnerships exist to fulfill one or all the aforementioned promises for underserved and at-risk youth (Hanglund et al, 2021; April, et al., 2020; Coles, 2012; Democracy Collaborative, 2012; Rabinowitz et al., 2021; Schwartz and Pisacreta, 2020; Leiderman, et al., 2001). In short, higher education institutions and communities have an equal investment in the success of students who live in their communities. Both are stakeholders involved in building and sustaining educational and social program(s) serving youth in their community (Pathways to College Network et al., 2021).

1.1. Current outcomes of HSTA participants

HSTA participants are African American, first generation to college, financially disadvantaged and rural high school students. They have consistently shown there is a correlation between intentions and outcomes relative to pursuing college and health sciences degrees. They have also shown an intent to work in their local communities after graduating college (Chester et al., 2020; McKendall et al., 2014, 2000). Over the course of two decades, HSTA has graduated over 3000 students with 99% attending college and 87% earning at least a four-year degree. Of the 1654 successfully tracked HSTA graduates with post-secondary degrees, 48.7% and 11.5% have attained Health Sciences and STEM degrees, respectively. Of the program graduates, 85% of non-health professional degree earners and 86% of health professional degree earners live and work in WV.

1.2. Predicting academic and community relations intentions

As a theoretical construct, The Theory of Planned Behavior (TPB) examines the predicative nature or intention of individuals to engage in a specific behavior(s) at a specific time (Ajzen, 1991; Cooper, Barkatsas, & Strathdee, 2016). The measurement of intentions must coincide with the exact circumstances in which the behavioral outcome is to occur (Ajzen & Fishbein, 1977; Ajzen, 1988). HSTA fosters connections between participants and their local communities through community service projects focused on health and wellness (McKendall et al., 2014). Within this context, measuring participants’ behavioral patterns relative to academic and community intentions organically occurs. TPB seeks to explain and predict behavior patterns based on three beliefs: behavioral, normative and control. Behavioral beliefs produce either a favorable or an unfavorable attitude towards the behavior considering possible positive and negative outcomes. Normative beliefs are subjective and result in perceptions of social/peer pressure in performing the behavior. Control beliefs are behavioral in nature and results in the ability to control the outcome/performance of the behavior (Health Communication Capacity Collaborative, 2017; David et al., 2020). The proposed scale seeks to assess if HSTA students intend to perform the following behaviors: complete the HSTA program, go to college, pursue a health care degree and career as well as make meaningful contributions to their community as students and adults.

Academic Intentions.

Academic intentions are the commitment to pursue or complete an educational goal/endeavor immediately or in the future. Pursuit of post-secondary studies is foundational to the HSTA model, and participants are actively encouraged to pursue this goal (Brigandi et al., 2020; McKendall et al., 2014). Academic intentions or post-secondary pursuits of STEM areas has been examined in several studies (Chittum et al., 2017; Ho & Sum, 2016; Katz et al., 2016; Kitchen et al., 2018; Hurtado et al., 2008; Hurtado et al., 2010; Meyer, 1970; Salto et al., 2014; Shah et al., 2018). Chittum et al. (2017) found that when middle school students participated in an afterschool stem program (Studio STEM), there was an increase in motivational beliefs about science and intentions to pursue college as well as engagement in problem-based science and engineering activities. Kitchen et al. (2018) examined college aspirations of high school students participating in STEM summer programs compared to a control group showing a relationship between participation and increased interest in STEM careers. Salto et al. (2014) found that over 60% of high school students who participated in a STEM summer experience graduated from college with a STEM degree resulting in increased diversity in the biomedical workforce. Furthermore, Katz, Barbosa-Leiker, and Benavides-Vaello (2016) found that summer research experiences could engage under-represented high school students’ interest in nursing as a profession further validating the need for OST programs.

Community Intentions.

Community intentions is allegiance to and commitment to improving the educational, social and/or economic structures of a community (i.e., family, local, state, country), which can be accomplished through building social awareness (Hall & Panerese, 2016; Weissberg et al., 2015). Connecting students to their communities occurs through increasing social awareness and promoting civic activities such as volunteering at a local food pantry or interacting with pre-school children to raise awareness on the importance of a healthy diet and physical activity (Chester et al., 2020). A primary goal of HSTA is to promote adolescent, adult, and community health and well-being in West Virginia. Accordingly, HSTA builds community intentions through the process of citizenship via community based participatory research engendering student connectedness to their communities (Berge et al., 2009; Smith et al., 2018; Hamilton et al., 2017). HSTA students may “engage in simple experiments such as using calipers to measure body fat, or visit a cadaver lab to observe how obesity, black lung disease and cancer affect human organs” (Elza, 2019; Goodrich, 2017). Building a sense of community through required community service and inspiring HSTA participants to help others is a key program strategy (Chester et al., 2020).

2. Methods

2.1. Survey development

HSTA developed the Academic, Work/Health Science, and Community Intentions scale (AWHSCI) in 2013 and 2014, along with other psychometric scales (Fig. 1). During this process, HSTA also developed a detailed program logic model highlighting the psychological and social growth HSTA attempts to promote in young people (McKendall et al., 2014). The programs’ leaders and stakeholders believed it was important to know participants’ educational intentions, career aspirations and their intentions to serve their home communities as they progressed through and exited this OST/CBO program.

Fig. 1.

Development of AWHSCI Psychometric Scale.

2.2. Procedure

Each year, HSTA participants complete on-line surveys of their experience while in the program. Data collection and management occurs via the Research Electronic Data Capture (REDCap) system hosted at West Virginia University (Harris et al., 2008; Harris et al., 2019). REDCap is a secure, web-based software platform designed to support data capture for research studies. This study uses a data set from the 2014 HSTA participants to perform exploratory factor analysis on the AWHSCI scale. In total, 567 students completed the survey. Of these 538 (95%) had complete data with no missing variables on the scale. One student had missing demographic data. The data was pre-screened, and multivariate outliers were deleted using Mahalanobis distance (McLachlan, 1999). Thus, 5 cases with Mahalanobis distances greater than χ2 at p < .00 were eliminated from the data set for a final N of 533. The procedures provided a ratio of 41 cases per variable (Neill, n.d). Demographics for study participants are 32% males and 68% females. Racial/ethnic categories were 26% African American, 68% White, and 6% Other. Approximately 54% identified as first-generation college goers, 48% as receiving free or reduced priced lunch, and 62% as rural.

We examined the variables for normality by exploring the descriptive statistics (See Table 1). Both the Shapiro-Wilk (p < .00) and Kolmogorov-Smirnov (p = .00 or p < .00) tests for normality showed overwhelming evidence that all items/variables have non-normal distributions. Examining the boxplots and Q-Q plots for each variable also showed extreme deviations from the established statistical constructs of normality. The scatterplots between the variables also exhibited non-monotonic relationships, which denotes that a change in one variable is not associated with a change in a positive or negative direction with another variable (Lewis-Beck et al., 2004). Since the data are non-normal and have non-monotonic relationships, we calculated the Hoeffding’s (D) measure of dependence, which is a non-parametric alternative to Pearson and Spearman correlations when there is violation of normality and monotonic relationships between variables exist (Wicklin, 2021). The test range is between − 0.5 and 1, with only large positive values indicating dependence. The D statistic values for our data are between .00 and .50. The Hoeffding D matrix differs from the Pearson correlation in that the diagonal elements, which show association between a variable and itself may not be 1 (SAS® Help Center, n.d.; SAS Institute Inc, 1999). We noticed that Intent 10 had a 0 loading for Intents 5 and 6, indicating independence. Furthermore, none of the Ds were very high, so the dependence of the variables is minimal, further indicating non-monotonic relationships. We attempted to correct the normalcy error by performing Lg10 transformations to either reduce or remove the skewness from the original data; however, this proved futile only serving to increase skewness of the variables (e.g., Intent 1 decreased from −1.92 to −2.40; Intent 2 from −1.84 to −2.25; etc.). The difference in the skew statistics after the LG10 transformations ranged from −.12 to −1.02 decreases, leading to further deviations from normality. Given these results, we decided to utilize the untransformed data set since factor extraction, Maximum Likelihood (ML) nor the Principal Axis Factor (PAF) methods require normality of data in larger data sets (SAS Institute Inc., 2013; O’Rourke & Hatcher, 2013; Costello & Osborne, 2005).

Table 1.

Descriptive Statistics for the Items.

	Mean (SD)	Median	Std. Error	Skew	Kurt	Var.	95% CI for Mean
							Lower Bound	Upper Bound
Intent-1 Successfully completing the HSTA program.	4.68(.63)	5.00	.027	−1.84	2.41	.39	4.62	4.73
Intent-2 Earning all A’s and B’s in my classes next school year.	4.65(.67)	5.00	.029	−1.794	2.25	.45	4.59	4.70
Intent-3 Earning high enough grades to be accepted into college.	4.70(.65)	5.00	.026	−1.944	2.69	.03	4.65	4.76
Intent-4 Preparing for my college entrance exams (SAT or ACT).	4.59(.72)	5.00	.031	−2.59	6.88	.52	4.53	4.66
Intent-5 Graduating from high school.	4.75(.59)	5.00	.026	−2.59	6.88	.35	4.70	4.80
Intent-6 Enrolling in college.	4.73(.61)	5.00	.026	−2.29	4.60	.37	4.68	4.78
Intent-7 Majoring in health sciences or a health science-related subject.	4.30 (1.02)	5.00	.044	−1.509	1.73	1.04	4.21	4.39
Intent-8 Attending medical, nursing, pharmacy, dental, or public health graduate or professional programs.	4.13 (1.17)	5.00	.051	−1.30	.78	1.38	4.03	4.23
Intent-9 Having a career providing health services or as a health scientist.	4.12 (1.17)	5.00	.050	−1.20	.53	1.36	4.02	4.22
Intent-10 Living in West Virginia after completing college.	3.56 (1.42)	4.00	.062	−.53	−1.05	2.03	3.44	3.68
Intent-11 Helping to improve my community as a high school student.	3.99 (1.09)	4.00	.047	−.78	−.35	1.20	3.90	4.08
Intent-12 Helping to improve my community as a college student.	4.00 (1.08)	4.00	.047	−.82	−.20	1.18	3.91	4.09
Intent-13 Helping to improve my community as a working adult.	4.10 (1.05)	4.00	046	−1.03	.32	1.11	4.01	4.19

Note: Items are measured on 5-point Likert-type scale (i.e., 1 =Not committed at all; 2 =A little committed; 3 =Committed a medium; 4 =Committed a lot; 5 =Completely committed). The std. errors for Skew and Kurt are.11 and.21, respectively.

2.3. Analyses

Analyses was performed using the IBM SPSS^® Statistics (Version 28) and the SAS 9.4 TS Level 1M6 software packages to conduct the Maximum Likelihood (ML) and Principal Axis Factoring (PAF) methods for EFA (Osborne & Banjanovic, 2016). We used the promax rotation, which assumes correlation of factors, since the factor correlations matrix for the three extracted components is .368, .401 and .475, exceeding the .32 threshold (Tabachnick & Fiddell, 2007; Brown, 2009). The factorability of the 13 Intention items were examined using several conditions, which included correlations above .3, the Kaiser-Meyer-Olkin measure of sampling adequacy, the Bartlett’s test of Sphericity, and the communalities of .33 or above. During analyses, we suppressed small coefficients and absolute values below .40 were not selected. The results produced the Kaiser-Meyer-Olkin (KMO) test for sampling adequacy. KMO determines how well the data is suited for factor analysis measuring sampling adequacy for each variable in the model as well as the complete model and returns a value between 0 and 1 (Costello & Osborne, 2005; Spicer, 2004; Matsunaga, 2010; Beavers et al., 2013). In other words, this test measures the proportion of variance among the variables that might be common. If the test returns a value between 0.8 and 1, and a p-value that is less than .05, we can reject the null hypothesis and continue to perform EFA on the data. Another important consideration was to examine the commonalities among the items. Fabrigar and Wegener (2012) suggest that under optimal conditions, communalities of .70 or greater and having 3–5 measured variables is acceptable. Osborne, Costello, & Kellow (2008) note that commonalities of .40 are acceptable as well. Fabrigar et al. (1999) suggest that communalities of .40 to .70 are moderately good along with a sample size of at least 200.

3. Results

The results indicate that the data is more than adequate for performing EFA with the items, since the sample size of 533 and the Kaiser-Meyer-Olkin measure of sampling adequacy indicated a ‘marvelous’ common variance for factor analysis. The overall MSA is.88 (SPSS and SAS), which is higher than the .6 recommended value (Spicer, 2004). Bartlett’s test of Sphericity was also significant (χ2 (78) = 7446.74; p = 0.00). Another important indicator that factor analysis is suitable for all 13 items is that the measuring of sampling adequacy (MSA) for all items ranged from 0.79 to 0.96 which is greater than 0.50, further confirming that each item shared some common variance with other items (Neil, n.d.). Given these overall indicators, factor analysis is suitable with all 13 items. The prior communality estimates (PCEs) are large except for Intent-10 (i.e., 0.37 and 0.35) for SAS and SPSS, respectively. Although it is suggested that communalities above 0.40 are acceptable, Intent 10 was retained because it measures HSTA participants’ intentions to remain in WV after graduating college which is important for HSTA analysis (Osborne et al., 2008; Gaskin, 2021).

We were interested in knowing if the factor loadings would differ between the ML Method and the PAF Method, despite having non-normal data. Both SPSS and SAS retained three factors as shown in the Fig. 2. In SPSS, the PAF method retained three factors with a cumulative variance of 78.66%. The percent variances of the eigenvalues for the ML method are 78.59%. The SAS system produced 100% cumulative variances for both methods. The graph shows that after the third factor, there is an elbow, showing that each proceeding variable attributes less and less to the total variance (UCLA Institute for Digital Research & Education, n.d).

Fig. 2.

Scree Plot of SPSS and SAS Eigenvalues for PAF Method with Promax Rotation.

The rotated factor loadings for the 13-item survey are in Tables 2 and 3. Intents-1 through 6 cluster around Factor 1; Intents-7 through 9 around Factor 3; and Intents 10 through 13 falls on Factor 2. Based on the grouping of the intents, we obtained the following components: Academic Intent, Work/Health Science Intent, and Community Intent. As shown in Tables 2 and 3, both the programs produced very similar component groupings. Since neither of the rotated factor patterns load on more than one construct at greater than 0.40, we retained all variables. The Cronbach’s coefficient alphas were more than sufficient for the overall scale (α = 0.89). For the individual scales of Academic (6 items), Work/Health Science (3 items), and Community (3 items), the Cronbach’s coefficient alphas of 0.96, 0.94, and 0.81 are greater than Nunnally and Bernstein’s (1994) 0.70 suggested value. See the descriptive statistics in Table 4 for the individual scales.

Table 2.

Rotated Factor Loading Patterns for Principal Axis Factoring with Promax Rotation.

	SPSS			SAS
	Factor 1	Factor 2	Factor 3	Factor 1	Factor 2	Factor3
Intent1	.82			.82
Intent2	.84			.83
Intent3	.99			.98
Intent4	.79			.78
Intent5	.93			.92
Intent6	.96			.95
Intent7			.79			.78
Intent8			.91			.93
Intent9			.94			.93
Intent10		.54			.54
Intent11		.94			.93
Intent12		.98			.97
Intent13		.87			.86

Table 3.

Rotated Factor Loading Patterns for Maximum Likelihood with Promax Rotation.

	SPSS			SAS
	Factor 1	Factor 2	Factor 3	Factor 1	Factor 2	Factor3
Intent1	.82			.81
Intent2	.83			.83
Intent3	.98			.97
Intent4	.79			.78
Intent5	.95			.94
Intent6	.96			.95
Intent7			.79			.78
Intent8			.94			.93
Intent9			.94			.93
Intent10		.51			.51
Intent11		.93			.92
Intent12		.99			.99
Intent13		.88			.87

Table 4.

Descriptive Statistics for Individual Intentions Scales.

	Range	Mean (SD)	Median	Std. Error	Skew	Kurt.	Var.	95% CI for Mean
Intentions Scales								Lower Bound	Upper Bound
Academic	0–24	22.10(3.46)b	24.00	.15	−1.94	2.87	11.97	21.81	22.40
Work/Health Science	0–12	9.55(3.15)	11.00	.14	−1.30	1.05	9.92	9.28	9.82
Community	0–16	11.65(4.05)	12.00	.18	−.64	−.43	16.36	11.31	12.00

Note: All values are less than or equal to the median.

^bMedian Test cannot be performed.

3.1. Non-parametric analyses

We also performed several non-parametric analyses to determine if there were differences in student academic, work/health science and community intentions by race and gender. Levene’s test for homogeneity of variances revealed unequal variances for the academic intention scale by race and the academic and work/health intention scale by gender. Thus, we were unable to test the medians for differences on these scales by these variables (Ruxton, 2006). Since our data violates the assumption of homogeneity of variance, we used the Welch’s ANOVA to compare the effect of race on academic intentions as well as the Independent Samples Tests to determine differences across males and females.

The Welch’s ANOVA (the Robust Tests of Equality of Means) revealed that there was a statistically significant difference in academic intention F(2, 79.10) = 3.52, p = .03 and community intent F(2, 82.93) = 4.49, p = .01 between at least two groups. Since the Welch’s adjusted F ratio was significant, we can conclude that at least two of the three race groups differ significantly on their average academic and community intentions scores. The mean scores are 21.50, 22.40 and 21.33 for African American (N = 138), White (N = 362) and Other (N = 33) participants, respectively. We used the Games-Howell post hoc procedure to conduct pairwise comparisons finding that the mean difference between African American and White participants academic intentions score is − .893 (p = .05), 95% CI [− 1.79 − .00]. There were no significant differences found. We also examined Cliff’s δ effect size (Zaiontz, 2021), which is defined as

$$\delta =\frac{1}{mn}{\sum }_{i=1}^m{\sum }_{j=1}^n\delta (i,j)$$

We created a matrix in Microsoft Excel of the academic and community intentions variables comparing the African American and White participants. Afterwards, we utilized the following Microsoft Excel worksheet formula to obtain effect sizes of − 0.02 and − 0.05:

$$=\text{SUM}(\text{AA}3:\text{NX}502)/\text{COUNT}(\text{AA}3:\text{NX}502).$$

According to Zaiontz, Cliff’s δ is interpreted as − 1 ≤ δ ≤ 1 … with values near ± 1 indicating no overlap between the samples, whereas values near zero indicate a lot of overlap. The obtained effect size indicates high overlap in the responses for African American and White participants. We measured the strength of association (ω²) between race and the dependent variables with the following formula:

$$\omega 2=\frac{S{S}_B-(\text{K}-1)M{S}_W}{S{S}_T+M{S}_W}$$

When we input our data from the ANOVA table into the formula, we obtain the following output:

$$\text{Academic Intent}\omega 2=\frac{79.729-(2-1)11.525}{5819.142+11.525}=\frac{68.204}{5830.667}=.01$$

$$\text{Community Intent}\omega 2=\frac{152.558-(2-1)16.104}{8172.550+16.104}=\frac{136.454}{8188.654}=.02$$

This indicates that the independent variable (race) accounts for approximately 1.2% of the variance in the dependent variable (academic intention). For the community intentions scale, race accounted for 1.7% of the variance. As can be seen, the Cliff’s δ effect size and the strength of association are similar in the results of differences between African Americans and Whites in their academic and community intention outcomes.

The Independent-Samples Kruskal-Wallis tests conducted on differences by race indicate non-significant differences for work/health science intent, χ2(2) = .05, p = .97 by race (i.e., African American, White, and Other). For the community intent scale, significant differences were detected, χ2(2) = 9.063, p = .01. Pairwise comparisons with adjustments of significant values by the Bonferroni correction for multiple tests across race show that the difference is between African American and White participants, p = .01 with a Cliff’s δ effect size − of 0.05. The sample average ranks are 235.34, 254.00, and 280.25 for African American, Other, and White (N = 362), respectively. We cannot test for differences in the medians since the shapes and distributions of the data are different, thus we can only report the average ranks.

The Independent Samples Tests for unequal variances were performed to determine differences between males (N = 172) and females (N = 361) for academic and work/health intentions. The average academic intentions scale for females is 22.47 and males is 21.34 showing a significant difference (t_270.893 = −3.257, p < .00). For the work/health science intention scale, the females average is 9.92 compared to the males’ average of 8.77, (t₅₃₁ = −3.99, p < .00). Furthermore, the Independent-Samples Kruskal-Wallis test also showed significant difference between males and females on community intent, χ2(1) = 5.56, p = .02. Given the non-monotonic relationship between race/gender (independent variables) and the intentions scales (dependent variables) we were unable to use Spearman’s correlation to determine the strength and direction of the relationship between these variables. However, we calculated the Cliff’s δ effect sizes for gender and academic, health science/work and community intentions, which are − 0.15, − 0.13, and − 0.12, respectively.

4. Discussion

Reliable and valid measures to describe evaluation processes relative to OST/STEM participants’ academic, work, and community intentions are important (Peterman, Withy & Boulay, 2018; Byars-Winston et al., 2016; Griffith, 2012). This current study describes the development of the first known psychometric scale designed to measure academic/work/community intentions of participants in OST/STEM programs. Exploratory Factor Analysis served as the tool to reduce the survey items and identify the latent constructs, which produced good or excellent internal reliability for three scales: Academic (α = 0.96), Work/Health Science (α = 0.94) and Community intentions (α = 0.81). This study is the first to our knowledge to provide an understanding of participants’ intentions, providing greater insight into the effectiveness of OST programs.

This study also advances the understanding of CBO and university partnerships as well as offer OST/STEM program participants’ perspectives from a qualitative research framework. Prior research highlights the opportunities OST programs provide to enhance the educational pipeline and strengthen the bridge of pursuit and achievement of higher degrees for underrepresented youth (Bausmith & France, 2012; Myers et al., 2010; McKendall et al., 2000). Strengthening the academic pipeline for underrepresented students also strengthens their earning power, greatly reducing the economic and social barriers they historically encounter, so that these students can become a source of inspiration to others in their community (Chester & Dooley, 2011). However, there is little insight given to CBO and higher education partnerships. Are such partnerships successful? Are participants maneuvering through the education pipeline? Do these investments make a difference in the lives of unprivileged youth? Are participants contributing or intend to contribute to their communities? Considering these questions, the purpose of this study was to examine the latent constructs for the proposed scale, which seeks to answer such questions.

We acknowledge the limitation of this study. In preparing for the analysis process, we encountered non-normal data. We eliminated outliers using Mahalanobis distances and attempted to normalize the data by performing Log10 transformation. We were confident that given this study’s sample size (N = 533), we would be able to perform transformation and thus conduct parametric testing; however, this was to no avail and lead to increased skewness for the variables. Since all the variables “are skewed to about the same moderate extent,” we were unable to make even slight improvements, despite transformations having the ability to provide at least marginal improvements in analysis (Tabachnick and Fiddell, pg. 87). Tabachnick and Fidell (2013) also note that.

“…with large samples, the significance level of skewness is not as important as its actual size (worse the farther from zero) and the visual appearance of the distribution. In a large sample, the impact of departure from zero kurtosis also diminishes. For example, underestimates of variance associated with positive kurtosis. disappear with samples of 100 or more cases; with negative kurtosis, underestimation of variance appears with samples of 200 or more (Waternaux, 1976)” (pg. 80).

There are several reasons as to why data elements fail normality tests (Minitab, 2015). However, in the case of our data, we have extreme values with many participants selecting 5 and 4 on the scales. Since data collection occurs via web-based surveys, we have no concerns for data-entry errors. Close examination of the fully rotated scales reveal that the academic and work/health science intention scales are non-normal data; however, the Normal Q and Q plot for the community intention scale shows more linearity between the variables since there is more of a spread in the data. Since our data deviate from normality, we could not use parametric tools to analyze outcomes. Nevertheless, we were able to perform EFA on the proposed data since normality is not an assumption for factor analysis (Costello & Osborne, 2005). Preliminary data analysis pointing towards high skewness, non-monotonic relationships and independence of variables seems to defy the prospect of producing good results. Although prior research indicates that the ML method is sensitive to skewed data (Briggs & MacCallum, 2003), the factor model of the ML method is identical to the PAF from SAS and SPSS with very similar factor loadings across all variables. This is a very promising outcome for the AWHSCI scale validation.

The non-parametric tools provide an interesting picture of intentions as it relates to participants by race and gender. The results indicated that Whites have significantly higher academic intentions than African American participants as well as females having significantly higher intentions than males across all scales. We performed the Games-Howell post hoc procedure to determine the pairwise differences. Although the procedure indicates a significant difference between African American and White participants’ academic and community intentions, the difference is relatively small. In other words, participants’ race has a miniscule effect on their intentions to complete HSTA, earn A’s and B’s, enroll in college, and to make meaningful contributions to their communities.

It also appears that HSTA females present higher levels of academic, health science/work and community intentions than males; however, the Cliff’s δ effect sizes show the differences as miniscule. Sullivan and Feinn (2012) discuss the importance of reporting effect size as well as p-value in order to display overall magnitude of the results. The effect sizes demonstrate that the intention for females is not very different from males. Nevertheless, it is very promising that female participants are on par with males regarding intentions. Prior research show that women dominate in the health care work sector, yet they continue to experience workplace challenges, such as holding fewer higher-level administrative positions (ALobaid et al., 2020; Carnes et al., 2008). HSTA hopes that future female graduates will be able to break the glass ceiling to become leaders in academic medicine. Despite such findings, we believe that if we were able to utilize parametric measures, we might have discovered intricate findings; thus, we realize the drawbacks in not utilizing a “normal” data set. Nevertheless, this study has theoretical and practical implications since three-factor structure delineates academic, work health science and community constructs showing high reliabilities and adding to the body of knowledge to assess OST participants’ intentions relative to pursuit of post-secondary studies as well as gauging intentions to live in and work in their communities. This study also provides evidence that HSTA participants not only intend to, but also are educationally motivated to achieve their academic goals, showing the HSTA framework as a possible vehicle for increasing the recruitment of underrepresented populations into health care/STEM professions.

Future studies could do comparative analysis of HSTA to other OST/health science/STEM program participants on intentions and outcomes. Such a study could reveal relevant analyses as to if HSTA participants are more intentional in their pursuit of health care careers as well as to live and work in their local communities than a similar comparison group. Future studies may also investigate the application of these scales to students from different backgrounds (e.g., demographical, geographical, cultural).

5. Conclusions

As an OST program with a primary goal of “home growing” medical practitioners, more so than ever, HSTA has a renewed commitment to helping students stay their intended course of going to college and ultimately pursuing Health Care careers. Similar programs with a focus on health/STEM careers may be able to utilize these scales to determine participant trajectories as well. We believe this study provides viable scales to examine the academic, work and community intentions of participants in an OST/STEM enrichment program. Since EFA is a major type of factor analysis designed to describe and summarize data by grouping together correlated variables to formulate hypotheses about underlying processes, this study provides insight into the HSTA processes of increasing student intentions. The authors are not aware of any such research that addresses underrepresented OST/STEM participants’ intentions to pursue higher education in the health professional/STEM areas nor intentions to make meaningful contributions to their communities. Thus, the desired outcome of this study is use of the scale across various OST and CBO/university academic and social enrichment programs, which positively affect the academic and community awareness of student participants.

Biographies

Sherron Benson McKendall, PhD is the Senior Research Associate for the Health Sciences and Technology Academy (HSTA) at West Virginia University, USA.

Alan McKendall, PhD is an Associate Professor in the Department of Industrial and Management Systems Engineering, West Virginia University, USA.

Ann Chester, PhD is the Editor for the Journal of STEM Outreach and founder of the Health Sciences and Technology Academy (HSTA), West Virginia University, USA.

Data Availability

Since the survey respondents are part of a vulnerable population (children), the respondents were assured that raw data would remain confidential and not be shared. Data not available/The data that has been used is confidential.