TEACHING STATISTICS ONLINE: COMPARING COMPETENCY-BASED AND TRADITIONAL LEARNING

Abstract

Background:

Interdisciplinary undergraduate students are expected to develop statistical competence to interpret research findings as they advance in their healthcare studies and careers. However, students often report anxiety related to learning statistical course content.

This descriptive study examined differences in statistical knowledge and self-efficacy scores of undergraduate students enrolled in online competency-based and traditional learning statistics courses.

A sample of 20 nursing and interdisciplinary, undergraduate students from a Midwestern University who were enrolled in introductory statistics courses were recruited for this study.

Methods:

Significant differences between pre-test and post-test statistical knowledge and self-efficacy scores were compared for students receiving online competency-based learning and traditional learning statistical course content. The Current Statistics Self-Efficacy (CSSE) and the Self-Efficacy to Learn Statistics (SELS) measures determined the statistics self-efficacy scores.

Results:

There was a significant difference in knowledge scores from pre- to post-test in the areas of hypothesis testing (p =.02), measures of central tendency (p =.001), and research design (p =.001), but there was not a significant difference in overall mean scores between competency-based learning and the traditional learning groups (p =.10). The pre-test to post-test Current Statistics Self-Efficacy student scores improved significantly in both the competency-based learning (p <.001) and traditional learning (p <.001) statistics course sections; and the Self Efficacy to Learn Statistics pre-test to post-test scores were also significantly improved in the competency-based (p <.001) and traditional (p =.02) learning groups.

Conclusions:

Both online competency-based and traditional learning methods improved interdisciplinary undergraduate students’ statistical knowledge and self-efficacy scores. Online competency-based learning was described by the students as at least as beneficial as traditional learning for studying statistics while allowing more flexibility to repeat content until it was mastered.

Purpose:

This study compared pre- and post-test differences in statistical knowledge and self-efficacy scores of students enrolled in online competency-based learning and traditional learning statistics class sections.

Statistical skills are paramount to understanding and solving complex problems. Statistics are being used everywhere, from decision-making in the boardroom to estimating disease risks to the public. This has played out around the world in determining one’s risk of acquiring the coronavirus disease and its variants, but also in retail industries. For example, the airline industry uses statistical analysis to maximize their profits by determining passenger airline load factors and the percentage of seats on the plane to fill to capacity.

BACKGROUND

Statistics is a key course requirement for most curricula today, but students in health-related disciplines often report anxiety related to learning statistical course content. Confidence in one’s own abilities to solve problems through the use of statistics is referred to as having statistical self-efficacy (Finney & Schraw, 2003). Those who believe they possess the ability to succeed at academic tasks have been suggested to perform better than those with lower self-efficacy (Bandura, 1977). In addition, higher self-efficacy has been associated with increased motivation and the likelihood of succeeding (Mantooth et al, 2021). Many undergraduate students, especially at the associate degree level, enter their first statistics course with preconceptions about the topic and their capabilities in that domain. Self-efficacy may influence whether individuals choose to attempt a new challenge, how difficult it will be, how long they will work at the task, how resilient they will be if they struggle with the task, how anxiety-provoking they will experience the task to be, and whether they will ultimately accomplish the task. Higher statistical self-efficacy may have been associated with positive attitudes toward statistics courses (Walker & Brakke, 2017), though little published information has been found addressing the effectiveness of the self-efficacy of entry-level undergraduate students enrolled in statistics courses.

Competency-based learning results in significantly enhanced educational program quality and efficiency (Hurtubise & Roman, 2014), and is thought to promote success in a career because self-directed learning and active learning will lead to students being fully committed to lifelong learning and continuous professional development. The development of a course with self-paced modules to study statistics, such as those used in competency-based learning, could help promote learning and increase self-efficacy for students to study statistics. Competency-based learning has an individualized teaching approach that promotes proficiency by having the student complete defined tasks that allow the student to reach sufficient competence for unsupervised practice (Moran et al, 2023). Proficiency is promoted by feedback with an opportunity to re-learn areas that are performed incorrectly (Spencer, 2022). A study conducted with graduate students enrolled in an online conceptual geometry course demonstrated that competency-based learning was effective and showed an increase in student efficacy in course knowledge and skills (Lee & Pant, 2021). However, a paucity of information exists that describes the effectiveness of competency-based learning to increase the statistical knowledge of entry-level, undergraduate students in health-related disciplines such as nursing.

Therefore, the purpose of this study was to examine the effectiveness of using competency-based learning versus traditional learning to teach statistics to entry-level, undergraduate students in health-related disciplines. In addition, we compared the self-efficacy scores of students enrolled in competency-based learning and traditional learning online statistics courses.

METHODS

Research design

This pretest-posttest study was guided by Bandura’s Social Cognitive Theory (Bandura, 1977), which suggests that those who believe they possess the ability to succeed at academic tasks will perform better than those with lower self-efficacy (Bandura, 1977). Changes in students’ statistical knowledge and self-efficacy scores were measured following their participation in asynchronous competency-based learning and traditional learning statistics courses. This study took place in the Midwest at a college with entry-level, undergraduate students enrolled in two statistics course sections delivered asynchronously online. Online course delivery was used due to COVID-19 restrictions still in place during the 2021 and 2022 academic years.

Study sample.

Participants were recruited from a population of students enrolled in an introductory statistics course. They were recruited in their third semester of academic coursework during the first two weeks of classes for that semester. A full description of the study and the consent forms were presented to potential participants. Students’ questions about the study were answered by the researchers. Participants were informed that participation was voluntary and that there would be no prejudice should they refuse to participate. Students were eligible for participation in the study if they were enrolled in an introductory statistics course and spoke and read English. The study protocols were approved by the university’s Institutional Review Board. Students who consented to participate in the study agreed to comply with the prescribed study protocols. They were also assured that study information was confidential and reported as aggregate data.

Of the 30 students enrolled in the asynchronous statistics courses, 21 students consented to participate in the study. The participants in the study were from the following health-related disciplines: Nursing, Forensic Science, Kinesiology, Occupational Therapy, Social Work, Psychology, Public Health, and Nutrition. Of the 20 students who completed the study, eight students were enrolled in the asynchronous competency-based learning statistics course and 12 students were in the asynchronous traditional learning statistics course.

Evaluating statistical knowledge

Differences between the students’ pretest and posttest knowledge scores were analyzed using within-subjects t-test comparisons. The students’ statistical knowledge levels in both the competency-based and traditional learning courses were evaluated using multiple-choice questions following the completion of each of the following statistical content areas: Hypothesis Testing, Central Tendency, Research Methods, Statistical Significance, and Research Design. Following an introduction to design and probability in research, students were introduced to hypothesis testing, which uses the probability of a calculated statistic to either reject or not reject the null hypothesis (Tokunaga, 2018). Other statistical topics included measures of central tendency, which are indices derived from the center of the score distribution; correlational analysis to determine the association or bond between variables; and statistical significance, which indicates reported results from sample data that are unlikely to result from chance at a specified level of probability (Polit & Beck, 2021). Course topics were developed with a focus on the student’s conceptual understanding of analytical skills. Students took a quiz before and after completing each topic area to allow for an initial assessment and summative evaluation of the statistical course content covered. Both the competency-based learning course and the traditional learning course included hands-on data entry assignments and computational analysis techniques using a commonly available statistical package (SPSS).

Competency-based learning course

Statistical modules for the entry-level competency-based statistics course were designed by a professor who was experienced in teaching statistics to undergraduate students. The planned modules included a research design component, a quantitative component, an interpretation component, and an evaluation of a qualitative open-ended question. The modules were developed for students in health-related disciplines so they could understand why the application of the correct quantitative approach was important to the analysis of study data, how to analyze the data using a statistical program, and how to properly interpret data results. The modules were presented within the context of research problems related to interdisciplinary biomedical and biobehavioral issues. The modules included pretests and posttests with randomized questions, interactive vocabulary exercises, recorded lectures, and application assignments.

Traditional learning course

In the traditional learning course, the instructor met with the students using Zoom for each class period, which included opportunities for active learning. The students watched recorded lectures on their own time. Therefore, it was considered by the professor to be an asynchronous class with traditional recorded lectures. Course evaluations provided feedback on the design and instruction of the courses. An evaluation survey helped provide an empirical measure of the objectives met during the course offerings. The effectiveness of the traditional learning course content was determined by comparing the students’ pretest and posttest scores.

Study measures

Demographic questions included the participant’s age, race, educational goals, years of education, employment status, and biological sex at birth. The participants also reported their performance in prior statistical coursework and research experience.

Self-efficacy data were collected using the 14-item Current Statistics Self-Efficacy (CSSE) scale and the 14-item Self-Efficacy to Learn Statistics (SELS) scale from students at the beginning and end of both statistics courses. The statistics self-efficacy instruments were developed by Finney and Schraw (2003), both with 14 identical statements using a 6-point Likert-type scale for response options to measure the student’s confidence in studying statistics. Confidence levels were rated from 1 (no confidence at all) to 6 (complete confidence) regarding current statistical knowledge and self-efficacy in learning statistics. The surveys were administered to participants before and after completion of each of the statistics courses. The questions in the CSSE scale focused on current self-efficacy, whereas those on the SELS focused on student self-efficacy to learn statistics. The study instruments had been psychometrically tested by their original authors. All study protocols were pre-tested at the research site. These data were used to confirm the validity and reliability of the study measures. Finney and Schraw (2003) found the reliability for the CSSE was alpha = .91 for 138 undergraduate students attending a large Midwestern university. The same study found a reliability measure for the SELS of alpha = .98 for 140 undergraduate students (Finney & Schraw, 2003; Schunk, 1989, 1991).

Initial validity was assessed for the CSSE and SELS scales developed by Finney and Schraw (2003). An Exploratory Factor Analysis confirmed that all items met the inclusion criteria with the lowest pattern coefficient equaling .55 (n=130). Therefore, the Factor Analysis showed the model was a good fit for the phenomenon being measured. When testing the SELS instrument in a norm group of 154 college students enrolled in an introductory statistical methods course, the validity of SELS to other variables was also reported (Perepiczka, Chandler & Becerra, 2011). For example, the SELS positively correlated with a Math Self-Efficacy scale and negatively correlated with a general and statistics Test Anxiety Inventory subscale, therefore providing evidence of concurrent validity. The CSSE was also found to be a valid measure in assessing graduate students’ level of statistical self-efficacy (Lu, Estrada & Pulos, 2018).

Plan for data entry and analysis

Frequencies were recorded for each of the demographic variables. Data were entered into the SPSS program for analysis. Demographic data were analyzed using frequencies, means, and medians, and frequency distributions were calculated. Data analysis of the study variables included comparisons of the pretest and posttest statistical knowledge and statistical self-efficacy scores. Minimum significance levels were set at p ≤ .05. Statistical knowledge was evaluated so that pretest and posttest differences between the student’s quiz scores could determine their levels of learning in each of the statistical content areas. To evaluate changes within groups before and after the completion of each of the course content areas, the Wilcoxon sign rank test was used to analyze score differences in pretest to posttest scores from the SELS and CSSE scales. Mann Whitney U tests were used to compare self-efficacy scores for differences between the competency-based learning and traditional learning statistical course groups. Differences between the students’ pretest and posttest grouped mean knowledge scores for both courses were analyzed using independent t-test comparisons.

DATA/RESULTS

Demographic characteristics and frequencies describing the students enrolled in both courses are listed in Table 1.

Table 1.

Demographic Characteristics of Undergraduate Students Enrolled in Statistics Courses (N=20)

	Competency-based learning course (n = 8)		Traditional learning course (n = 12)
	n	%	n	%
Biological sex at birth
Female	7	87.5	11	91.7
Male	1	12.5	1	8.3
Ethnicity
White	5	62.5	10	83.3
Black†	1	12.5	1	8.3
American Indian††	0	0	1	8.3
Other†††	2	25.0	0	0
Current year of college
2nd year	-	-	8	66.7
3rd year	5	62.5	3	25.0
4th year	3	37.5	1	8.3
Student employment status
Employed full-time	0	0	1	8.3
Employed part-time	8	100.0	0	0
Student looking for work	0	0	3	25.0
Unemployed	0	0	2	16.7
Highest-level statistics content
No formal education	1	12.5	9	75.0
High school course	1	12.5	2	16.7
College course	6	75.0	1	8.3
			-	-

^†or African American,

^††or Alaska Native,

^†††or Mixed Race

The average age of the students was 22 years (range 17 to 36 years), and 80% of the participants were female. The ethnicity of the study sample included 15 Caucasian students, 2 Black students, 1 American Indian or Alaskan Native student, and 2 Mixed Race students.

Students’ statistical knowledge

Student scores in the competency-based learning group improved by an overall average mean difference of 18%, compared to 29% in the traditional learning group. However, this difference was not significantly different (p = .10). The overall mean difference scores from pretest to post-test for the competency-based and traditional learning groups were 18.07 vs. 29.22 (p > .10). The overall mean post-test score for the traditional learning group was 75.11 in comparison to the competency-based learning group, which was 88.21. The overall post-test mean scores were significantly different when comparing the competency-based and traditional learning groups (t = 8.12, df = 18, p< .0001) with a 95% CI [9.71, 16.49]. Individual statistical knowledge scores were compared for the following content areas of the two groups: Hypothesis testing, Central tendency, Correlations, Statistical significance, and Research design. These scores are listed in Table 2.

Table 2.

Competency-Based Scores vs. Traditional Learning Scores (n=20)

	Competency-based scores			Traditional learning scores
Statistical Content Areas	Pretest (%)	Posttest (%)	Difference (%)	Pretest (%)	Posttest (%)	Difference (%)	t	P
Hypothesis testing (control groups)	59.82	82.14	22.32	64.71	63.42	−1.29	3.79	.02*
Central tendency (basic statistics)	66.67	87.50	20.83	43.14	90.25	47.11	−7.08	.001***
Correlations (research methods)	69.44	78.57	9.13	50.99	73.18	22.19	−1.67	.14
Statistical significance	83.33	86.11	2.78	30.93	83.55	52.62	−0.88	.41
Research design (Ind/Dep variables)	80.00	87.50	7.50	74.51	100.00	25.49	−7.07	.001***

Note: Independent t-tests were used to analyze differences between the competency-based group mean test scores and the traditional learning group mean scores with a significance level of .025 following a Bonferroni correction for a small number of multiple comparisons.

Significance: *p ≤ .05 **p ≤ .01 ***p≤ .001

Pretest and posttest Current Statistics Self-Efficacy (CSSE) score comparisons

Statistics self-efficacy was evaluated just before the asynchronous online statistics courses began and after both courses were completed. Self-efficacy was evaluated according to the student’s current perception of their own statistics self-efficacy and according to their self-efficacy to learn statistics using the CSSE scale, as listed in Table 3. A Wilcoxon sign rank test revealed that CSSE scores were significantly higher after the competency-based learning course (Md = 63.75, n = 8) compared to before the course (Md = 42.00, n = 8, z = −2.52, p = .01), with a large effect size (r = .63). Similarly, participants in the traditional statistics course had significantly improved CSSE scores after the completion of the course (Md = 35.5, n = 12) compared to before starting the course (Md = 35.50, n = 12, z = −3.07, p = .002), with a large effect size (r = .89). A comparison of CSSE scores from the students in the competency-based learning course were not significantly different (pretest Md = 42.00, n = 8) from those of the students in the traditional learning course (pretest Md = 35.50, n = 12, U = 30.00, z = −1.39, p = .18). The competency-based learning students’ posttest CSSE scores (Md = 63.75, n = 8) were also not significantly different from those in the traditional learning course (posttest Md = 68.00, n = 12, U = 44.50, z = −0.27, p = .79).

Table 3.

Analysis of Self-Efficacy for Statistics Among Undergraduate Students Using Wilcoxson Signed Rank Test (n=20)

	Mdna	Mdna	n	z	p	r
Current Statistics Self-Efficacy (CSSE)
Competency-based learning	42.00	63.75	8	−2.52	.01*	.63
Traditional learning	35.50	68.00	12	−3.07	.001**	.89
Self-Efficacy to Learn Statistics (SELS)
Competency-based learning	68.00	67.00	8	−0.70	.48	.18
Traditional learning	52.00	70.00	12	−2.04	.04	.42

Note: *p ≤ .0125 was used following Bonferroni adjustment for the use of multiple comparisons.

Significance: *p ≤ .01, **p≤ .001

Pretest and posttest Self-Efficacy to Learn Statistics (SELS) score comparisons

A Wilcoxon sign rank test indicated that the SELS scores of participants in the competency-based learning or the traditional learning course were not significantly higher after course completion when compared to prior to the course, as listed in Table 3. A comparison of SELS scores from the students in the competency-based learning course were not significantly different (pretest Md = 68.00, n = 8) from those of the students in the traditional learning course (pretest Md = 52.00, n = 12, U = 38.00, z = −.77, p = .47). The competency-based learning students’ posttest SELS scores (Md = 67.00, n = 8) were also not significantly different from those in the traditional learning course (posttest Md = 70.00, n = 12, U = 41.50, z = −.50, p = .62).

DISCUSSION

The knowledge scores of the students improved in both the competency-based learning and traditional learning introductory statistics courses. With its focus on the demonstration of competence, Acikgoz and Babadogan (2021) concluded that competency-based learning may help advance student learning through ongoing formative assessments that provide support at each stage of learning. Statistical modules developed for the competency-based course were designed to allow immediate formative assessments that could be repeated, if the students desired, prior to moving forward to the next module. Others have noted student satisfaction with the ongoing formative assessment characteristics of the competency-based learning approach. Patra and Khan (2019) noted that students enrolled in a competency-based research methods course were not only satisfied but also motivated to do further research after completing the course.

Unfortunately, pre-conceived anxiety and attitudes toward statistics courses are predictors of lower self-efficacy to learn statistics (Perepiczka, Chandler & Becerra, 2011). Conversely, students with higher self-efficacy have improved performance in statistics, and some have noted that anxiety may not directly impact performance (Hoegler & Nelson, 2018). In our study sample, we noted differences when the students reported they had participated in mentored research experiences. The competency-based learning group reported having more research and prior statistical exposure. Some researchers have indicated that positive attitudes toward statistics tend to increase as self-efficacy increases, but not the final course grade (Walker & Brakke, 2017). However, it is possible that increased statistical knowledge may have been related to demographic differences between the competency-based and traditional learning groups, such as previous research or statistical experience.

Self-efficacy theory has been used to understand student engagement with learning knowledge that is needed for practice; a lack of self-efficacy has been found to negatively influence students’ motivation to continue their training (Hayat et al, 2020). Competency-based learning has demonstrated effectiveness in increasing self-efficacy (Lee & Pant, 2021). In this study, CSSE scores significantly improved among students in both the competency-based learning and traditional learning statistics courses (p = .01 and p = .001, respectively). Qualitative feedback from students after the competency-based learning course was overwhelmingly positive, with students reporting that the modules enhanced their education, improved their confidence, and were appropriate for their level of education.

A theme reported by students in the competency-based learning group was that the opportunity to complete the modules on their own time was an advantage, thereby enhancing their learning. The ability to self-regulate the learning process has been studied previously in higher education. Barbosa et al. (2016) found that self-direction has another important benefit of providing protection against academic burnout among first-year medical students. The combined flexibility of the self-paced delivery of the competency-based learning model and the self-directedness were reported as attractive when feedback was received in the end-of-course evaluations.

SELS scores were not significantly different from the pretest to posttest in the competency-based learning group, but this group started with a higher median SELS score than the traditional learning group. Walker and Brakke (2017) noted that SELS scores increased significantly across the semester for introductory statistics students, but not for students in the advanced statistics courses. The researchers explained that it was likely that there was a ceiling effect for self-efficacy among more experienced students. Findings from our study did not detect significant increases in SELS scores in either group from the pretest to the posttest. We did note that the mean pretest SELS score was initially much higher in the competency-based learning group than the traditional learning group, which may be related to differences in group characteristics because of prior statistical research knowledge, such that SELS scores for the competency-based learning group may have experienced a ceiling effect.

The limitations of this study include the lack of random assignment to study the course sections, which may decrease internal validity, as samples may differ. The use of a pretest-posttest study design could also result in participants being more familiar with the tool when the posttest was administered, possibly influencing results and the ability to detect a change from baseline. However, the time interval between the administration of the tests was several months to help minimize this limitation. In addition, the small sample size of this study could limit the ability to generalize the findings.

Implications for education and practice.

Because the students reporting prior research experiences had higher pretest SELS scores, incorporating statistics lessons using a competency-based learning model may be an effective strategy to increase statistics knowledge and confidence for entry-level, undergraduate students. It is important that students in health-related disciplines develop competence with statistics and are prepared to interpret research findings in their chosen clinical practice fields. Using a self-paced model of competency-based learning may be as effective as a traditional learning course to increase knowledge and statistical self-efficacy.

CONCLUSIONS

Students enrolled in a competency-based learning course had significantly improved knowledge scores in the content areas of hypothesis testing, central tendency, and research design compared to the students in a traditional learning course. Self-efficacy improved in both groups. Therefore, both learning methods could be effective in increasing statistical knowledge and self-efficacy scores of associate and baccalaureate degree students in nursing and health-related disciplines.

Highlights.

• Both competency-based learning and traditional learning methods were effective methods for teaching statistics.

• Students felt competency-based learning allowed more flexibility with repeating course content until it was mastered.

• Statistics self-efficacy was comparably improved when students were enrolled in competency-based vs. traditional learning statistics course sections.