Design and evaluation of a revised ARCS motivational model for online classes in higher education

Abstract

In recent years, online MOOCs (Massive Open Online Courses) have been quite popular among universities which helps learners to enhance their competency skills apart from learning the regular college/university curriculum. Although distance education and online learning have been adopted gradually recently, it has become the 'New Normal.' In this situation of uncertainty, facilitators must keep themselves updated with the various teaching/learning strategies and encourage learners to get accustomed to the online classroom environment. Furthermore, assisting the learners with active engagement in the classes is essential. Hence, to create an instigated environment for assessing the competency level and addressing the motivational behaviour of the learners in the online courses, a modified version of the "ARCS" (Attention, Relevance, Confidence, and Satisfaction) model is used in this research work. The core objective of this model is to apply a modified motivational model, namely "ARCS-PC," where PC represents Professional Competency. Professional competency includes Critical Thinking skills, Digital literacy, Creative Thinking, Problem-solving, and Time Management. The incorporation of digital quizzes, assignments, and interactive activities using Information and Communication Technology (ICT) tools was done in the ARCS-PC Model. The online classroom lectures and activities were conducted using the Microsoft Teams (MS Teams) educational platform. Linear regression is performed to analyze the modified ARCS-PC model. These technology-enabled online classes and ICT tools have helped teach lifelong learning, collaborative learning, a student-centric approach, and better competency skills to effectively engage students in online courses. In our proposed method, an improvement of 11.21 % was observed in the student's performance compared to a maximum of 8.8 % in the existing traditional models. Detailed analysis and quantification of the proposed method are given in the paper.

1. Introduction

Technological advancements in education have streamlined the teaching-learning process to deliver that knowledge from any part of the world without much hassle. This method of facilitating learners using innovative tools and techniques in the courses has inculcated self-paced and extended learning for our students. Over a decade, there has been a positive shift in the teaching methodology, gradually replacing traditional and blended teaching methods with virtual classrooms. These virtual classrooms and online learning strategies have shifted learners' interest in education and provided them with easy grasping of concepts. This transition, in turn, has led to the fact that Millennials are more prone to technology than relying on traditional teaching-learning methods. The affinity of learners toward technology-driven education demands more of an online study than in traditional class settings.

Especially in the last decade, institutes worldwide have been accustomed to adapting online teaching-learning practices, including digital media and innovative pedagogical models. The blended learning scenario was adapted positively as the learners had the flexibility to utilize gadgets in the class; they replaced monotonous lectures with activities and had the autonomy to attend the lessons at their own pace. Furthermore, these interactive media create an enhanced look and feel in the classroom, grabbing learners' attention for longer. Ultimately, it elevated the satisfaction level of the students to attend the lectures without boredom and their confidence level compared to traditional lecturing sessions. However, despite the pedagogical models being successful in blended classroom teaching, applying them to online classes was still a big challenge. While delivering lectures, the primary concern occurs when the classes are conducted online and utterly dependent on digital platforms. According to the authors [1], various issues such as lack of infrastructure and motivation to attend online classes have been listed. Additionally, irrelevant content, improper time management, inadequate motivational support, lack of knowledge in using the technology, lack of clarity on the learners' progress, and absence of face-to-face interactions were also identified in online classes.

Thus, the primary motivation to carry out this research is to address some of the fundamental issues the learners face in online teaching-learning. These issues included the duration of the online class, lack of active participation, technical glitches, unavailability of supporting infrastructure, and lack of skill set among facilitators to handle online courses. Therefore, to suit the needs of Gen Z learners, there is a huge necessity to transfer classroom learning to completely digitized platforms, inculcating a student-centric environment around them. The proposed Model serves this purpose by creating an interactive and dynamic classroom environment for the students, using ICT tools and educational platforms. Furthermore, the strategies adopted in this Model focused on enhancing the learner's attention span, boosting motivation for online classes, and enhancing the student's engagement for a better academic record of the learners.

The professional competency component introduced in this study drastically impacted the learners' conceptual and higher-order thinking skills. Feedback provided to the learners based on their performances also played a significant role in online teaching-learning. The motivational and constructive feedback on their emotions helped the students boost their morale. The suggestions provided by the students on this proposed motivational model helped the faculty design their lectures better to align the concepts appropriately with the learning outcomes. Positively post-implementation of the Model, the learners in the experimental group had a significant increase in their academic scores and were persistent in the learner's performance.

1.1. Scope of improvement

The following research gap has been identified. Points a, b, c, d, and e are addressed in the proposed ARCS-PC Model.

• a.Lack of professional competency measure in the traditional ARCS model.
• b.Facilitators achieved an ambiguous measure of competency level during the progress of the respective courses.
• c.Confusing the learners by providing unclear instructions to deal with the online tests and assignments.
• d.Unavailability of feedback to facilitators during the progress of the course. As a result, the facilitators cannot adjust to the student's learning needs.
• e.The relation between student digital engagement and their performance is not emphasized.
• f.Lack of expertise and technical knowledge to use the learning management systems by both the facilitators and learners.

2. Related works

The extensive literature review focuses on various instructional models incorporated with the ARCS motivation model to reflect educators' views and their effect on the learner's performance. Multiple studies on traditional and digital learning environments using different pedagogical strategies with the ARCS motivation model are considered and compared here. The techniques discussed below include blended learning, flipped classroom methodology, implementation of ICT tools in the classroom, game-based, group-based, problem-based learning, etc., whose benefits and limitations have provided a foundation for the proposed work.

These limitations are thereby addressed in this study. This research work has contributed to augmenting the competency skills of the learners through the amalgamation of the ARCS motivational model with various skill-enhancing activities. Digital platforms are used to conduct the activities, and the competency level of the students has been measured and analyzed on different standards.

2.1. ARCS motivation model in STEM education

Many research works focused on various instructional design models like the ASSURE model (Hilal Karakis et al., 2016) to apply multimedia applications in STEM education [2]. Flipped classroom models [3,4] involve active learning strategies to teach concepts interactively and effectively. Both these models have witnessed a positive shift in the learner's attitude toward online teaching and activities applying critical thinking and analytical skills [5]. The significant benefit observed is the student's active participation in classroom activities. This rising demand for virtual classes has made the teachers upgrade themselves with the skills and expertise needed to conduct online courses using the current ICT tools in the market. Despite having the flexibility and convenience to attend online lectures and apply modernized tools and techniques, a group of learners experienced a lack of motivation and determination [6] in an online course. Compared to a traditional classroom, the primary issues faced in online classes [7] were the absence of an instructor's physical presence, lack of communication with instructors and peers, immediate responses, and a sense of seclusion experienced by learners.

Thus, an exclusive ARCS-Motivational Model [8] was designed by John Keller to address these motivational issues. This model exclusively focuses on the essential momentum through which learners can be encouraged to learn new concepts and skills inside and outside the classroom. In this Model, "A" stands for attention which is further subdivided into Perceptual, Inquiry Arousal, and, Variability. Secondly, “R” represents the topic's relevance, categorized into Goal Orientation, Motive Matching, and Familiarity with the subject. The third aspect, named confidence, is denoted by "C". It is subdivided into Performance Standards, Success Opportunities, and Personal Control provided to the learners. The final component, "S," represents satisfaction promoted through Internal and External motivation and Constructive feedback given by the teachers. Keeping this motivational Model as the foundation [9], many other integrated models were developed for different subjects across different domains [10]. One such study involves the application of Game-based learning (GBL) [11,12] along with ARCS Motivational models [13]. to design their mathematical courses and curriculum, which resulted in the positive academic achievement of learners through computerized games. Using multimedia resources with the ARCS motivational model (Wilujeng. I, 2021) has also influenced the students' learning experiences.

Even in the industrial environment, the ARCS model has successfully motivated the employees by infusing gamification techniques [14] in each component for disseminating knowledge and managing it effortlessly. Both the GBL (W.D. Huang et al., 2021) and gamification techniques have a tremendous positive impact on the cognitive thinking capability of the learners [15]. The pooled effect of both methods provides the learners with opportunities to exploit their higher-order thinking skills to solve real-time problems. Courses like mathematics, programming, and subjects related to electronics which demand critical thinking capability [16], are well taught with innovative pedagogical strategies and gaming components, as they make the learning process interactive and engaging.

It also incorporated Gagne's nine events for developing instructional sessions and an interactive program to learn analytical and theoretical geometry named “Geometer's Sketchpad” [17]. Furthermore, it helped [18] to develop effective lectures and practical sessions for aiding less-experienced teachers to bridge the gap between the delivery of the mathematical course and the grasping of the concepts by the learners. Though the result of the study was not satisfactory enough, it created a scope for further research in the pedagogical aspects. Like this research, course restructuring has been done for HTML language using gamification elements [19] like quizzes, scaffolding codes, and assignments to enhance students' engagement in class through these activities. However, this study had mixed responses from students as few observed the activities unrelated to the content, and even the facilitators were unsure about the sustainability of these techniques until the end of semester classes.

Overall, the ARCS motivational model has been perceived differently both by the teachers and students. For facilitators, the driving factors were attention and confidence, whereas, for the students, the crucial aspects were satisfaction and relevance [20]. Likewise, apart from general courses, MOOC courses conducted for Chemistry (Li, Kun., 2017) and Flipped classrooms for Physics [21] also incorporated the ARCS model into the course plan. These methodologies enabled the students to improve their cognitive skills and addressed the different learning styles of the learners. It also helped the facilitators design the course plan and contents (Ilknur Ozpinar et al., 2016) per the learner's expectations. Even the quality of teaching was enhanced as the combined pedagogical strategy was interesting for the facilitators (Y B [22] and also created modified and reliable course content to deliver to the learners of varying learning styles.

2.2. ARCS motivation model with other pedagogical models

Another level of advancement was introduced into a Learning Management System (LMS) platform (Erny Arniza Ahmad, 2021) “Moodle” in the form of boosters to motivate the learners [23]. Two groups of students were exposed to the Moodle learning platform for the course “Software Engineering." The experimental group had their sessions on Moodle with boosters in between, whereas the control group was devoid of motivational factors. The outcomes indicate that the experimental group was highly motivated to complete the tasks compared to the control group. Even for engineering education, the Flipped classroom and the ARCS model have proved to be the most accepted models for designing a course for expecting highly satisfactory performances from the learners [9]. [24] did a comparative case study on teaching programming language to computer science students. This study was conducted through Problem-Based Learning (PBL) and a flipped classroom model integrated with the ARCS model that helped the academicians formulate a lecture plan for civil engineering students [25]. Even the current advancements in Artificial Intelligence (AI) (Lin. P. Y. et al., 2021) and its implementation in the form of robots have been incorporated into the ARCS motivational model [26] to help students overcome the challenges faced in classrooms. In this method, robots were used to help the students after class hours and provided them with instant feedback based on their emotions. The motivational and emotional feedback [27] would make the student comfortable with the respective course faculty. As a result, the scores and performances of the students were improved after implementing the blended learning model.

Additionally, after a comparative analysis, the study motivated more male students toward the subject than female candidates. Similarly, it also generated automated feedback from the learners on their personal experiences [28] about the course undertaken and the quality of teaching. The recent research incorporated the English language with the ARCS model with mobile-driven tools [29] to drive students’ attention and teach seamless learning in the classroom. When applied with motivational models, these digital and mobile assistance techniques [30] positively impacted the students' practical learning. In addition, they improved teaching-learning methods to be applied in the classrooms.

Another recent development was observed in this motivational Model by combining it with Augmented Reality (AR) [31] to impart an enriched and high-quality learning experience to engineering students [32]. Using AR [33] in classrooms creates a realistic environment where the learners can apply their theoretical knowledge to understand a concept with more excitement and have fun simultaneously. Furthermore, in courses like history and biological sciences [34], where the focus is mainly on facts, dates, and pictures, AR turned out to be very handy in making the lecture sessions interactive (Khalid et al., 2021) and providing the students with a virtual learning experience. With the implications obtained through the interviews and feedback received from the learner's group [35], it was evident that the students exhibited more retention capability, improved engagement, and enhanced satisfaction.

In 2019, a comparative study on face-to-face teaching and online class were conducted for the Environmental Science course (EVS) [36], which resulted in no significant difference in the student's academic scores. The major findings stated that the lack of the physical presence of teachers made learners demotivated to attend the lectures. The practical sessions could have been much better with a group of students practising lab experiments as a team; they even gave suggestions to upgrade the evaluation methods for assessing students' grades with better accuracy. A recent study on the analysis of online teaching-learning due to the COVID-19 crisis [37] has represented both the positive and negative emotions of online classes (Gulsum Asiksoy & Fezile Ozdamli, 2016). The adoption of flipped classrooms in the ARCS model is used to determine learners' level of independence and achievement. Various studies applied strategies like semi-structured interviews, concept tests, and simulations to enhance learners' academic performance and strengthen their cognitive skills.

Despite an increase in the self-sufficiency scores of students, they lacked cooperative learning capabilities as they used the techniques only for basic concepts. Another study was performed for a mathematical course with (Hilal Karakis et al., 2016), ASSURE instructional Model, and ARCS for analyzing student's attitudes toward the computer-assisted teaching-learning process. Satisfactory results were observed in learning compared to the traditional method, but students also expressed their concerns about the duration of the activities. Even the lack of constructive course materials was observed in a few scenarios. As a result of the research, it was inferred that apart from the usage of technology, they require a relaxed environment for learning, motivational strategies to boost their morale and confidence, autonomy in selecting courses and career paths, and industry exposure.

Based on these factors [8], using the ARCS-V Model addressed the motivational challenges faced by learners and emphasized self-regulated Learning (SRL). Furthermore, Keller introduced an additional factor named volition to improve the quality of work produced by the learners. Roleplay, online discussions, and simulations were used as a part of the experiment, which led to satisfactory results. In this research work [38], proposed an integrated model where the ARCS motivational model collaborates with the Problem-based learning (PBL) strategy in a flipped classroom. The principal objective was to measure the proficiency of learners in programming skills. Even though worksheets, teaching videos, and questionnaires were used to increase the reliability, the collaboration skills among the learners were still lacking. Another related work [39] focused on educational materials designed with the motivational factors employed in the ARCS model. This study's quantitative and qualitative measures increased confidence and relevance, whereas the attention and satisfaction factors were less desirable.

An effective virtual learning environment is provided to the students through this revamped version of the ARCS motivational model, namely ARCS-PC. In this Model, the ARCS represents Attention, Relevance, Confidence, and Satisfaction, and the PC denotes Professional Competency. Furthermore, various activities were conducted to augment these skills, including digital games, assignments, polls, and video feedback conducted with the help of tools such as Kahoot, Mentimeter, Flipgrid, Praise app, etc. Thus, the ARCS-PC model gives the necessary exposure to the millennials to achieve competency skills and motivates the learners to complete the course sustaining learning outcomes.

2.3. Contributions of the paper

• a.ARCS – PC Model: Augmenting the competency skills of the learners through the amalgamation of the ARCS motivational model with various PC skill-enhancing activities.
• b.A specific measure of competency level: Measuring and analyzing the competency level of the learners on varied modalities in addition to the traditional tests and exam scores. Various modalities include Digital Assignments, Digital Quizzes, Hands-on Labs (digital), and the use of other Digital Resources.
• c.Rubrics in ARCS-PC Model: Detailed rubrics are provided for all digital activities to provide clarity and transparency in the evaluation process.
• d.Feedback from students: In addition to traditional measures like marks, input in the form of student digital interaction is measured by the facilitators during the progress of the course. This interaction measure helps them adapt to the student's current learning needs.
• e.Regression Analysis: The relation between the student's digital engagement level and performance is established through this analysis.

Further, the learners were highly motivated to complete the courses with expertise through this motivational-competency model.

3. Proposed methodology

In this section, the ARCS-PC model architecture, the data collection and sampling techniques, key constructs with respective hypotheses, tools/activities utilized, and competencies achieved are explained in detail.

3.1. Overview of ARCS-PC model

The proposed Model ARCS-PC (Professional Competency) accommodates higher-order thinking skills implemented through Gamification, Quizzes, and Flipped classroom pedagogical model. In the proposed Model, the course instructor must evaluate in one scale range between 5 and 1 score (5-high,1-low) based on the student's performance in each activity. A detailed representation of the "Professional Competency" skills is represented in Fig. 1, depicting the revised ARCS-PC model architecture with the various tools and competency skills.

Fig. 1.

Architecture of ARCS-Professional competency (PC) model.

3.2. Data collection and sampling technique

The control and experimental group implementation involve two different groups of students. One was a group of 34 students for the course Software Architecture and Design (SAAD), and another was a group of 37 students for the course Multicore Architecture (MA) as given in Table 1. The courses were conducted post the pandemic era, therefore a population size of 71 was considered appropriate to understand the effects of the proposed method in an online classroom. Additionally, a t-test statistical method is chosen for this population as the number of participants is 71 and the performances of only 2 different groups (control and experimental) are compared.

Table 1.

Data collection.

Course	Group	Mode	Model Used	No. of Students	Measure of Performance
Software Analysis and Development	Control	Online	ARCS	34	CAT1
	Experimental	Online	ARCS-PC		CAT2, FAT
Multicore Architecture	Control	Online	ARCS	37	CAT1
	Experimental	Online	ARCS-PC		CAT2, FAT

However, based on the performances obtained after the implementation of the proposed model the population size could be expanded in the future and the application of the ARCS-PC model can be incorporated into other undergraduate and postgraduate courses as well.

3.2.1. Control group

In this group, facilitators delivered the topics and concepts through online classes. In addition, the Continuous Assessment Test 1(CAT 1) was conducted to measure learners' progress.

3.2.2. Experimental group

In this, the ARCS-PC model was used to deliver the topics. The online lecture session comprised 30 min and dedicated 20 min to online activities. On completion of all the activities, teachers used rubrics to evaluate the individual marks of the learners. Post-implementation of the proposed method, the professional competency of every student was evaluated using numerous activities and tools. First, we analyzed the individual competencies of the learners to prove the corresponding hypothesis for five different competency factors. The pre-test and post-test scores were analyzed to determine the performance level difference before and after implementing the proposed method.

3.2.3. Sampling technique

PC in the five sub-factors (problem-solving, creative, and critical thinking, digital literacy rate, and time management skills) was measured using t-test statistics. First, the t-test obtained the results for each sub-factor. Then these results were compared with the hypotheses framed. The t-test statistical method is applied in this work to determine whether the proposed ARCS-PC model has better effects on the student's performance and engagement or not. After the t-test analysis, the overall competency was measured using the linear regression method by comparing the assessment scores obtained by the learners in CAT 1 and the implementation of the ARCS-PC model.

3.3. Key constructs of the ARCS- PC model

The five primary key constructs of the proposed work are listed as follows:

• •
  Critical Thinking: μ_CT

  • o
    The quantification measure used for this factor is represented by the ratio of pre-and post-scores obtained in the final assessment test marks.
• •
  Problem Solving: μ_PS

  • o
    The ratio of pre-and post-scores obtained in the final assessment for laboratories is the construct used to measure the students' problem-solving skills.
• •
  Creative Thinking: μ_CrT

  • o
    The ratio of pre-and post-scores obtained through the number of innovative products developed and delivered by the students is the quantification measure used for creative Thinking.
• •
  Digital Literacy: μ_DL

  • o
    The pre-and post-scores of the final assessment are obtained via digital quizzes, and the knowledge to utilize online resources is used to measure the learners' digital literacy.
• •
  Time Management: μ_TM

  • o
    The quantification measure used for time management is represented by the ratio of pre-and post-scores obtained in the final assessment test based on the amount of time spent on various digital activities.

3.4. Hypothesis for key constructs of PC component

• •
  Critical Thinking:

  • o
    Facilitators gave video-based assignments to the students to leverage their critical thinking skills. First, the class was divided into a few heterogeneous groups, and each group chose a topic in the respective course. Then, the students submitted videos based on the chosen topics by every individual.

Hypothesis: "The higher the probability of students who have created the videos, the higher the performance of the learners."

• •
  Problem Solving:

  • o
    The teachers used laboratory questions based on Higher-Order thinking skills to measure the problem-solving capability of the students. Primarily, questions based on complex calculations were assigned to the learners.

Hypothesis: "The more the number of HOT questions asked, the higher the probability of an increase in the performance of the learner."

• •
  Creative Thinking:

  • o
    The project-based learning provided Hands-on experiences to foster the learners' creative thinking skills.

Hypothesis: "The higher the probability of exposing the students to practical lab exercises, the higher the probability of the learner performing well."

• •
  Digital Literacy:

  • o
    This factor emphasizes how well-versed a student uses the digital tools applicable to the courses. Using the Microsoft Teams platform seamlessly, learners were encouraged to get accustomed to various processing, simulation, and graphical tools.

Hypothesis: "The more the number of tools explored by the students, the higher the probability of understanding the concept and performing well."

• •
  Time management:

  • o
    Time management was measured using the learner's submissions for digital activities. The students were expected to complete the digital assignments within the stipulated time.
  • o
    While engaging in these activities, they also challenged the learners to think, apply and analyze the questions by managing their time efficiently.

Hypothesis: "The students take less time to complete the gamification activity, the higher the level of their performance."

Facilitators can use the following method to improve professional competency and evaluate it. First, a teacher will divide the class timing into three parts.

• 1.First, 20–25 % of the class time will be dedicated to Interactive Quizzes, Gamification activities, flipped classroom sessions, and Digital activities like submitting a self-recorded video to define the understanding of a topic.
• 2.50 % of the time will be utilized for delivering the course concepts.
• 3.20–25 % of the class timing will be used for clarification and interactions.

The above method will improve the student's competency, and the course instructor will also be able to evaluate and understand the level of competency students achieve in a class. It will also change the monotonous tone of the online lesson.

Table 2 summarizes the evaluation metrics used to measure the learners' competency skills and the hypotheses framed.

Table 2.

Hypothesis for key constructs of PC component.

Factors	Hypothesis	Data tested	Quantification Metric
Critical Thinking	H0: Post-score value ≥ μ_CT * pre-score value. H1: Post-score value < μ_CT * pre-score value.	Video Assignment marks Post-score: Sum of marks scored in questions relevant to the video assignments submitted by a student. Pre-score: Sum of marks scored in all the other questions.	The ratio of the post-to-pre-score μ_CT = 1.8
Problem-Solving	H0: Post-score value ≥ μ_PS * pre-score value. H1: Post-score value < μ_PS * pre-score value.	Lab Final Assessment marks Pre-score: Marks scored in the FAT lab for CAM with <50 % HOT questions. Post-score: Marks scored in FAT lab for CAM with ≥ 50 % HOT questions.	The ratio of the post to pre-score μ_PS = 1.86
Creative Thinking	H0: Post-score value ≥ μ_CrT * pre-score value. H1: Post-score value < μ_CrT * pre-score value	Innovative Product Development Post-score: No. of student teams contributed to the innovative product. Pre-score: No. of students who have not contributed to the innovative development.	The ratio of the post to pre-score μ_CrT = 2.02
Digital Literacy	H0: Post-score value ≥ μ_DL * pre-score value. H1: Post-score value < μ_DL * pre-score value	Digital Quiz Marks Post-score: Sum of marks scored in questions relevant to the tools used by a student in lab/project/classroom activities. Pre-score: Sum of marks scored in all the other questions.	The ratio of the post to pre-score μ_DL = 2.04
Time Management	H0: Post-score value ≥ μ_TM * pre-score value. H1: Post-score value < μ_TM * pre-score value	Digital Activity Marks Post-score: Sum of marks scored in digital activity submissions within ≤ 60 % of the time. Pre-score: Sum of marks scored in all the other activities.	The ratio of the post-to-pre-score μ_TM = 1.61

3.5. Tools/activities for PC component

The overall proposed ARCS-PC model depicted is a modified version of John Keller's ARCS – ARCS-Motivational model [8], which mainly works on enhancing the core competency skills of learners through motivational models in the classes. This Model involves the fusion of the original ARCS factors with the proposed PC component.

These competency skills involve critical thinking, problem-solving, creative thinking, digital technology literacy, and time management. These skills are mandatory in every individual's life to tackle real-world problems and achieve lifelong learning skills.

This research is carried out on two undergraduate and postgraduate courses: Software Architecture and Design (SAAD) and Multicore Architecture (MA). The online classes are conducted using the Microsoft Teams platform. Table 2 represents the hypotheses of PC components. Table 3 given below depicts the activities and tools for the PC component involved in designing the course plan for these two courses.

Table 3.

Tools/activities for objective function optimization (PC component).

Factor	Activities	Time Allotted	Tools
Critical Thinking	Video assignments: Video-based concept explanation by a team of students	20 min	1. Whiteboard - collaborative learning, Brainstorming 2. Stream – short flipped videos
Problem-Solving	Higher-Order Thinking (HOT) Lab Questions	60 to 90 min	1. Session plan/Lab manual- Shared with learners before class 2. Tinkercad - Simulation
Creative Thinking	Hands-on Project	3 months	1. Wiki–application-oriented questions
Digital Literacy	Digital Quizzes, Online resources	20 min	1. Kahoot and Quizziz– Game-based activity
Time Management	Digital activities	20 to 30 min	1. Mentimeter and Polly: Open-ended questions, word cloud, and opinion polling 2. Open book Tests: measuring analytical and logical skills

All the tools are available in Microsoft Teams, a collaborative platform to conduct many live events, online classes, and activities with an enormous number of user-friendly features, and third-party apps.

4. Implementation

The implementation section focuses on the activities implemented to achieve the professional competencies considered along with the corresponding rubrics.

4.1. Implementing activities for PC (professional competency) model

The activities influencing the five key constructs are categorized below.

• •
  Critical Thinking, Problem-Solving, and Creative Thinking Skills:

  • o
    These skills are assessed through various hands-on experiences in the laboratory, application-oriented, and project-based questions. Solving such HOT questions helps the learners to unleash their capabilities to prove their expertise and improve their critical thinking, problem-solving, and creative thinking skills. (Proof attached in APPENDIX A.1.)
• •
  Digital Literacy Rate:

  • o
    The assignments in Microsoft Teams are very much customized as they aid the facilitators in observing the status of the tasks and submission time. In addition, the MS-teams app also provided grade points and feedback for those individual assignments. (Proof attached in APPENDIX A.2.)
  • o
    Exposure to varied digital resources and digital quizzes was also incorporated to enhance the digital literacy factor and to obtain a quick insight into the learners' basic knowledge about the respective course. (Proof attached in APPENDIX A.3.)
• •
  Time Management Skills:

  • o
    The facilitators also conducted digital classroom activities like video assignments and projects to strengthen the students' time management skills and concentration through an enjoyable experience. (Proof attached in APPENDIX A.4.)

4.2. Rubrics for PC-component activities

On completion of these activities, the outcomes were assessed using rubrics. The rubrics are designed for seven categories on a 4-point scale, i.e., a total of 28 marks and the two marks are allotted based on the summary reports given by Insight App. Table 4shows the rubrics designed for the PC component.

Table 4.

Rubrics for PC component activities.

The insight app in Microsoft Teams enables the facilitators to have a complete perception and summary report of the student's engagement, involvement in discussions, activities, and overall grades in the course. (Proof attached in APPENDIX A.5.)

5. Experimental analysis

The experimental analysis is performed to achieve the contributions listed under section 2.4. First, the pre-scores are obtained from Continuous Assessment Test 1 (CAT) marks post-implementation of the traditional ARCS model. Then, after incorporating the proposed ARCS-PC model, the post-scores are computed from CAT 2 and Final Assessment Test marks (FAT).

5.1. Evaluation of key constructs

The pre-and post-scores of each student were used to measure the five individual professional competency factors.

• •Critical Thinking (CT):

For measuring the critical thinking skills of the learner, the video assignments submitted by them are considered. These test scores are segregated as pre and post-test scores given in Table 5 based on their scores.

Table 5.

Computation of Critical Thinking.

Post-score: This is the sum of marks scored by the students in questions relevant to the video assignments submitted by them on a respective topic.

Pre-score: This represents the sum of marks scored in all the other questions.

In the case of the post-score, the actual mark is multiplied by "1," and for the pre-score, the mark is multiplied by "0". Therefore, in the case of Q1, the mark will be multiplied by 1, whereas, in the case of Q2, the mark will be multiplied by 0. Thus, the actual mark obtained for Q1 will be 7, and for Q2, it will be 0. Therefore, it is evident that after the video assignment submission, the learner's answering capability improved on that particular topic. Hence, they scored well in Q1 (Post-score) to Q2 (Pre-score), for which they did not submit the assignment. The quantification metrics for critical Thinking denoted as "μ_CT” are calculated using the pre/post scores for which the equation is given below as follows:

Pre-score = Total Marks obtained formula – Post score (1)

Post-score = ∑ (Marks obtained in each question * Pre/Post Score) (2)

Total score = ∑ (Marks obtained in each question) (3)

After calculating the pre-and post-scores for every individual using the above formula, "μ_CT” is obtained through the ratio of post-score to pre-score. The value of “μ_CT” obtained is 1.8. Henceforth, our hypothesis H0 is proved to be accurate as the post-score value ≥ μ_CT * pre-score value.

• •Problem-Solving (PS):

The quantification of problem-solving skills is determined by the number of higher-order thinking (HOT) lab-based questions answered by the learner. The pre-and post-scores presented in Table 6 are derived from the FAT lab scores obtained by the students.

Table 6.

Computation of Problem-Solving.

Post-score: This is the sum of marks scored by the students in the FAT lab for the Continuous Assessment Test (CAT) with ≥ 50 % HOT questions.

Pre-score: This is the sum of marks scored by the students in the FAT lab for the Continuous Assessment Test (CAT) with <50 % HOT questions.

In the case of Lab 1, the mark will be multiplied by 0, whereas, in the case of Lab 3, the mark will be multiplied by 1. Thus, the actual mark obtained for Lab 1 will be 0, and for Lab 3, it will be 9. Thus, the above table depicts that in whichever lab the number of HOT questions increased, i.e., Lab 3,5, and 6 (post-score), the learner's performance was much higher compared to Lab 1,2, and 4 (pre-score). Using the same equations (1), (2), (3) mentioned above, the pre- and post-scores are determined for the remaining samples. The value of “μ_PS” obtained is 1.86, which proves that our hypothesis H0 is valid as the post-score value ≥ μ_PS * pre-score value.

• •Creative Thinking (CrT):

Creative thinking skills are measured through project-based learning and hands-on experiences. The pre-and post-scores are determined from the number of innovative products the students developed after the course completion, as depicted in Table 7.

Table 7.

Computation of Creative Thinking.

Post-score: No. of student teams contributed to the development of the innovative product.

Pre-score: No. of students who have not contributed to the innovative product development.

In the case of student 'AAA,' the mark will be multiplied by 1 whereas, in the case of the student named 'DDD,' the mark will be multiplied by 0. Thus, the actual mark obtained by 'AAA' will be 75, and by 'DDD,' it will be 0. Then, using the (1), (2), (3), all the remaining pre- and post-scores are calculated for every individual. As a result, the value of "μ_CrT” obtained is 2.02, which proves that our hypothesis H0 is valid as the post-score value ≥ μ_CrT * pre-score value.

• •Digital Literacy (DL):

The literacy rate of attempting and attending digital quizzes on different platforms in the respective courses has been quantified. The pre-and post-scores are obtained from digital quiz marks given in Table 8 that the learners acquire.

Table 8.

Computation of Digital Literacy.

Post-score: The sum of marks scored in questions relevant to the tools used by a student in lab/project/classroom activities.

Pre-score: The sum of marks scored in all the other questions.

In the case of the post-score, the mark is multiplied by "1," and for the pre-score, the mark is multiplied by "0". Therefore, in the case of Q4 mark will be multiplied by 0, whereas, in the case of Q5, the mark will be multiplied by 1. Thus, the actual mark obtained for Q4 will be 3, and for Q5, it will be 9. Therefore, it implies that as the students had experience with the tool, they could answer the Q5(Post-score) in a better way when compared to Q4 (Pre-Score). Similarly, for all other students, the pre-and post-scores are determined using the (1), (2), (3) mentioned above. On completion, the value of “μ_DL” obtained is 2.04, which proves that our hypothesis H0 is valid as the post-score value ≥ μ_DL * pre-score value.

• •Time Management (TM):

Time management skills have been quantified through various online digital activities ). The marks scored in performing the digital activities are considered to calculate pre- and post-scores represented in Table 9 under this category.

Table 9.

Computation of Time Management.

Post-score: Sum of marks scored in digital activity submissions within ≤ 60 % of the time.

Pre-score: Sum of marks scored in all the other activities.

In the case of the post-score, the mark is multiplied by "1," and for the pre-score, the mark is multiplied by "0". Therefore, in the case of Q2 mark will be multiplied by 0, whereas, in the case of Q3, the mark will be multiplied by 1. Thus, the actual mark obtained for Q2 will be 4, and for Q3, it will be 9. The above table implies that since the student has completed the digital activities within or less than 60 % of the specified time, they have scored better in Q1, Q3, Q5, and Q6(Post-score) when compared to Q4 (Pre-Score), took more than 60 % of the time. Similarly, the pre-and post-scores are determined for all other students using the (1), (2), (3). Post completion, the value of “μ_TM” obtained is 1.61, proving that our hypothesis H0 is accurate as the post-score value ≥ μ_TM * pre-score value.

The above calculations prove that all 5 hypotheses framed for the individual key constructs of the PC component are true. Therefore, since there has been an increase in individual professional competency components, it can be concluded that overall competency is also enhanced after the implementation of the ARCS-PC model.

5.2. Overall professional competency achieved through individual key constructs

This section comprises the pre-and post-scores obtained by the students through the implementation of the ARCS and ARCS-PC models. First, the pre-scores are obtained from Continuous Assessment Test 1 (CAT) marks by implementing the traditional ARCS model. Then, after incorporating the proposed ARCS-PC model, the post-scores are computed from CAT 2 and Final Assessment Test marks (FAT).

The pre-scores listed in Table 10 represent the marks obtained by the learners in CAT 1 for the course SAAD, and Table 11 represents the marks for the MA course. Whereas, Table 12 and Table 13 represent the marks obtained by the experimental group after implementing the ARCS-PC model for the same subjects. The grading scheme is implemented with the help of the rubrics presented in Table 4 ).

Table 10.

CAT 1 marks - Software Architecture and Design.

S. No	Dept.	Institution	CAT 1 marks	Attendance
1	CSE	VIT Chennai	8	20
2	CSE	VIT Chennai	9	22
3	CSE	VIT Chennai	13	23

Table: 11.

CAT 1 marks – Multicore Architecture.

S. No	Dept	Institution	CAT 1 marks	Attendance
1	CSE	VIT Chennai	24	20
2	CSE	VIT Chennai	20	22
3	CSE	VIT Chennai	15	22

Table 12.

CAT 2 and FAT marks - Software Architecture and Design.

Dept.	Quiz	GBL	Wiki	Online class	Whiteboard	Flipgrid	Assignment	Total	Attendance
CSE	3	3	1	3	4	4	4	22	30
CSE	4	3	2	3	4	4	3	23	27
CSE	2	3	4	2	3	3	3	20	24

Table 13.

CAT 2 and FAT marks - Multicore Architecture.

Dept.	Quiz	GBL	Wiki	Online class	Whiteboard	Flipgrid	Assignment	Total	Attendance
CSE	3	3	4	3	2	4	4	23	30
CSE	3	4	2	3	3	1	3	19	30
CSE	2	4	4	2	3	3	3	21	24

5.3. Establishing relations between professional competency and student engagement

Based on the above marks, we did a regression analysis for the control and experimental groups in the respective courses. The variables used were: the student's engagement in the class, i.e., the independent variable as “X values”, and the marks obtained by the learner, i.e., the dependent variable as “Y values”. The individual's engagement in the online class has been obtained by applying the insights available in Microsoft Teams. Linear regression is applied to predict the relationship between the student's academic performance and their engagement level. The academic score is predicted, and Student engagement is the driving factor.

5.3.1. Regression analysis for Software Architecture and Design

Fig. 2 represents the linear regression graph plotted for SAAD to identify the significant difference in the student's marks and engagement levels based on the teaching methodology applied in the classrooms.

Fig. 2.

Regression analysis of software architecture and design.

In this analysis, the post-scores of the learners gradually increased with the pedagogical Model in the online classroom. However, contrary to the post-scores, in the pre-scores for CAT1, student engagement is high, but there is no significant increase in the learners' performance. Thus, this regression analysis shows that in the experimental group, there is a drastic increase in the trend of the performance of students when compared to the control group.

5.3.2. Regression analysis for Multicore Architecture

The regression analysis for the Multicore architecture course is given in Fig. 3. It shows that despite having a good engagement level, a high deterioration was observed in the learners' performance in the pre-scores. In contrast, the experimental group produced good results after implementing the ARCS-PC model.

Fig. 3.

Regression analysis of multicore architecture.

5.4. Overall accuracy of the ARCS- PC model

Post calculation of the regression analysis, the t-stat value has been determined for both Multicore Architecture (MA) and Software Analysis and Design (SAAD) courses to evaluate the correctness of our hypotheses. The t-stat value obtained for SAAD is 1.45 and for MA is 0.76, indicating that the learner's post-scores are significantly different from the average of their pre-scores, thus, proving our hypotheses to be true. Then, the mean absolute percentage error (MAPE) is computed for the respective subjects separately. The overall accuracy of the Model was calculated using this MAPE value. The MAPE value for SAAD and MA is calculated as 10.31 and 11.655, indicating that the SAAD course has 89.69 % accuracy and the MA course has 88.34 % accuracy. Finally, the mean absolute percentage error for the overall dataset is calculated as 11.21 % using the actual and predicted values as shown in Table 14. The model's accuracy is obtained as 88.79 %.

Table 14.

Datasets for MAPE calculation.

S. No	Actual Values	Predicted Values	ABS
1	23	18.14	21.13043
2	19	19.58	3.052632
3	21	22.46	6.952381
MAPE			11.21907

The implications of this research demonstrated that the calculated "t-stat" values for SAAD (1.45) and MA (0.76) showed a significant difference in the average pre- and post-scores of the students. Thus, the obtained outcomes proved the hypotheses framed to be true in all 5 professional competency components. Therefore, the ARCS-PC proposed model has a better performance as it clearly shows that every activity performed in this Model positively impacted the student's attitude, academic scores, and engagement level. Furthermore, the graphical results obtained from the regression analysis also depicted how the proposed Model addressed the present-day challenges faced by learners in an online classroom through technology.

• i.Gradual enhancement was observed in the learners' competency skills and engagement levels.
• ii.Students were encouraged to work on challenging tasks to exploit their critical and creative thinking skills.
• iii.These digital tools and platforms also elevated the digital literacy rate, time management, and problem-solving skills.

6. Findings and discussions

We have a comparison of our proposed ARCS-PC model with traditional methods based on all four factors of ARCS. The inferences observed prove that the modified version of the ARCS model has created an instigated online environment to engage learners for a longer duration and elevate their performances in digital courses. The significant advantage observed post-implementation of this proposed Model was the inculcation of student-centric learning among the students and a thorough motivation to enrol in the online classes and attend them without hesitation. Even the assessments to measure professional competency were learner-oriented and focused on challenging the student's ability to leverage their cognitive and creative thinking skills. The overall success rate of the proposed Model is depicted in Table 15 to represent the positive shift in the performance and engagement levels of the students.

Table 15.

Comparative results of Traditional Methodology (ARCS Model) with the proposed methodology (ARCS-PC Model).

Thus, the experimental group's quantifiable results demonstrated the learners' positive attitude and behaviour [40] toward this teaching-learning approach. In addition, the proposed Model drastically motivated the teachers to incorporate and use other tools and techniques in online sessions. Additionally, implementing this innovative pedagogical approach also increased the satisfaction level of the teachers and encouraged them to apply various other teaching-learning approaches in their regular teaching patterns. Furthermore, a comparative analysis of the model's overall performance with the existing works related to the ARCS motivation model represented in Fig. 4 is performed to support the model's efficiency.

Fig. 4.

Comparative analysis of the overall performance of the Model.

The AI-enabled ARCS model proposed by (Lin, P. Y, 2021) produced a difference of 6.18 when implementing the ARCS model in their work compared to traditional classroom teaching. In this work, apart from the hypothesis framed for satisfaction, all the other 3 factors were satisfied. Whereas the STEM-enabled ARCS proposed by (Y B [22] stated that there was a gradual increase in the performance of the population over time. The difference in the motivational model's pre- and post-incorporation was 11.32. Another study involving the multimedia-enabled ARCS model demonstrated by Ref. [41] calculated the overall difference as 35.42 after comparing the average of all the factors before and after the successful implementation of the ARCS model. While the ARCS-PC model proposed in this work produced almost similar results as that of the Multimedia-based ARCS model about ARCS factors. In addition, incorporating the PC component with the ARCS produced a difference of 46.63 while comparing the performances of both the control and experimental groups. Therefore, based on the above comparison, it is evident that the proposed ARCS-PC model has produced a drastic positive shift of 11.21 % in the performance and engagement levels of the learners compared to the other ARCS-based models. It is also more effective and accurate than the existing works cited above.

However, a few shortcomings of this research work are the diverse population chosen to apply the ARCS-PC model. The proposed Model was applied to a group of learners from different batches. This implementation resulted in having students with different levels of intelligence and skills, thus affecting the overall performance of the class. Selecting the sample population from the same batch of students might produce better comparisons regarding their competency skills. Another limitation is the number of factors that determine the learners' performance. Only two factors, namely the assessment scores and the engagement level of the students in the activities conducted, considered in this work determined the performance levels of the learners. This work is missing considering the non-academic factors affecting the learner's achievement. The other limitation is using only one educational platform with few basic active learning activities to conduct virtual classes despite a wide range of tools and techniques available in the market.

7. Conclusion and future work

The inferences show that the proposed ARCS-PC model was successful and has served its purpose of increasing students’ academic performance [42], engagement, and participation in online courses.

• •The competency skills and the retention rate [43] of the students are augmented through the incorporation of the PC component in the traditional ARCS model.
• •The competency level of the learners is measured successfully using different modes of assessment.
• •The creation and sharing of rubrics with the students provided a transparent grading system for the students.
• •The excellent feedback from the learners about the digital interaction helped the facilitators innovate their teaching process for the learners' betterment and guided them individually whenever required.
• •The proposed method also established a linear relationship between the student's performance and engagement (Rebecca Miller Waltz, 2021).

Since the contributions of the work have been achieved, the proposed Model has successfully boosted the confidence and morale of the students to excel in the courses, on the ‘motivational factor’ and ‘professional competency. Furthermore, the activities throughout the course motivated the students to attend the online classes with high enthusiasm and motivation [44], as each session was filled with challenging tasks. Apart from the activities used in this research work, the course instructors were also encouraged to integrate other pedagogical techniques and strategies into this motivational Model to leverage other students' computational skills.

However, the lack of expertise in using and implementing other technological tools and techniques still needs to be addressed in future work. Apart from this, to obtain a detailed report on the overall accomplishment of an individual learner, we can consider various other academic and non-academic factors to get a holistic view of the learner's progress throughout a particular course. Furthermore, this research work can be extended by applying other ICT tools and pedagogical models like Flipped classrooms, gamification, and Augmented Reality (AR)/Virtual Reality (VR) to maximize students’ active engagement and enhance the quality of teaching [45]. Furthermore, other technology-enabled strategies and educational platforms can also be incorporated as a part of this Model for various other online undergraduate and postgraduate courses to exploit learners' knowledge and conceptual skills. Additionally, extensive analysis can be carried out using Learning Analytics to obtain greater insight into every student's progress individually.

Data availability

The data used in this research has been uploaded to the given Github Link: Data Availability.

Additional information

No additional information is available for this paper.

CRediT authorship contribution statement

Monica Maiti: Writing - review & editing, Writing - original draft, Software, Methodology, Formal analysis, Data curation, Conceptualization. M. Priyaadharshini: Supervision, Resources, Methodology, Conceptualization. Harini S: Writing - review & editing, Validation, Supervision, Software, Resources, Methodology, Formal analysis, Data curation.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A.

Professional competency mainly measures the learners' creative and critical thinking skills. The activities under this category impact the students' time management and digital literacy rate as well as the learners' analytical and logical thinking capability.

The problem-solving and critical thinking skills are assessed through various student assignments given on Microsoft Teams comprising application-oriented and scenario-based questions. Solving such questions helps the learners to unleash their capabilities to gain knowledge and prove their expertise.

• 1.Student Assignment - Software Architecture and Design

Furthermore, the status of the assignments and the time at which the assignments were submitted are visible to the facilitators in a customized manner. It can even provide grade points and feedback for the assignments submitted individually.

• 2.Students Assignment Status- Software Architecture and Design

Digital quizzes conducted on Kahoot provided the learners with an engaging and fun-filled experience in the online sessions. Additionally, the digital assessments acted as a perfect medium to measure the digital literacy rate of the students.

• 3.Digital Quiz on Kahoot

Apart from just conducting the digital quizzes on Kahoot!, the leaderboard feature added extra learning enthusiasm among the learners. Such, rewarding features helped the facilitators maintain a healthy learning environment in their classrooms.

• 4.Leaderboard on Kahoot - Software Architecture and Design

The insight app in Microsoft Teams enables the facilitators to completely perceive the student's engagement, involvement in discussions, activities, and overall grades in the course. This information can further help perform learning analytics on the student's data.

• 5.Insight App in MS Teams- Software Architecture and Design

The game-based activity and opinion polling induced Perceptual and Inquiry arousal among learners. These activities were conducted using Polly and Mentimeter, which help conduct quizzes and polls for the live audiences.

• 6.Polls using Polly App – Multicore Architecture

The familiarity of the topic is presented to the learners through collaborative learning, making them work on application-oriented scenarios via Wiki, an interactive and collaborative tool to share information seamlessly.

• 7.Wiki activity - Software Architecture and Design

The Performance requirements for the activity are clearly stated by brainstorming various ideas and encouraging the learners to perform an interactive activity using Whiteboard, an innovative canvas to work collaboratively with the instructors and peers at a time.

• 8.Whiteboard Activity- Software Architecture and Design

Open book exams provided Success opportunities, and faculty members gave feedback via emails and personal comments to the students to give them personal control over their performances and progress.

• 9.Open Book Exams – Multicore Architecture

Praise application in Microsoft Teams proved to be an excellent tool to boost extrinsic motivation in learners. In addition, digital badges are posted for toppers in classroom activities, which helps boost the learner's morale.

• 10.Praise App Digital Badges - Software Architecture and Design