Effect of animated and interactive video variations on learners' motivation in distance Education

Abstract

One of the objectives of this research is to develop and validate the Instructional Material Motivation Scale for Single-Use (IMMS-SU) instrument in the Turkish context. The IMMS-SU was developed and validated in a two-phased process on a sample of 1654 students. The Exploratory Factor Analysis revealed that IMMS-SU included 14 items (χ2 = 332.59; sd = 74; p < 0.001), the fitness indices were found to be RMSEA = .077; SRMR = .040; AGFI = .88; NFI = .95; CFI = .96; and GFI = .92. The Cronbach’s Alpha coefficients regarding the whole scale was calculated as α = 0.95. Thereafter, in the second study, the animated and interactive video materials used in distance education were scrutinized in the context of openness to different materials, time spent viewing, motivation, and cognitive load. A total of 933 students participated who had a distance education experience. In order to collect data, the extraneous cognitive load instrument (Kalyuga et al., Human Factors, 40(1), 1-17, Kalyuga, S., Chandler, P., & Sweller, J. (1998). Levels of expertise and instructional design. Human Factors, 40(1), 1–17. 10.1518/001872098779480587), IMMS-SU, and questionnaire items were used. According to the findings, it was determined that animation and interactive video materials did not cause a higher level of cognitive load on the participants, and both groups had higher material motivation. In addition, it was revealed that interactive video materials caused a higher extraneous cognitive load in participants than animation group. It was figured out that as the openness levels of the participants watching the animation and interactive materials decreased, their cognitive load levels increased. In the light of the results, some suggestions have been recommended for further research.

Introduction

Distance learning is one of the areas most strongly affected by technology. The spread of information and communication technology in particular have led to the emergence of a variety of alternatives for distance education (Anderson & Dron, 2011). The development of computer-based multimedia materials enables learners to interact with rich content and have an active learning experience in virtual learning environments (Adarkwah, 2021; Mayer, 2005b; Suthers, 2006). The learner's interaction with the content has a key position in the context of the interaction dimensions defined by Moore (1993). Moore states that multifaceted interaction (i.e., learner-learner, learner-instructor, and learner-content) could be effectively achieved with brand new technological learning environments (Dursun et al., 2015). These learning environments could be shaped as: text-based, audio-based, pictorial-based, video-based, or interactive software-based (Wang et al., 2020). With learning technologies, distance learning activities are experiencing a significant transformation. This transformation is experienced intensively in teaching materials. Individual learning objectives might be met using materials based on a multifaceted of content kinds in online learning environments. Research on multimedia learning experience revealed that alternative contents could have various effects on the learning process. The learning experience has changed dramatically in recent years, thanks to the efficient utilization of audio-visual resources. The transition from text-oriented, non-interactive materials using static images to video and animated materials, where the learner will be more motivated and learn, explains this transformation. When the reflections of current technologies on distance learning activities are examined, both animated (Azizah & Widiartin, 2019; Handayani et al., 2020; Shiu et al., 2019; Zheng et al., 2020) and interactive materials (Daita et al., 2019; Iasha et al., 2019; Richtberg & Girwidz, 2019; Teese et al., 2020) are often used in distance-learning activities.

Multimedia-rich learning environments not only create a unique learning experience for participants, but also have an impact on their engagement. When varied of multimedia materials are incorporated into learning activities, individuals exhibit different types of engagement, especially behavioral, cognitive, and emotional engagement, (Lin & Li, 2018). Behavioral engagement refers to an individual’s focus and effort; emotional engagement refers to an individual’s emotional nature, including interest and motivation, whereas cognitive engagement refers to investment of learning which focuses on learning strategies and self-regulation (Filsecker & Kerres, 2014; Fredicks et al., 2004; O’Brien & Toms, 2008; Phillips et al., 2014). Individuals who cognitively and behaviorally focus on learning may participate in the learning process to a greater degree, and, by understanding the material, they may establish an advanced learning experience (Blumenfeld et al., 2004). Individuals' learning is improved when they take an active role in the learning process and engage with effort (Paas et al., 2003). Therefore, engagement in the learning process is crucial. Individuals who engage in the learning process, on the other hand, will utilize more cognitive resources and process in more detail, affecting their effective cognitive load (Miller, 2015). Accordingly, the utilization of cognitive processes is critical in the formation of cognitive load. Cognitive load occurs when engagement in learning content is ensured with attention and effort. (Kirschner, 2002). The Cognitive Load Theory states that the ways in which information is presented has an impact on the learners' performance (van Merrienboer & Sweller, 2005). Therefore, it is anticipated that extraneous cognitive load would diminish as learning becomes simpler for those who engage in the learning content behaviorally and cognitively. The effects of emotional engagement may be defined in terms of motivation. When a learning material is presented in a multimedia environment in a way that stimulates motivation, this can secure the emotional engagement of individuals and increase their interest in learning (Hsu, 2016). The engagement is positively associated with motivation and learning performance (Bailey et al., 2021; Guthrie & Wigfield, 2000; Lee et al., 2019; Rajabalee & Santally, 2021). In various conceptual contexts, the term engagement has been defined in multifaceted ways (Engle & Conant, 2002; Gresalfi et al., 2009; Hickey & Anderson, 2007). In the current study, the notion of engagement describes how a learner acts by exerting effort and converting effort into action through the use of learning materials within the context of an activity (Filsecker & Kerres, 2014). The utilization of learning materials should thus be encouraged cognitively, behaviorally, and emotionally when considering the influence of engagement on learners' learning process. Thus, the extraneous cognitive load may be avoided, and learning can be supported by enhancing motivation. Within this context, this study examined two types of innovative learning material: animated and interactive video.

Animated video

The use of animation in the learning process has been shown to concretize complex subjects and create an enjoyable learning experience without causing anxieties regarding time (Davis & Landay, 2004; Lin & Li, 2018). Animations represent an advantage over still images in that they are able to visually represent conceptual changes, processes and dynamics. When compared to still images, they are not only more realistic, they are also better for teaching practical information (Höffler & Leutner, 2007). Moreover, material presented in such a visually distinct manner is thought to increase students’ motivation in connection with their interest in advanced technology (Alessi & Trollip, 2001; Qaddumi et al., 2021). Another educational medium that can increase motivation and support the efficient functioning of cognitive load is interactive video.

Interactive video

Through the inclusion of interactive elements in videos, users are able to interact with the video content as well as with the developer and with other users during the learning process. With interactive video, the individual is no longer a passive viewer (Wachtler et al., 2016), but is able to direct the process and communicate with others (Geri et al., 2017). This innovative type of video was first introduced into distance-learning platforms with the aim of increasing interactivity during the learning process (Cherrett et al., 2009; Schwan & Riempp, 2004). In addition to providing meaningful support for learning, the increased ability of individuals to interact through interactive video has also made the learning process more enjoyable and led to increased motivation (Bakla, 2017; Zhang et al., 2006). At the same time, interactive video material has been shown to favor the development of higher-level cognitive skills (Wilson, 2016). In this regard, Multimedia Learning Environment Theory (Mayer, 2005a) plays an important role in ensuring learning through the effective use of cognitive loading.

The literature includes numerous studies comparing traditional and technological teaching materials (Clayton et al., 2010; Sole & Lindquist, 2001; Uygarer & Uzunboylu, 2017). The development of innovative technologies has been recommended as a means of closing the gap between distance and face-to-face learning (Guri-Rosenblit, 2019). It has been suggested that rather than comparing traditional and innovative teaching material, future research might more meaningfully focus on comparing different types of innovative technologies that can be used in distance learning. The literature stresses that there is a need for research that compares innovative technologies and examines learner openness to the use of new technology as factors that could affect learning in connection with the use of innovative technology in the learning process. Simultaneously, it is stated that individual engagement in learning processes affects learning performance (Blasco-Arcas et al., 2013; Qureshi et al., 2021), and that learning materials that are successful in participation might be a key factor. It is critical to provide contemporary learning materials that support individuals' behavioral, cognitive, and affective engagement. Accordingly, motivation and cognitive load are thought to be deciding factors in the development of materials that would encourage engagement. Hence, examining changes in material motivation and cognitive load as factors with the potential to affect learning could also be of interest in relation to innovative learning materials. In order to promote learners' engagement in distance education, multimedia materials that boost motivation and reduce extraneous cognitive load might be produced. In brief, the current study is believed to have the potential to address the gap in the relevant literature in terms of guiding to instructors, instructional designers, and learners. An examination of the measurement tools aimed at describing material motivation that are available in the literature shows that in general, these tools, have been used in long-term learning activities and teaching guides (Aydemir & Öztürk, 2013; Dede & Yaman, 2008; Dinçer & Doğanay, 2016; Erdem & Gözüküçük, 2013; Kara, 2008; Karataş et al., 2016; Kazu et al., 2016; Kutu & Sözbilir, 2011; Özgüler et al., 2017; Saricam et al., 2014; Yılmaz & Çavaş, 2007). However, there does not appear to be any appropriate measurement tool for measuring material motivation in relation to a situation in which technologically based education is utilized once at a distance and without any feedback.

Therefore, this study took the development of a new “Instructional Materials Motivation Scale” for single-use material as its starting point as well as one of its main goals. In consequence, it would be feasible to evaluate how materials based on single-use formation and that may be used without instructors’ direction impact learners' motivation, as opposed to materials utilized in a course and under instructor leadership. For this reason, this study aimed to evaluate animated and interactive videos used in distance education in terms of participant motivation, cognitive load, openness to different innovative materials, and willingness to recommend the material used. In order to fulfill the main goal of the current study, which is based on the quantitative research method, Research Questions (RQs) were first shaped as proposed in the literature (Williams et al., 2022). The following RQs are being examined in the current study:

• RQ1: What are the psychometric properties of the Instructional Material Motivation Scale for Single-Use (IMMS-SU)?

• RQ2: What is the effect of studying with interactive and animated video materials used in distance education on recommendation to materials, openness to different materials, extraneous cognitive load, and motivation?

• RQ3: Do participants' motivation and cognitive load levels change in distance education depending on (a) the kind of materials (i.e., interactive or animated), (b) the amount of time they spend studying with materials, and (c) their openness to various materials?

Method

In this study, the posttest control group design was used to investigate the relationships between the research topics (Fraenkel et al., 2012). The data-collection tool implemented during the study consisted of the cognitive load instrument by Kalyuga et al. (1998) and Instructional Material Motivation Scale for Single-Use (IMMS-SU) as well as survey questions designed to determine whether participants would recommend the material to friends and the degree to which participants were open to new materials.

Participants

Data was collected in three stages from different sets of participants. The first two stages consisted of the development of the IMMS-SU, which involved the development of the scale using different sample groups. The third stage of the study was conducted with participants who used the animated and interactive video materials as part of a learning activity in an attempt to answer the research questions. The first and second stages were conducted with 433 (n_Females = 170, 39.26%) and 1.221 (n_Females = 660, 54.10%) participants, respectively, and the third stage, which included implementation of the animated and interactive videos, was conducted with 933 participants (n_Females = 317, 34%). In total, 2587 individuals participated in the three stages of data collection.

Experimental conditions

In the current research, those who used interactive materials were in the experimental group, while those who used animated materials were in the control group. The goal of this research is to examine the effect of two contemporary materials in the context of motivation and cognitive load in an experimental setting, rather than comparing traditional and up-to-date methods between each other. In the institution where the research was performed, the text-based, animated, and interactive materials are currently employed in the online learning. Traditional learning materials, on the other hand, have been found to be less motivating by pupils (Saputri et al., 2018). It is also considered that combining contemporary learning materials with traditional techniques might help pupils stay motivated (Son & Simonian, 2016). Learners prefer contemporary (i.e., visual-based) learning materials over traditional (i.e., text-based) learning materials, according to the findings of experimental research (Nja et al., 2019; Sulaiman et al., 2017). In brief, rather than comparing traditional and contemporary materials, this study employed well-accepted technique recommended in the literature to compare animated and interactive materials. The multifaceted contents (i.e., text, audio, video, & animations) presented in interactive materials used in learning activities attract the attention of individuals. This scenario, it is accentuated, has a booster impact on learning motivation (Darmawan, 2012; Sanaky, 2013). Therefore, interactive materials are considered to boost motivation positively. Because the use of animation has a more traditional structure (i.e., become ordinary) and interactive materials have additional features that enable user-material interaction, those who use interactive materials were assigned to the experimental group, while those who use animated materials were assigned to the control group. Participants were randomly assigned to the experimental and control groups via the digital form, and the materials assigned in the digital form were also randomly assigned.

The research employed interactive and animated video materials developed for the same unit of the Introduction to Economics II course, which is a fourth-semester course offered through a state university's department of open and distance education. The experimental implementation was carried out with the participation of freshmen who have not yet experienced the Introduction to Economics II course. Thus, the experimental implementation was intended to eradicate any potential prior knowledge of the course among the participants. Within the scope of the experimental research, only motivation and cognitive load were examined, and no assessment for the effect of the applied materials on learning was conducted. Hence, the achievement test was omitted. The IMMS-SU and instruments for cognitive load measurement were utilized as post-test in this context.

The animated and interactive materials utilized in the research were all created for the exact same subject and learning objectives. Because the subject's difficulty level and content are the exact same, there is no differentiation of learning content based on the materials’ formation. Interactive and animated materials were approximately 5 min long with same qualifications (i.e., H.264 encoding; 1280 × 720 resolution; 30fps; 6000kbps bitrate). This research was based on voluntarily participation. The precise period of studying with the materials is unknown because the experimental implementation was carried out using a platform that did not keep any usage time logs for participants. The stages of the experimental procedure are shown in Fig. 1.

Fig. 1.

The research procedure

Materials

The effect of two materials (i.e., animated video and interactive video), were investigated in the experimental implementation. The topic matter of these materials was developed by field specialists and lecturers who taught the Introduction to Economics II course. The animation design and interactive elements utilized in the materials were created by Computer Education and Instructional Technology (CEIT) specialists and reviewed and approved by other experts in the same department. Only learners participating in the department of open and distance learning had access to the developed materials. Vyond—GoAnimate was used to produce animated video materials. The animated video material included moving and talking characters and moving objects in addition to text and audio-visual elements. On the other hand, Articulate Storyline and Adobe Captivate were used to create interactive video materials. Whereas the interactive video featured links, optional additional information, and question balloons in addition to text and audio-visual elements. Individuals using the interactive material were able to select additional information to complete the information that was missing and to benefit from certain other features, such as being directed to the field with the correct answer by pop-up balloons or accessing additional information when they gave an incorrect answer.

Data collection tools

During the study, differences in motivation and cognitive load were examined according to the different types of material used in terms of time spent viewing the material and openness to different material. In this context, a material motivation scale (i.e., IMMS-SU), an instrument to measure the cognitive load, and an online questionnaire form were used in this study.

To measure the cognitive load, the degree of learning difficulty presented by the multimedia material was measured according to cognitive load based on the response to the question, “How difficult was it for you to learn with the material?” Responses were graded using a 7-point Likert scale ranging from “very easy” to “very difficult” (Kalyuga et al., 1998). As stated by İlic and Akbulut (2019), the justification for preferability of this instrument is that its focuses on the material rather than the content (Cierniak et al., 2009). A score over the mean in the assessment of the mentioned one-item instrument, as in the relevant implementation instances in the literature, denotes the presence of extraneous cognitive load (Cheng et al., 2015; Cierniak et al., 2009; Hasler et al., 2007; Kalyuga et al., 1998; Marcus et al., 1996; Tindall-Ford et al., 1997). Single-item instruments were scrutinized as one of the recommended cognitive load measurement methods because do not hinder the learning task and are easy to use (Paas et al., 1994).

There is a gap in the literature regarding the measurement of motivation for single-use materials. In consequence, the material motivation of participants was evaluated according to the “Instructional Material Motivation Scale for Single-Use” (IMMS-SU) developed within the framework of the study.

The other items contained in the online data collecting tool are:

• “How much of the material did you watch?” It is structured in a 5-point Likert ranging from 1 (I never watched) to 5 (I watched completely).

• “I will recommend to my friends to study with this kind of material.” It is structured in a 5-point Likert ranging from 1 (completely disagree) to 5 (completely agree).

• “I would like to use it whenever I come across a material that's different from what we are used to.” It is structured in a 5-point Likert ranging from 1 (completely disagree) to 5 (completely agree).

• Demographics (i.e., gender, age).

Development of data collection tool

• RQ1. What are the psychometric properties of the IMMS-SU?

The development procedure of IMMS-SU was summarized in this section in order to answer RQ1. In developing the Instructional Material Motivation Scale (IMMS-SU), a pool of 40 items was comprised by examining Keller’s ARCS-V Motivation Model (Keller, 2016), Wlodkowski’s motivation model (Wlodkowski, 1985), and theories of intrinsic and extrinsic motivation found in the literature. The pool was reduced to 20 items by evaluating each item in terms of its appropriateness to the research aim and the existing institutional infrastructure. A 20-item draft form was developed and submitted to expert review, and the number of items was further reduced to 14. Accordingly, the form was reconstructed using a five-point Likert Scale ranging from “Completely Disagree” to “Completely Agree”. To facilitate correlatability of the 14-item draft data-collection tool with concrete learning experiences, it was presented immediately after a video on First Aid-Basic Life Support for Adults (Kızılay, 2018), an on-line learning activity for single-use implementation. The on-line data collection tool was implemented with participants who were attending open and distance education programs (n = 433; n_Female = 170, 39.26%). Prior to analysis, the data obtained was subjected to parametric testing, and within the framework of multivariate normality testing, extreme data (outliers) were excluded from analysis.

In developing the scale assumptions were examined in preparation for Exploratory Factor Analysis (EFA) of the subsample. Maximum Likelihood Estimation was performed in order to determine the minimum number of factors required to best reflect the relationship between variables (Aldrich, 1997; Field, 2009). Results of analysis and examination of the scree plot showed a single factor with an Eigenvalue greater than 1. EFA was repeated for a single-factor structure, and 14 items were identified that accounted for 81.49 percent of variance. Factor loading on the 14-item factor exceeded 0.85 (Pallant, 2001). Cronbach’s Alpha Reliability was calculated as 0.98. The single-factor, 14-item model obtained through EFA analysis was subjected to Confirmatory Factor Analysis (CFA) with the cross-validated subsample. After preliminary analysis of the items, the single-factor, 14-item scale model obtained through EFA was subjected to CFA analysis using the Maximum Likelihood method (χ² = 286.25; df = 74; χ²/df = 286.25/74 = 3.87; p < 0.001). The following values were obtained for independent model fit indices; SRMR = 0.01, CFI = 0.97, NFI = 0.96, GFI = 0.84 and AGFI = 0.77. Although all these values fell within the range of acceptability, RMSEA value of > 0.1 indicated that the scale did not have an acceptable level of consistency (Kelloway, 1998; Tabachnick & Fidell, 2007). The Cronbach-Alpha value for the internal consistency of the overall scale was calculated as 0.989, with total item correlations ranging between 0.78–0.92. In examining the scale’s goodness to fit conditions, the unacceptable intervals revealed by the RMSEA value indicated that the scale did not possess consistency and could not be confirmed. Inspection of the structural modelling and path coefficients performed as part of CFA showed Item 5 to interfere with the other two items of the scale. The scale was reviewed by three subject-matter experts and a measurement evaluation expert, and it was determined that Item 5 was not sufficiently differentiated in terms of meaning. The decision was made to resume the data-collection process after revising this item to make it sufficiently clear and differentiated. On the recommendation of experts, the “Instructional Material Motivation Scale” was revised by rewriting the item in question, and a second scale development procedure was commenced. The revised on-line data-collection tool was implemented with participants who were attending open and distance education, and the data set was prepared for analysis (n = 1221; n_Female = 660, 54.1%; mean_Age = 29.25 years; df = 8.99). In order to determine the factorial structure of the scale items CFA was conducted with the sub-sample (n = 610), after which EFA was conducted for cross-validation (n = 611). In order to conduct EFA with the sub-sample for scale development, assumptions were examined, and outlying values were discarded in order to secure multivariate normality (n = 594; n_Female = 333, 56.1%; mean_Age = 28.89 years; df = 8.64). The sample was found to be suitable for analysis based on dual correlation values that did not exceed 0.90, a meaningful outcome of Bartlett’s Sphericity Test (χ² = 6880.93; df = 91; p < 0.001), and a Kaiser–Meyer–Olkin value of 0.96 (Comrey & Lee, 1992; Pallant, 2001). Maximum Likelihood was used as the factoring method for EFA to determine the minimum number of factors that best reflect the relationship between variables (Aldrich, 1997; Field, 2009). Based on a Kaiser value of > 1 and using Catell’s screeplot graphic method, a single-factor structure with an Eigenvalue of > 1 and 14 items were identified (Fig. 2).

Fig. 2.

Scree plot of the IMMS-SU

Analysis was repeated for the single-factor structure and showed the 14-items accounted for 61.58 percent of variance. Examination of the loading of the 14-item factor shown in Table 1 shows a factor loading of > 0.63, indicating it meets the requirement of > 0.30 mentioned in the literature (Pallant, 2001). Cronbach’s Alpha coefficient of reliability was calculated as 0.96.

Table 1.

EFA results of the IMMS-SU

Items	$\overline{x}$	SD	Item-Total r	Factor Loading
1. I studied with this material easier than I expected	3.85	.978	.647	.639
2. I remember more easily what I have studied with this material	3.80	.954	.635	.626
3. There were features in this material that made me realize the crucial information	3.88	.916	.688	.689
4. I enjoyed studying with this material	3.78	.975	.784	.798
5. This material was intriguing	3.67	1.026	.773	.795
6. This material allowed me to focus on the topic	3.79	1.007	.832	.850
7. I believed that studying with this material would be beneficial to me	3.86	.994	.815	.834
8. This material has increased my belief that I can be successful	3.74	1.036	.792	.813
9. I prefer to study with such materials on complex topics	3.89	.990	.779	.798
10. This material contained interesting properties	3.66	1.006	.780	.810
11. This material increased my desire to study	3.69	1.010	.813	.842
12. I watched this material with pleasure to the end	3.75	1.033	.825	.852
13. I spent less effort studying with this material	3.82	1.034	.753	.778
14. It was exciting for me to study with this material	3.59	1.045	.788	.816
Total (α = 0.98)

The single-factor, 14-item model obtained from EFA was subjected to Confirmatory Factor Analysis for sub-sampling cross-validation. Prior to performing CFA, assumptions were examined, and multivariate outliers were excluded (n = 594; n_Female = 305, 51.3%; mean_Age = 29.65 years; df = 9.27). The split data sets were examined to determine whether they met the requirements for CFA analysis. According to the results of Bartlett’s Sphericity (χ² = 6292.07; df = 91; p < 0.001) and Kaiser–Meyer–Olkin (0.96) tests, the requirements for CFA were met (Comrey & Lee, 1992; Pallant, 2001). Following the preliminary analysis, the 14-item single-factor model previously obtained through EFA was subjected to CFA with Maximum Likelihood. According to the CFA results, goodness-to-fit index and RMSEA values indicated good fit for the data set (Table 2).

Table 2.

Model fit indices for measurement model

Fit criteria	Perfect Fita	Accepted Valuesb	Model Results	Rationale
Sample size		80 ≤ Item*5	594	Kass and Tinsley (1979); Tanaka et al., (1990)
x2/df	x2/df ≤ 3	3 < x2/df ≤ 5	332.59/74 = 4.49b	Kline (2014)
RMSEA	0 ≤ RMSEA ≤ .05	.05 < RMSEA ≤ .08	.077 b	Kelloway (1998); Tabachnick and Fidell (2007)
SRMR	SRMR ≤ .05	SRMR ≤ .10	.04a	Hu and Bentler (1999); Lacobucci (2010)
AGFI	.80 < AGFI ≤ 1.00	.70 ≤ AGFI ≤ .80	.88a	Byrne (1994); Qi et al., (2004)
NFI	.95 < NFI ≤ 1.00	.90 ≤ NFI ≤ 95	.95b	Tabachnick and Fidell (2007); Thompson (2004)
CFI	.95 < CFI ≤ 1.00	.90 ≤ CFI ≤ .95	.96a	Hu and Bentler (1999)
GFI	.95 < GFI ≤ 1.00	.80 ≤ GFI ≤ .95	.92b	Hair et al., (2009); Hooper et al., (2008)

The correlation coefficients and degree of variance explained by the items for each factor as determined by CFA conducted once the scale were finalized are shown in Fig. 3.

Fig. 3.

Structural model of the IMMS-SU and standardized path coefficients of the second CFA

The construct validity of the data-collection tools was tested by assessing convergent and divergent validity according to Fornell and Larcker (1981). Construct validity was determined by calculating Average Variance Extracted (AVE) values. Accordingly, the AVE value should be higher than the value of Composite Reliability (CR) for internal consistency, and each AVE value should be greater than 0.5 (Demir & Yurdugül, 2014; Hair et al., 2017). The IMMS-SU for single-use material was found to have an AVE value of 0.62 and a Composite Reliability value (ω) of 0.96, indicating that the scale and all of its dimensions met the requirements for construct validity. A Cronbach Alpha reliability coefficient of 0.95 was calculated for the entire single-factor, 14-item scale. According to the literature on scale development in the social sciences, a coefficient of reliability of 0.80 is a high measure indicative of reliability (Field, 2009; Vaske, 2008), and a Cronbach Alpha value of over 0.90 is a sign of a very high measure of consistency (Cohen et al., 2007). Correlation values for the individual items on the scale ranged between 0.63 and 0.81.

Results

In order to answer the research questions, the data-collection tool completed in the development phase was implemented with individuals in the open and distance education programs of a state university (n = 933; n_Female = 317, 34%; mean_Age = 32.28 years; df = 10.19). Parametric test conditions were controlled for.

• RQ2. What is the effect of studying with interactive and animated video materials used in distance education on recommendation to materials, openness to different materials, extraneous cognitive load, and motivation?

Descriptive analyzes were conducted in order to obtain the findings for RQ2 and sub-dimensions. Descriptive analysis of variables indicated that emotional and cognitive loads were below average for both those watching the animated videos ($\overline{x}$ = 3.22; df = 1.29) and those watching the interactive videos ($\overline{x}$ = 3.40; df = 1.25). Thus, neither of the materials prepared for distance learning caused an emotional or cognitive overload in participants. When the scores for whether or not participants would recommend the materials used in the learning activities were examined, both materials received above-average scores ($\overline{x}$_Animated = 3.47, df = 1.08; $\overline{x}$_Interactive = 3.44, df = 0.96), indicating that participants would recommend both the animated and interactive video material to their friends. Furthermore, when participant interest in using different materials was examined, above-average scores for both groups suggested participants were open to using different materials. In general, when the material motivation of participants was examined, it was seen that the material motivation was above average for both the animated video group ($\overline{x}$ = 3.35; df = 0.79) and the interactive video group ($\overline{x}$ = 3.26; df = 0.76). When the two groups are compared, the animated video group was found to have higher material motivation scores than the interactive video group.

• RQ3(a): Do participants' motivation and cognitive load levels change in distance education depending on the kind of materials (i.e., interactive, or animated)?

A t-test for independent groups was performed to identify any differences in participant motivation and cognitive load in connection with the different types of material used. No statistically significant difference was observed between the motivation of participants working with the animated video and those working with the interactive video (t₍₉₃₁₎ = 1.77; p > 0.05; d = 0.12; power = 0.55). When the cognitive load scores for both material types were examined, the interactive video group score was significantly higher than that of the animated video group score (t₍₉₃₁₎ = -2.15; p < 0.05; d = -0.14; power = 0.69), although a difference of ˂0.2 in Cohen d values indicated there to be only a small effect (Cohen, 1988).

• RQ3(b): Do participants' motivation and cognitive load levels change in distance education depending on the amount of time they spend studying with materials?

One-Way ANOVA for independent groups was used to determine if the amount of time spent viewing the material correlated with any differences in participant motivation or cognitive load. The study measured time spent viewing material using a 5-point Likert scale ranging from “1 – I didn’t watch at all” to “5 – I watched the entire video.” Individuals with a score of “1” indicating they did not watch the video at all were removed from the dataset, and the resulting analysis is presented in Table 3.

Table 3.

Comparison of the motivation and cognitive load levels of the participants with the time spent

			Sum of Squares	df	Mean Square	F	p	ηp2	Power
Animated Video Material	Motivation	Between Groups	35.88	3	11.96	21.90	.000**	.13	1
		Within Groups	235.41	431	.55
		Total	271.29	434
	Cognitive Load	Between Groups	14.58	3	4.86	2.96	.032*	.02	.70
		Within Groups	708.23	431	1.64
		Total	722.81	434
Interactive Video Material	Motivation	Between Groups	68.13	3	22.71	51.14	.000**	.24	1
		Within Groups	219.39	494	.44
		Total	287.51	497
	Cognitive Load	Between Groups	27.75	3	9.25	6.14	.000**	.04	.96
		Within Groups	743.73	494	1.51
		Total	771.48	497

^*(p < .05); **(p < .001)

Results of Tamhane tests were examined in order to determine whether or not there were significant differences in motivation according to the scores shown in Table 3 for time spent watching the animated and interactive videos. Accordingly, for those watching the animated video, no significant differences were found between the scores of “2” and “3” or between the scores of “4” and “5”; however, motivation was observed to increase significantly in connection with the amount of time spent viewing material for the remaining intervals between “2” and “5” (p < 0.001). Examination of the motivation scores in relation to the amount of time spent viewing the interactive material indicated that motivation increased significantly with the amount of time spent viewing material for all intervals between “2” and “5” with the exception of the interval between scores of “4” and “5” (p < 0.001). Results of Scheffe’s post-hoc comparisons were examined to identify any significant differences in cognitive load scores in connection with the amount of time viewing for participants who watched the animated and interactive videos. While no differences were observed with the animated video, for those watching the interactive video, cognitive load increased significantly (p < 0.05) with an increase in a score from “2” to “3” as well as with an increase in a score from “3” to “5” (p < 0.001) for the amount of time spent viewing.

• RQ3: Do participants' motivation and cognitive load levels change in distance education depending on their openness to various materials?

The openness of participants to new material was determined based on their response to the item, “When I come across material that is different from what I am used to, I want to use it.” Responses were scored using a Likert Scale ranging between “1” (completely disagree) and “5” (completely agree). One-way ANOVA was conducted to assess the differences between the participants' motivation and cognitive load levels according to their openness to the different materials (i.e., interactive, and animated). Table 4 summarizes the findings of the related analysis.

Table 4.

Motivation and cognitive load score comparisons according to the openness of different materials

			Sum of Squares	df	Mean Square	F	p	ηp2	Power
Animated Video Material	Motivation	Between Groups	101.21	4	25.30	63.97	.000**	.37	1
		Within Groups	170.07	430	.40
		Total	271.29	434
	Cognitive Load	Between Groups	86.24	4	21.56	14.56	.000**	.12	1
		Within Groups	636.57	430	1.48
		Total	722.81	434
Interactive Video Material	Motivation	Between Groups	99.32	4	24.83	65.04	.000**	.34	1
		Within Groups	188.20	493	.38
		Total	287.51	497
	Cognitive Load	Between Groups	21.58	4	5.40	3.55	.007**	.03	.87
		Within Groups	749.90	493	1.52
		Total	771.48	497

^*(p < .05); **(p < .001)

As Table 4 shows, significant differences were observed in the motivation and cognitive load scores of participants with different degrees of openness to new materials in both the interactive and animated video groups. In order to determine the specific differences between groups, the results of Tamhane comparisons were examined. Accordingly, with the exception of the differences between those scoring “2” and “3”, significant differences were found in the motivation of those with different scores for openness to new material in the group of participants who watched the animated video, with the motivation increasing in connection with the degree of openness to new materials (p < 0.001). In the group of participants who watched the animated video, cognitive load was significantly higher for those scoring “1” (p < 0.05) or “3” (p < 0.001) as compared to those scoring “4” or “5” on the degree of openness to new materials. In the group of participants who watched the interactive video, with the exception of the differences between those scoring “1” and “2”, significant differences were found in the motivation of those with different scores for openness to new material, with significant increases in motivation occurring in line with increases in openness to new material (p < 0.001). In the group of participants who watched the interactive video, a significant increase in cognitive load scores was also observed in connection with the increase in a score of “3” to “5” for the degree of openness to new material (p < 0.05).

Discussion

This study examined the motivation and cognitive load of participants in terms of the amount of time they spent viewing materials and their openness to different materials in connection with the interactive and animated video material frequently used in distance learning. Distance learning has become increasingly popular, and in conjunction with the Covid-19 pandemic, the demand for on-line instructional material has increased even further. Considering the current circumstances, the findings comparing the two types of materials in terms of motivation are extremely important. When we look beyond just the period of the pandemic and take into consideration expectations that in the future, individuals will gravitate towards self-accessed learning resources, efforts to develop an IMMS-SU that can be used by content developers and researchers in this area becomes increasingly important. According to the findings of this study, the use of animated and interactive material does not produce excessive cognitive loading in users. Moreover, users of both types of materials were found to have high levels of motivation. These findings may be explained by the innovative nature of these materials. In addition, the study showed that users of both animated and interactive material were open to the use of different materials and would recommend their use to others. These findings indicate that using innovative instructional materials in distance-learning environments could help support the learning process.

Motivation may be defined as the internal condition that secures and maintains an individual’s engagement in the learning process, and although cognitive theory has described the importance of motivation in connection with multi-media material, this phenomenon has not been sufficiently researched (Mayer, 2005a, p.171). Despite the lack of an agreed upon theoretical framework for the incorporation of motivational elements within instructional design, designers are able to incorporate a variety of characteristics into instructional material to increase user motivation and interest (Hartnett, 2016). Another aim of the present study was to examine the effects that the inclusion of attractive elements such as interactivity and animation in distance-learning material has on the level of motivation among participants. Taking into consideration the ARCS-V and Wlodkowski Motivational Models, an IMMS-SU for single-use materials was developed and implemented in order to examine the effects of the materials on the level of user motivation. Although the results did not show any statistically significant differences in participant motivation in connection with the type of material used, the material motivation of the animation group was found to be higher than that of the interactive group. For both the animated and interactive material, increases in view time and the level of openness to materials were observed to correlate with certain increases in motivation. With regard to the use of animated material in instruction, Plötzner and Lowe (2004) have stated that while animation is generally used to achieve an aesthetic and welcoming presentation, there is no agreement as to exactly how animation should be designed to facilitate learning. In line with the above, motivation may be affected by the design quality and watchability of materials. Thus, it may be suggested that improvements in participant motivation may stem from the ability of the moving visual elements found in animated video material to attract the attention of users. On the other hand, Pekrun and Stephens (2012) have stated that the affective states of learners also play a role in determining learning and success levels, which would explain from an affective standpoint why increases in motivation were observed in connection with openness to different materials. Accordingly, the idea of emotional engagement proposed by Fredicks et al. (2004) may be interpreted within this framework. It may be argued that being open to different materials and the use of attention-grabbing visuals support emotional engagement as well as motivation. Because emotional engagement is explained as a form of participation based on the learners' interest and motivation (Fredicks et al., 2004). According to the literature, the inclusion of decorative visuals pique learner interest, making it easier for learners to focus (Leutner, 2014, p.174). Moreover, the preparation of attractive materials and content in line with personalized learning approaches has been shown to support motivation (Alamri et al., 2020). It has also been noted that learning materials need to be presented so as to best meet the individual learning requirements of learners with different learning styles and facilitate learning (Hornbæk et al., 2002). Merely providing learners with information is not enough; rather, materials should feature interactivity in order to encourage deep learning (Amiryousefi & Geld, 2021; Bolkan et al., 2016). It has been suggested that interactive features activate the learner and support learning by attracting learner interest (Domagk et al., 2010; Fisch et al., 2014), and by securing a dynamic environment during the learning process, they create an individualized and motivated learning environment (Moreno & Mayer, 2007; Mudinillah, 2019; Petan et al., 2014). From this perspective, the observed increases in motivation can be attributed to the animated video material’s use of interesting visuals and the opportunities for individualized learning offered by the interactive learning materials. The assumption that these materials have positive effects on engagement may be backed by literature that motivation and participation have a positive correlated (Cazan, 2015). Preferred learning styles (Eysink et al., 2009) and learner motivation (Astleitner & Wiesner, 2004) have direct effects on the performance of learners using multimedia materials (Chen & Wu, 2015; Lauc et al., 2020). For this reason, instructional materials that increase motivation and instructional approaches that support learning should be used together during distance learning activities.

When cognitive load was examined in relation to the type of material used, the interactive material was found to create a significantly higher level of cognitive loading than the animated material. It is possible that some of the specific features of the interactive material, such as the “undo/redo” keys and text balloons containing supplementary information, made it more difficult for users to construct meaning from the content, and that the greater cognitive effort involved in interacting with the interactive video material produced a greater cognitive load in the user. It may also be suggested that the simultaneous appearance of the explanations and question balloons in conjunction with the text-heavy content of the interactive video is responsible for the cognitive load. This is in line with the dual-channel theory of Mayer and Moreno (2003), which states that loading on a single channel, combined with the limited capacity of working memory, produces a cognitive load. While learners can access supplementary information through the interactive video, the capacity of working memory is limited (Mayer & Johnson, 2008), and the selection and processing of important supplementary information included in the interactive video reduces the available amount of cognitive capacity, which may create an undesirable cognitive load. Interactive materials include both shapes and text. The simultaneous processing of text and sound requires greater cognitive effort, thereby increasing cognitive load, with this observable cognitive load stemming from the redundancy principle associated with multimedia material (Sweller, 2020). The Cognitive Theory of Multimedia Learning (Mayer, 2005a) also states that the processing of basic information featured in decorative visuals may exhaust cognitive resources and have a potentially negative effect on learning. The animated material would, according to Mayer’s theory, be expected to exhaust cognitive resources and produce a cognitive load because it features more decorative visuals than the interactive material. However, as Magner et al. (2013) point out, multimedia research has stressed that the inclusion of decorative visuals in a multimedia environment is not necessarily a detriment. In fact, the literature states that the attractiveness of decorative visuals make it easier for learners to focus and reduces the amount of cognitive effort expended during the learning process (Leutner, 2014, p. 174). Due to the form of expression employed by animations, which are mainly visual and contain only small amounts of highly focused text, such as itemized lists, they do not push the limits of the capacity of the visual channel (Mayer & Johnson, 2008), and, in line with the dual channel theory (Mayer & Moreno, 2003), do not create a heavy flow of information on one channel. Thus, it may be stated that animated designs, which are comprised mainly of visuals, do not produce a cognitive load because they contain attractive elements. Accordingly, while animated video materials encourage cognitive engagement because interactive materials cause cognitive load, it can be stated that “cognitive engagement” conceptualized by Fredicks et al. (2004) could be hard to realize in interactive video materials. The elements (e.g., attention-grabbing and focusing), on the other hand, are considered to support behavioral engagement in both materials. In contrast to this belief, Curum and Khedo (2020) have stated that the use of more complex instructional materials based on new technologies create a cognitive load. From this perspective, it may be suggested that the interactive tools found in interactive materials make them more complex and thus result in an excessive cognitive load. Whereas an increase in viewing time did not result in an increase in cognitive load among those who watched the animated material in this study, in certain instances, an increase in viewing time was observed to produce a significant increase in cognitive load for the viewers of the interactive material. In other words, with an increase in viewing time of the interactive material, as individuals continued to interact with the content, greater cognitive effort was required, and thus an increase in cognitive load was observed. Surprisingly, however, those scoring “3” for time spent viewing the interactive video had higher cognitive loads than those scoring “5”. It is possible that the concentration of content in the middle sections of the material exhausted the capacity of working memory. Considering that cognitive load may be reduced sometime after the loading of working memory, as stress is decreased and recovery occurs (Leahy & Sweller, 2019), it is possible that any cognitive load that had developed in individuals who watched until the end of the interactive material had diminished because they had already received supplementary information and completed the majority of the interpretation process. Conversely, those who scored “3” for the length of time watching the interactive video might have an elevated cognitive load, as they had reached the point in the middle of the interactive material where the provision of additional information, responses to question balloons and immersion in interactivity were concentrated. In addition, the additional information and question balloons found in the middle of the interactive video most likely meant that the visual channel, which is presumed to be limited (Plass et al., 2010; Sweller et al., 2011), was loaded with excessive information. Moreover, for the viewers of the animated as well as the interactive material, in some cases, as the level of openness to materials decreased, significant increases in cognitive load were observed. Accordingly, activities designed to increase participants’ openness to new materials could be beneficial in terms of preventing increases in cognitive loading.

Implications for further research and practice

While the outcomes of the current study help determine valuable clues regarding the impact of instructional materials, there are still various issues that need to be investigated. Thanks to the IMMS-SU developed within the scope of this research, it is possible to measure the motivation without the need for instructors' guidance and procedure protracted. In brief, the IMMS-SU may benefit multimedia learning researchers and practitioners who are indecisive in choosing materials. When the effect of animated and interactive materials used in the research on motivation is examined, it was determined that both encouraged high motivation. Similarly, it was observed that motivation was supported at some intervals as openness to the material and the duration of watching the material increased. In this respect, it is thought that individuals' level of openness to reformer materials and material duration is crucial to support learning motivation. Therefore, we propose deciding based on an individual's social traits and materials' duration levels in material preferences.

When the effect of the materials on cognitive load was investigated, it was revealed that the interactive material resulted in a higher cognitive load than the animated material. It is considered that the load in question may arise from the additional tasks created due to the nature of the interactive materials. In consequence, because it has been determined that the cognitive load rises as the duration of viewing interactive content increases, it is critical to control the duration of video materials so that they do not have a detrimental effect. In this case, we have highlighted the necessity to investigate and re-examine the types of cognitive load in further experimental research. Therefore, the effects of different types of materials on motivation and cognitive load is still a controversial issue that needs to be investigated in distance education. Whilst attracting the learners to contemporary learning environments, managing problems on the cognitive load, on the other hand, is a significant problem for academics and practitioners. Researchers and practitioners should thus be cautious to utilize instructional designs which encourage learning through the use of various materials. In addition, the impact of material design characteristics on learners should be investigated in online learning environments. Furthermore, we believe that it would be beneficial to the literature to create various materials that could promote various forms of engagement and to investigate the potential effects through further experimental research.

In order to achieve meaningful learning, it is important that multimedia instructional materials are designed to reduce the cognitive load of individuals (Mayer et al., 2004). Lauc et al. (2020) note that materials need to be formulated in such a way as to encourage active engagement in cognitive processes and increase motivational attachment. In the distance learning process, the use of animation with visuals designed to support motivation and interactive material that allows for individualized use can help to support instruction and facilitate learning. In order to avoid cognitive overload in connection with materials, it is recommended that interactive elements that facilitate use are integrated into the materials. It is also necessary to develop applications that support participant openness to different materials in order to increase motivation and prevent cognitive overload in connection with the materials used in distance learning. Learner openness to innovation may be encouraged through the development of different types of instructional materials with diverse content that make them more interesting and more attractive.

Limitations

This study was conducted only with students enrolled in open and distance education program. Since the study focused only on motivation and cognitive load, the effect of materials on learning (i.e., learning outcomes) was not examined. In this regard, the fact that learning is not addressed in the current study should be considered as a limitation or out-of-scope. Another limitation of the study is that the cognitive abilities of the participants were not measured before implementation. One of the primary reasons triggering to this limitation is the fact that the current research was conducted online. Therefore, it is recommended to examine learning outcomes and cognitive abilities in further studies on contemporary learning materials in distance education. The research is limited in that it relies only on the responses to a single data-collection instrument and the digitally implemented learning material used by participants. Future studies may be conducted with different digital materials as well as different platforms, such as social networks, to enable comparisons between different computer applications. The utilization of innovative technologies, including evaluation methods such as social network analysis and learning analytics, in the implementation of participant evaluations could also yield noteworthy results.

Appendix

Final Version of the IMMS-SU.

Dear Participant,

Please read the following statements carefully and indicate how they typically apply to you by choosing the appropriate box. Please answer every item and indicate your demographics.

Gender:	[  ] Female	[  ] Male
Age:	[  ]

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
1. I studied with this material easier than I expected.	□	□	□	□	□
2. I remember more easily what I have studied with this material.	□	□	□	□	□
3. There were features in this material that made me realize the crucial information.	□	□	□	□	□
4. I enjoyed studying with this material.	□	□	□	□	□
5. This material was intriguing.	□	□	□	□	□
6. This material allowed me to focus on the topic.	□	□	□	□	□
7. I believed that studying with this material would be beneficial to me.	□	□	□	□	□
8. This material has increased my belief that I can be successful.	□	□	□	□	□
9. I prefer to study with such materials on complex topics.	□	□	□	□	□
10. This material contained interesting properties.	□	□	□	□	□
11. This material increased my desire to study.	□	□	□	□	□
12. I watched this material with pleasure to the end.	□	□	□	□	□
13. I spent less effort studying with this material.	□	□	□	□	□
14. It was exciting for me to study with this material.	□	□	□	□	□

The Turkish version of the IMMS-SU and related supplementary materials are available upon request to the corresponding author.

Declarations

Conflict of interest

The authors do not have any conflicts of interests.