How do different cognitive styles Learners deal with the bullet screen interruption in instructional videos? An eye-tracking study

Abstract

Dense bullet screens at the top of the video screen may be disruptive. With the help of eye-tracking technology, this study explores the effects of bullet screen interruptions on instructional video learning from the perspective of different cognitive styles. Participants (N = 84) were required to complete a post-test after watching the video course “Human Body Temperature and Regulation” under a 2 (bullet-screen: bullet-screen, bullet-screen-free) × 2 (cognitive style: field-independent, field-dependent) between-subjects design. The study found that the bullet screen distracted both field-independent and field-dependent cognitive style learners’ attention and increased the cognitive load of field-dependent learners. As a result, the learning outcomes of field-dependent learners with bullet screens was significantly lower than those of learners with bullet-screen-free condition, while field-independent learners did not exhibit any significant differences. These results revealed that bullet screen has a negative impact on instructional video learning outcomes for field-dependent learners, causing cognitive overload and reducing their learning outcomes, but has no impact for field-independent learners. Compared to field-dependent learners, field-independent learners can better cope with the interruption of bullet-screens in instructional videos.

Introduction

The development of the internet and information technology has enabled online education to overcome limitations of time and location, making it crucial for individuals to acquire knowledge. Mainstream online education apps or e-platforms extensively utilize course videos as the primary form of learning resource [1, 2]. While this video-based multimedia learning provides a comprehensive and seamless learning experience, it lacks interactivity. Thus, online education apps have started to draw inspiration from online video platforms. They have incorporated a new video interactive technology called “bullet screen” into their learning videos [3].

Bullet-screen in instructional videos

Bullet screen (also known as “Danmaku” or “Danmu”) refers to viewer-edited commentary subtitles that appear on the video in real-time through scrolling, lingering, and even with various other action effects [4]. As the comments float across the screen, they resemble bullets in a flying shooting game [5]. Featuring immediacy, situationality, and diversity, bullet screen was first introduced to the public in 2006 by a Japanese video-sharing website Niconico.jp [6]. After entering China in 2007, it has been used by more and more online education platforms (such as Bilibili, Rain Classroom) to facilitate instant and interactive communication.

There are a lot of online video websites with bullet-screen function, such as YouTube and Bilibili website. Both platforms contain a large number of educational video resources. Bullet-screens can help learner to maintain motivation for continued learning [7]. By reading the bullet-screen content, learners can obtain additional information beyond the video content and understand the thoughts and attitudes of other learners “at this point in time”, as if they are learning together with other learners [8]. However, when watching bullet screen videos on these online video platforms, the bullet screen content will cover the entire screen interface, and its presentation can also be varied in both size and color [3]. This complex presentation form of bullet-screens may add to the cognitive load of the learner’s learning process [9]. Besides, the content of the bullet-screens may or may not be related to the content of the video. Bullet-screens related to video content can help the viewer to understand the learning content, and irrelevant bullet screen will prevent the viewer from integrating the video content [7, 10].

The effects of bullet-screen in instructional videos

According to the cognitive load theory, when cognitive resources required for information processing exceeds an individual’s working memory capacity, the individual’s cognitive load will become overloaded. Therefore, instructional design should aim to increase students’ learning effectiveness by reducing extraneous cognitive load and increasing germane cognitive load [11, 12]. Therefore, adding bullet screen comments in instructional videos changes the organization form of instructional design, and increases the extraneous cognitive load formed by learners processing bullet screen content compared to traditional video-based learning [13]. In addition, bullet screens not only create a cognitive resource load on the working memory capacity, but also consume more attention resources, and impair the allocation of attention resources, all of which may hinder learners from deep learning and reduce their academic performance [9, 14]. For example, a study explored the impact of bullet screens on learners’ attention and learning performance in the condition of self-control and system control videos. The results showed that bullet screens distracted learners’ attention and reduced their learning performance under both conditions [15]. Eye-tracking data analysis also indicated that more attention to bullet screens was associated with poorer learning retention. However, some studies have found that although bullet screens disperse learners’ attention, they do not have a negative impact on cognitive load or memory task performance [16].

The different results of cognitive and attentional load caused by bullet-screens on video learning may be due to individual differences. Yang et al. conducted a meta-analysis to investigate the effects of bullet screen on learning, examining its impact on both the learning process and learning outcomes. The meta-analysis showed that bullet screen had no significant effect on the learning process, but had a significant effect on learning outcomes, with individual learner variables being an important factor [17].

The moderating effect of cognitive style on the effects of bullet-screen

Cognitive style is a crucial individual variable, reflecting how learners represent and organize information. The same learning material may have different cognitive load effects on learners with different cognitive styles. Currently, there is limited research on the combined impact of cognitive styles and bullet screens on video-based learning, with many studies focusing on the combined impact of different learning styles (e.g. reflective vs. active) and bullet screens on video-based learning processes and outcomes. For example, Yi investigated the impact of bullet screen and learning styles (reflective vs. active) on video-based learning, using subjective cognitive load (sum of intrinsic and germane cognitive load), intrinsic cognitive load, germane cognitive load, retention test scores, and transfer test scores as dependent variables [18]. The results showed that for declarative knowledge learning, bullet screen only reduced the germane cognitive load and retention test scores of reflective learners, but had no significant effect on those of active learners. Duan used eye-tracking technology to explore the effects of bullet screen on video-based learning for learners with different learning styles (reflective vs. active). The results showed that active learners had better learning outcomes than reflective learners did, under positive, novel, video-content-related, and encouraging feedback-kind bullet screens, which yet disrupted the learning of reflective learners, lowering their test scores [7]. However, there are limitations in using individual variables of reflective and active learning styles to explore the effects of bullet screen on video-based learning.

Following the cognitive control model by Riding [19], cognitive style plays a central role in organizing information from internal states and external environments. Video-based learning demands the integration and organization of external information with internal experience [20], which entails the perception of both the learner’s internal mental states and the external learning environments. Reflective and active learning styles, which mainly classify learners based on reaction time and accuracy in problem solving, belong to different response strategy tendencies in the cognitive control model [21]. However, these classifications cannot fully reflect individual differences in learners’ perception tendencies towards bullet screen environments compared to the field-independent and field-dependent cognitive styles classified based on perception [22]. Field-independent learners rely on internal references, while field-dependent learners rely on external references [23]. Therefore, further research is necessary to examine whether there is an interaction between bullet screens and individual perception variances during the perception phase, and if this interaction affects video-based learning. This finding contributes to our understanding of the extent to which individuals with different attentional patterns are attuned to bullet screens in instructional videos, which has implications for the application of individualized educational practices.

Attention allocation patterns in learners with different cognitive styles

Previous studies have shown that there are significant differences in the eye movement patterns of people with field-independent and field-dependent cognitive styles during visual search tasks. To illustrate, in visual search tasks with embedded shapes and visual search tasks with hidden faces, the heat maps of the eye movements of field-independent learners demonstrate that they search for the target at a faster rate than field-dependent learners and with greater accuracy than field-dependent learners. Overall, field-dependent learners have a wider visual search range and their fixation points are widely distributed, covering all visual stimuli in the visual field, while the fixation points of field-independent learners are narrower in scope, basically oriented towards the search target, with relatively fixed eye movement trajectories [24]. The results of some physiological mechanism studies have shown that field-independent learners allocate more attention and working memory resources to perceptual tasks than field-dependent learners [25, 26]. A recent fMRI study has demonstrated the biological basis of the advantage in perceptual search in field-independent individuals, namely their enhanced connection among the left temporal pole, left anterior middle temporal gyrus/left posterior middle temporal gyrus and the vermis. These regions are involved in cognitive processes such as color perception, attention and working memory. The effective coordination between these regions provides the biological basis for the advantage in visual attention resource allocation in field-independent individuals [27].

The utilization of eye tracking technology as a potential measure of users’ cognitive ability during visual processing provides insight into the impact of information and cognitive overload on user perception and interaction within virtual learning environments [28, 29]. However, there is little research on how different cognitive styles learn from bullet-screen videos. There is also little discussion of the eye movement mechanisms involved. Recently, researcher begins to explore the impact of bullet screens on video-based learning for learners with different cognitive styles [10]. The bilibili website was used as a video-based learning platform, and the study investigated the variables of learning process (learning satisfaction, subjective cognitive load) and learning outcomes (retention test scores) for field-independent and field-dependent learners. The results showed that there were no significant differences in learning outcomes between field-independent and field-dependent learners. Nevertheless, this study has limitations. The study only looks at how field-independent and field-dependent learners acquire information from bullet-screen video. There was no control group for the condition without bullet-screen video, so we cannot fully understand the interaction between learning video with and without bullet-screen and different cognitive style. Besides, in this study, the number and content of the bullet-screen viewed by field-independent and field-dependent learners were inconsistent. This experiment did not control for these additional variables. It is also noteworthy that this study did not investigate the impact mechanism of bullet screens on the learning effects of learners with different cognitive styles.

The current study

In essence, how do learners with different cognitive styles cope with the hindrance of bullet-screens in videos? Do bullet-screens in videos impact the video learning outcomes of learners with different cognitive styles? These questions remain unanswered. The findings of the limited number of existing studies are also inconclusive. Thus, in order to answer the research question and to address the limitations in existing studies, this empirical study using a 2 (bullet screen: with or without bullet screens) × 2 (cognitive styles: field-dependent, field-independent) between-subjects true-experimental design will be conducted in a laboratory setting with eye-tracking technology to explore the effects on video-based learning.

The study proposes the following hypotheses:

Hypothesis 1

The bullet-screen interruption in the video will decrease the learning outcomes of field-dependent learners, but will not affect the learning outcomes of field-independent learners.

Hypothesis 2

The bullet-screen interruption in the video will increase the field-dependent learner’s eye movement indicators in the bullet screen area and decreases their eye movement indicators in the learning area, which ultimately affects their learning outcomes.

Materials and methods

Participants

Ninety students who were not majoring in physiology were recruited from a university in Wuhan, China. Six participants with pre-test scores greater than or equal to 20 and eye tracking sampling rates lower than 75% were deemed invalid and excluded. The remaining 84 participants (age: M = 21.18, SD = 1.95) were randomly assigned to either the bullet-screen or bullet-screen-free video learning groups based on their cognitive styles, i.e., the bullet-screen/field-independent group (22), the bullet-screen/field-dependent group (20), the bullet0-screen-free/field-independent group (22), and the bullet-screen-free/field-dependent group (20). The participants were all of Chinese nationality and spoke Chinese as their native language. There were no significant differences in prior knowledge between the groups (F (3, 80) = 0.34, p = 0.793). Furthermore, no notable discrepancies were observed in the age and gender distributions across the groups, F (3, 80) = 2.47, p = 0.068, χ² = 2.64, p = 0.451. All participants had normal or corrected visual acuity and no color weakness or blindness.

Design

The experiment was a 2 (bullet-screen: bullet-screen, bullet-screen-free) × 2 (cognitive style: field-dependent, field-independent) between-subjects experimental design. The dependent variables were video learning outcome (measured by retention test scores) and learning process (measured by subjective cognitive load and learning satisfaction).

Materials

Three scientific knowledge videos were selected, and 10 participants rated their difficulty on a 3-point scale (1–3 representing easy, moderate, difficult), with the video having a mean subjective rating of 2.00 being selected as the learning content. The correlation coefficient of rater consistency was 0.64 (p = 0.002). The chosen video was sourced from the MOOC website “https://www.icourse163.org/course/CSU-1001930016?from=searchPage, ” specifically from the course “Human Body Temperature and Regulation” in the National Excellence Course “Physiology” at Central South University. The video was 4 minutes and 44 seconds long, with a resolution of 1280*720 (720p) in MP4 format, and was made into a PPT handout video using Course Maker recording software.

Afterwards, video editing software was used to edit the video, adding 221 bullet screens in accordance with time process and distribution density similar to that of real bullet-screen environments. On average, one bullet screen was inserted every 2 s with consistent color, quantity, size, and position. The length was controlled between 6.68 and 13.20 words. The study included both learning-related (121) and learning-irrelevant (100) bullet screens, based on previous research [10, 18]. Learning-related bullet screens included video content recording (“Body temperature is the temperature inside the human body”), interpretation (“Heat must be produced and dissipated in balance”), questions and answers (“What does the central nervous system include? “), feedback on learning progress (“Knowing this regulation mechanism is important”), personal opinions (“I believe 37°C is considered a low-grade fever”), etc., while learning-irrelevant bullet screens included roasting (“Ha ha ha ha ha!“), communication and interaction (“Can anyone see my comment?“), personal revelations (“There are lots of comments”), etc. All bullet screens are displayed as two lines of text at the top of the screen. The video material with or without bullet screens had the exact same learning content. Eye-tracking measurements were also used throughout the study.

Measures

Pre-test questionnaire

The pre-test questionnaire consisted of demographic information, a survey on video-based learning, and a prior knowledge test measuring participants’ prior knowledge. The survey on video-based learning included whether participants had previously watched online instructional videos, the video learning platform, and the type of learning knowledge. The prior knowledge test had six questions, the first four testing their background of physiological knowledge, and the last two directly related to the learning content. The test consisted of two subjective ratings of prior knowledge, two single-choice questions, and two multiple-choice questions each worth 5 points, totaling 30 points, options can be missed but incorrect choices received zero points [7, 30].

Cognitive style questionnaire

Cognitive style was assessed using the Group Embedded Figures Test (GEFT) [31], which was translated and revised by Meng et al. from Beijing Normal University [32]. The test comprised three parts, with the first part consisting of nine complex figures used as practice and not included in the total score. The second and third parts had ten questions each, with varying levels of difficulty. Questions 1 and 2 in each part carried 0.5 points each, while questions 3 and 4 carried 1 point each, and questions 5 to 10 carried 1.5 points each. Blank and incorrectly identified figures received 0 points, with a maximum possible score of 24 points. T-scores were calculated using standard gender norms, where scores greater than 50 indicated a tendency toward a field-independent type, and scores lower than 50 indicated a tendency toward a field-dependent type.

Video-based learning measurement questionnaire

The measurement of video-based learning included assessing the learning process (using a learning satisfaction questionnaire and a subjective cognitive load questionnaire) as well as the learning outcome (using a learning retention test).

The learning satisfaction questionnaire comprised three scored questions using a 7-point Likert scale, with an internal consistency coefficient of 0.86.

The subjective cognitive load questionnaire utilized the cognitive load self-assessment scale created by Paas et al. [33]. The scale included two 9-point Likert-type questions. The first question evaluated the mental effort exerted by learners during the learning process, measuring the germane cognitive load. The second question assessed learners’ perceived difficulty of materials, measuring the intrinsic cognitive load. The combined score of both questions represented the total subjective cognitive load, with higher scores indicating greater cognitive load experienced by participants during the video learning process.

There were five questions in the learning retention test: one single-choice question and one judgment question, each worth 1 point; two fill-in-the-blank questions, each worth 3 points; and one short answer question worth 10 points. In total, the test was worth 18 points.

Experimental instrument

The experiment was conducted using an SMI RED 250 eye tracker (Senso Motoric Instruments, Teltow, Germany) with a sampling rate of 250 Hz and gaze accuracy of less than 0.4 degrees. Stimuli were presented on a 22-inch monitor (resolution 1680 × 1050, width: 473 mm, length: 296 mm) with a participant’s head fixed in place by a bracket at a visual distance of 70 cm. The horizontal and vertical visual angles of the stimuli were 31.9 degrees and 24.1 degrees, respectively.

Procedures

The experiment took place in a laboratory setting. The participants read an information letter explaining the study and gave their consent by participating. They completed a pre-test questionnaire and a cognitive style questionnaire based on which they were classified as field-dependent or field-independent learners They were then randomly assigned to either the bullet-screen group or the bullet-screen-free group, depending on their cognitive styles. Participants were informed that they would need to take a test after watching the video. Afterward, they all viewed the same length of video, completed a learning retention test and a questionnaire about subjective cognitive load and learning satisfaction. The experiment lasted around 30 min.

Results

Descriptive data (means and standard deviations) are provided in Table 1.

Table 1.

Mean and standard deviation of video-based learning of learners with different cognitive styles in bullet-screen or bullet-screen-free context

	Measurement index	Group	Field-independent			Field-dependent
			M	SD		M	SD
learning outcome	retention test	Bullet-screen-free group	7.34	2.18		9.35	3.05
		Bullet-screen group	8.25	2.76		5.70	3.44
Subjective cognitive load	Intrinsic cognitive load	Bullet-screen-free group	5.55	1.22		6.00	1.12
		Bullet-screen group	4.95	1.84		6.50	1.40
	Germane cognitive load	Bullet-screen-free group	6.73	1.35		7.30	1.30
		Bullet-screen group	7.14	1.17		6.70	1.22
Learning satisfaction		Bullet-screen-free group	14.82	3.03		15.70	4.54
		Bullet-screen group	15.73	2.47		16.20	4.63

Note: the performance of video-based learning includes three indexes, learning outcome, subjective cognitive load and learning satisfaction. And 84 participants (age: M = 21.18, SD = 1.95) were randomly assigned to either the bullet-screen or bullet-screen-free video learning groups based on their cognitive styles

Behavioral data on the effects of bullet screen in video-based learning

A two-way ANOVA was conducted to analysis learning process, with bullet screen and cognitive style as independent variables, and learning satisfaction and subjective cognitive load as dependent variables.

When learning satisfaction was used as the dependent variable, there was no main effect for bullet screen (F(1, 80) = 0.74, p = 0.392), no main effect for cognitive style (F(1, 80) = 0.69, p = 0.410), no bullet-screen × cognitive style interaction (F(1, 80) = 0.06, p = 0.803).

When subjective cognitive load was taken as the dependent variable, there was no main effect for bullet screen, F(1, 80) = 0.10, p = 0.757, and no bullet-screen × cognitive-style interaction, F(1, 80) = 0.01, p = 0.928. There was significant main effect for cognitive style, F(1, 80) = 5.53, p = 0.021, = 0.07, 95% CI = [0.16, 1.97], field-dependent learners perceived significantly higher subjective cognitive load than field-independent learners. Further analysis of intrinsic cognitive load and germane cognitive load as dependent variables revealed that only in the analysis of intrinsic cognitive load, the main effect of cognitive style was significant, F(1, 80) = 10.26, p = 0.002, = 0.11, 95% CI = [0.38, 1.62]. Field-dependent learners perceived significantly higher difficulty of the material than field-independent learners. There were no other group differences (both p’s > 0.050).

The learning outcome data met the conditions of covariance analysis. A two-factor analysis of covariance was conducted on learners’ retention test scores with prior knowledge scores as covariates, and presence or absence of bullet screens and cognitive style as independent variables. The analysis revealed a significant main effect for bullet screen on learning performance, F(1, 79) = 4.99, p = 0.028, = 0.06, 95% CI = [0.12, 2.62], with the bullet-screen group performing significantly lower than the bullet-screen-free group. The main effect of cognitive style was not significant, F(1, 79) = 0.11, p = 0.743, but the bullet-screen × cognitive style interaction was significant, F(1, 79) = 12.91, p = 0.001, = 0.14.

Follow-up simple effects analyses for the interaction revealed that in the bullet-screen-free condition, field-dependent learners had significantly higher retention test scores than field-independent learners, F(1, 79) = 7.64, p = 0.007, = 0.09, 95% CI = [0.78, 4.32], whereas in the bullet-screen context, field-dependent learners had significantly lower retention test scores than field-independent learners, F(1, 79) = 7.64, p = 0.007, = 0.09, 95% CI = [0.78, 4.32]. There was no significant difference in retention test scores between field-independent learners when watching the video with or without bullet screens. F(1, 79) = 0.97, p = 0.329. However, field-dependent learners’ retention test performance in the bullet-screen condition was significantly lower than that in the bullet-screen-free condition, F(1, 79) = 16.23, p < 0.001, = 0.17, 95% CI = [1.84, 5.46]. This pattern of results was displayed in Fig. 1.

Fig. 1.

Different effects of bullet screen on the performance of video-based learning for learners with different cognitive styles (N = 84). The vertical axis reflects retention test score, and the horizontal axis reflects different learning conditions. Significant differences are marked with an asterisk. Error bars depict the standard error of the mean

Eye-movement data of learners with different cognitive styles

The video for learning was divided into bullet-screen area and learning area, as shown in Fig. 2. The division of the region of interest (ROI) is based on the findings of previous research and the presentation interface of the Bilibili video website [10, 34]. The area covered by the bullet screen is the bullet screen area, and the area not covered by the bullet screen is the learning area. In order to examine the attention resource allocation patterns of people with different cognitive styles in the learning context of videos with/without bullet-screens. Total fixation time, average fixation time, fixation count, and average regression count were used as indicators to reflect the attention allocation and cognitive processing of learners in multimedia learning. The total fixation time and average fixation time are indicative of the cognitive resources expended by the learner in processing the multimedia learning materials, and thus serve as a measure of the cognitive load borne by the learner during the learning process. The average fixation time can be defined as the mean fixation duration of the gaze point within the region of interest. This value is equal to the total fixation time divided by the total number of fixations. A fixation is defined as a gaze point, and the fixation count represents the total number of gaze points. This not only indicates the learner’s ability to recognize multimedia information but also reflects their attention to the information and the allocation of cognitive resources. The return movement of the eye from right to left to the previously fixated content is referred to as regression. The average regression count is calculated by dividing the total regression time by the number of regressions. This measure not only reflects the allocation of attention by learners when learning multimedia but also indicates the cognitive coherence and the degree of reprocessing of information during learning [35]. The eye-tracking data of learners with different cognitive styles on the bullet-screen area and learning area were shown in Table 2.

Fig. 2.

The video learning screen in the upper left shows the division of the eye movement area of interest. The Red wire frame is the bullet screen area, and the blue wire frame is the learning content area. The eye movement areas of interest in the other three video learning screens are consistent with the upper-left corner. The four video learning interfaces present the learning content at different moments

Table 2.

Means and standard deviations of eye movement indicators for learners with different cognitive styles

		Measurement index	With or without bullet screens	Field-independent		Field-dependent
				M	SD	M	SD
Bullet screen area	Fixation indicators	Total fixation time (ms)	Bullet-screen-free group	4015	2858	3261	1127
			Bullet-screen group	15,164	9614	18,765	13,819
		Average fixation time (ms)	Bullet-screen-free group	229	61	210	45
			Bullet-screen group	227	50	226	26
		Fixation count	Bullet-screen-free group	19	15	16	6
			Bullet-screen group	70	47	81	58
	Saccade indicators	Average regression count	Bullet-screen-free group	11	8	10	4
			Bullet-screen group	28	18	29	20
Learning area	Fixation indicators	Total fixation time (ms)	Bullet-screen-free group	204,552	26,758	195,351	33,477
			Bullet-screen group	192,301	27,115	200,324	23,859
		Average fixation time (ms)	Bullet-screen-free group	279	56	248	42
			Bullet-screen group	279	59	348	126
		Fixation count	Bullet-screen-free group	746	74	790	87
			Bullet-screen group	709	129	627	171
	Saccade indicators	Average regression count	Bullet-screen-free group	54	31	64	24
			Bullet-screen group	67	35	72	48

Note: The video images were divided into bullet-screen area and learning area, and the fixation and saccade eye movement indicators of field-independent and field-dependent learners were collected respectively. The fixation indicators include total fixation time, average fixation time and fixation count, and the saccade indicators include average regression count

Eye-movement data in the bullet-screen area

A two-factor ANOVA was conducted with bullet screens and cognitive style as independent variables and the indicators of the bullet-screen area as dependent variables.

The total fixation time was used as the dependent variable. There was a significant main effect for bullet screen, F(1, 80) = 51.64, p < 0.001,= 0.39, 95% CI = [9636.4, 17018], indicating that learners spent more time on the bullet-screen area in the bullet-screen condition (M = 16964.71) compared to the bullet-screen-free condition (M = 3637.78). However, neither the main effect of cognitive style (F(1, 80) = 0.59, p = 0.445) nor the interaction between cognitive style and bullet screen (F(1, 80) = 1.38, p = 0.244) was significant.

The average fixation time in the bullet-screen area was taken as the dependent variable. There was no significant main effect for bullet screen, F(1, 80) = 0.47, p = 0.497, no significant main effect for cognitive style, F(1, 80) = 0.86, p = 0.357, and no significant bullet-screen × cognitive style interaction, F(1, 80) = 0.74, p = 0.394).

The fixation count in the bullet-screen area was used as the dependent variable. There was a main effect for bullet screen, F(1,80) = 48.82, p < 0.001, = 0.38, 95% CI = [41.64, 74.81], no main effect of cognitive style, F(1, 80) = 0.30, p = 0.584 and no significant bullet-screen × cognitive style interaction, F(1, 80) = 0.76, p = 0.385. Learners’ fixation count in the bullet-screen area was significantly higher in the bullet-screen condition (M = 75.47) compared to the bullet-screen-free condition (M = 17.25).

The average regression count was taken as the dependent variable. The analysis results revealed a significant main effect for bullet screen, F(1, 80) = 35.80, p < 0.001, = 0.31, 95% CI = [12.19, 24.33]. Learners’ average regression count to the bullet-screen area was significantly higher in the bullet-screen condition (M = 28.45) than the bullet-screen-free condition (M = 10.19). However, there was no main effect for cognitive style, F(1, 80) = 0.01, p = 0.930), and no significant bullet-screen × cognitive style interaction, F(1, 80) = 0.22, p = 0.638.

Eye-movement data in the learning area

A two-factor ANOVA was conducted with bullet screen and cognitive style as independent variables and the indicators of learning area as dependent variables.

The total fixation time in the learning area was used as the dependent variable. There was no main effect for bullet screen, F(1, 80) = 0.36, p = 0.553, there was no main effect for cognitive style, F(1, 80) = 0.01, p = 0.923, and no significant bullet-screen × cognitive style interaction, F(1, 80) = 1.99, p = 0.163.

The average fixation time in the learning area was taken as the dependent variable. The main effect for bullet screen was significant, F(1, 80) = 8.73, p = 0.004, = 0.10, 95% CI = [16.25, 83.35], there was no main effect of cognitive style, F(1, 80) = 1.28, p = 0.260, and there was a significant bullet-screen × cognitive style interaction, F(1, 80) = 8.84, p = 0.004, = 0.10.

Follow-up simple effects analyses for the interaction showed that in the bullet-screen condition, field-dependent learners had a significantly longer average fixation time to the learning area than field-independent learners, F(1, 80) = 8.43, p = 0.005, = 0.10, 95% CI = [21.77, 116.66], while in the bullet-screen-free condition, there was no difference in average fixation time to the learning area between the learners with different styles, F(1, 80) = 1.69, p = 0.197. Field-dependent learners in the bullet-screen condition had a significantly longer average fixation time on the learning area than those in the bullet-screen-free condition, F(1, 80) = 16.76, p < 0.001, = 0.17, 95% CI = [51.35, 148.47]. Field-independent learners showed no significant difference in average fixation time to the learning area between bullet-screen and bullet-screen-free conditions, F(1, 80) = 0.00, p = 0.989. The results were shown in Fig. 3.

Fig. 3.

Different effects of bullet screen on the average fixation time of learners with different styles in the learning area (N = 84). The vertical axis reflects the average fixation time, and the horizontal axis reflects different learning conditions. Significant differences are marked with an asterisk. Error bars depict the standard error of the mean. There was no significant interaction between total fixation time and average regression count in video learning area

When fixation count in the learning area was used as the dependent variable, the analysis revealed a significant main effect for bullet screen, F(1, 80) = 14.33, p < 0.001, = 0.15, 95% CI = [47.39, 152.41]. The main effect of cognitive style was not significant, F(1, 80) = 0.52, p = 0.474, but the interaction between bullet screens and cognitive style was significant, F(1, 80) = 5.59, p = 0.020, = 0.07. Simple effects for interaction indicated that in the bullet-screen condition, field-dependent learners had lower fixation count in the learning area than the field-independent learners F(1, 80) = 4.76, p = 0.032, = 0.06, 95% CI = [7.11, 155.63], and there was no significant difference between the learners with different styles in the bullet-screen-free condition, F(1, 80) = 1.35, p = 0.248. The field-dependent learners in the bullet-screen condition gazed at the learning area significantly less often than in the bullet-screen-free condition, F(1, 80) = 18.06, p < 0.001, = 0.18, 95% CI = [86.29, 238.31], and there was no significant difference for independent learners in the presence or absence of bullet screens, F(1,80) = 1.06, p = 0.306. The results were shown in Fig. 4.

Fig. 4.

Different effects of bullet screens on fixation count of learners with different styles in the learning area (N = 84). The vertical axis reflects fixation count, and the horizontal axis reflects different learning conditions. There was no significant interaction between total fixation time and average regression count in video learning area

The average regression count in the learning area was used as the dependent variable. There was no main effect for bullet screen, F(1, 80) = 1.77, p = 0.187, no main effect for cognitive style, F(1, 80) = 1.01, p = 0.318, and no significant bullet-screen × cognitive style interaction, F(1, 80) = 0.10, p = 0.758.

Discussion

The study examined the impacts of bullet screen and cognitive style on video-based learning using eye tracking technology. The findings of the study are presented in terms of learning satisfaction and video learning outcomes. The results demonstrate that, while the bullet screen does not affect the learning satisfaction of learners with different cognitive styles, it does have different effects on the video learning outcomes of learners with different cognitive styles. This is evident in the fact that the bullet screen does not affect the video learning outcomes of field-independent learners, but rather reduces the video learning outcomes of field-dependent learners. The analysis of eye-movement indicators suggests that the underlying mechanism of this effect may involve an increase in the cognitive load experienced by field-dependent learners. The combined results of both learning satisfaction and video learning outcomes indicate that the impact of bullet screen interruptions in instructional videos is moderated by the cognitive styles of video learners, with field-dependent learners demonstrating a reduction in learning outcomes due to these interruptions.

Bullet screen does not affect the learning satisfaction of learners

Bullet screen interruptions in instructional video have no effect on the learning satisfaction of field-independent and field-dependent learners, neither increasing nor decreasing it. This is inconsistent with the results of existing studies. For example, some quasi-experimental studies have found that bullet screen can increase learners’ learning satisfaction [10, 34]. This may be because, in this study, learners cannot control the video or send bullet screens spontaneously, and may not receive the learning feedback they deserve, so bullet screens may play a limited role in meaningful interactivity for video-based learning. Learning interactivity is a key component in both face-to-face classroom instruction and synchronous or asynchronous online teaching [36]. The activation and match satisfaction model suggests that field-dependent learners with the need for learning interactivity are activated by bullet-screen videos, but in this study, they could only passively watch bullet screens, which did not satisfy their need for interactivity to elicit a high level of learning satisfaction from the learning process [37, 8].

Our findings have extended this existing literature by demonstrating that interactivity relies on learner engagement, and only immersive engagement can achieve a high level of interaction. Simply incorporating bullet screens into an online learning environment without significant interaction may fail to produce learner satisfaction. Allowing learners to interact with the content, on the other hand, can lead to higher satisfaction [38]. For instance, learners can control the video’s progress, ask questions, and receive feedback while watching. Bullet screen interactivity depends heavily on learners’ engagement with the technology (human-computer interaction), and only immersive engagement fosters high levels of interaction. Psychological interaction may not be as effective as actual interaction in practice [39].

Bullet screen reduces video learning outcomes of field-dependent learners

At the aspect of video learning outcomes, the research results suggested that bullet screen has a negative impact on field-dependent learners, partially consistent with Pi et al.‘s findings [15]. Video-based learning combines audio, video, motion, static, texts, and pictures. Successful video-based learning requires learners to select and effectively integrate various forms of information [9, 40]. As a special instructional design, bullet screen transmits information through moving texts, demanding more attention selection or distribution from learners and making it difficult to acquire and integrate information. Poor selection and allocation of attention resources may impede information integration, leading to poorer learning outcomes. The findings also corroborate previous literature demonstrating that when the instructional design does not match learners’ cognitive style, it will also harm learning outcomes [11].

Individual variables like cognitive style also affect attention allocation and information integration. The results of the study confirm Hypothesis 1. This may be attributed to the following factors: On the one hand, field-dependent learners are more influenced by external cues such as bullet screens, making it harder to filter and integrate effective information from the complex bullet-screen environment [22, 35]. On the other hand, field-independent learners can use analytical abilities to reconstruct spontaneously cluttered information from the learning context without overloading their cognitive resources even though bullet screens also increases their extrinsic cognitive load, resulting in better learning outcomes [41, 42]. The findings support the cognitive load theory based on working memory capacity limitations and confirm the principles of optimal instructional design in multimedia learning based on cognitive load theory.

Bullet screen increases the cognitive load of field-dependent learners

In order to explore the reasons why field-independent and field-dependent learners have different video learning outcomes, the study reveals the cognitive load in video learning of the two cognitive styles in terms of subjective cognitive load and eye movement indicators. The study found no effect of bullet screens on the subjective cognitive load of learners with different cognitive styles. However, the bullet screen had an impact on the eye movement indicators of learners with different cognitive styles. These eye movement indicators reflect the allocation of attention resources by learners during video learning. The results of the eye movement indicators in bullet screen area show that when bullet screen appear at the top of the screen, both cognitive style learners allocate a greater proportion of their attention resources to the bullet screen. The results of the eye movement indicators in learning area show that when bullet screen appear at the top of the screen, it is more probable that field-dependent learners will allocate their attention to this screen, thereby increasing the cognitive load on these learners. This results in a reduction in the attention resources invested by field-dependent learners in the learning content, as evidenced by a decline in the fixation count and an increase in the average fixation time for the learning content. In conclusion, the objective eye-movement experiment showed that bullet screens distracted learners with both cognitive styles, and increased the cognitive load for field-dependent learners. The research results support Hypothesis 2. The results partially consistent with previous studies’ results [15].

Current research results support that when the sum of extraneous, intrinsic, and germane cognitive load exceeds limited capacity of working memory, cognitive overload will occur and inevitably have a negative impact on learning outcomes [12]. Because of no significant difference in the working memory capacities of learners with different cognitive styles [43], the three cognitive loads followed the “ebb and flow” principle. Thus, when field-dependent learners subjectively perceived high intrinsic cognitive load, resulting in reduced cognitive resources, extraneous cognitive load increased (e.g., bullet screens added in multimedia learning). Consequently, field-dependent learners increased their mental effort and cognitive load to ensure task completion. The total cognitive load may have exceeded their working memory capacity, leading to adverse learning outcomes.

According to the perspective of attention resources. Visual attention refers to how an individual orients and concentrates when perceiving external stimuli [44]. Information processing occurs in two stages: pre-attention, which is automatically influenced by external characteristics, and controlled attention, which involves conscious allocation of attention resources as a prerequisite for completing learning tasks. High consumption of attention resources correlates with a high cognitive load. Bullet screens scrolling at the top of the screen can attract constantly learners’ uncontrolled attention [6]. Simultaneously, due to the novel content in bullet screens and the text information related to the learning content, learners consciously allocate their attention to bullet screens or the learning content, the process of which occupies attention and cognitive resources. In addition, the shared attention model suggests that field-dependent learners relying on external references pay more attention to the bullet-screen area that gathers shared attention, and invest more cognitive resources in the bullet-screen content [14]. In summary, the designed instructional bullet screens created extraneous cognitive load for learners with both cognitive styles, while field-dependent learners had the potential to generate more extraneous cognitive load.

We have further contributed to existing knowledge in this area by demonstrating that bullet screens also increased the germane cognitive load for field-dependent learners in video-based learning. Eye-movement results in this study indicated that bullet screens increased the average fixation time of field-dependent learners and decreased their fixation count in the learning ar ea, which could reflect the efficiency of such learners’ visual search process [45]. When bullet screens appear in video-based learning, field-dependent learners whereas improved their efficiency in acquiring and processing learning information and made more mental effort, i.e., bullet screens increased the germane cognitive load of field-dependent learners, which is consistent with the findings of leng et al. [46]. It is worth noting that the results of the bullet screen area eye movement, while showing that the bullet screens distracted the attention of learners of both cognitive styles, the reason for this may also be related to the fact that the titles of the instructional videos in the study overlapped with the bullet screen area. This suggests that field-independent learners may be less likely to be affected by bullet screens, and that current exposure to bullet screens may simply be due to distraction from the video titles.

Research limitations and prospects

The study explored the impact of bullet screen interruption on video-based learning for different cognitive style learners using eye tracking. However, certain limitations were identified. Firstly, the study did not re-administer the Pre-test Questionnaire, the Cognitive Style Questionnaire, and the Learning Retention Test. Consequently, the test-retest reliability of these instruments cannot be clearly determined. It is imperative that future studies on learning effectiveness give full consideration to the accuracy of learning effectiveness measurement and the sustainability of learning effectiveness. Secondly, the study only examined the impact of bullet screens on declarative knowledge acquisition, whereas the cognitive processes involved in procedural knowledge acquisition, representation, and activation were not analyzed. Therefore, it is important to explore the effect of bullet screens on procedural knowledge acquisition in future studies. Secondly, categorizing the content of bullet screens (including the relevance to the learning content, its emotional attributes, the number and frequency of bullet screens sent) is necessary to explore the learning outcomes for learners with different cognitive style. Lastly, further research is needed to investigate how individual differences in cognitive processing, such as working memory capacity and metacognition, interact with bullet screens to affect video learning outcomes.

Despite its limitations, our study has significant practical implications for video-based learning and provides evidence for educators and individual learners to recognize the use of bullet screens based on different cognitive styles. Field-dependent learners should use the bullet screen feature moderately when learning alone, while educators may consider using bullet screens in natural classrooms to increase the novelty of instructional forms. To maintain a moderate level of mental effort, teaching materials with bullet screens should avoid presenting redundant information, focus on attention rather than novelty, and organize knowledge without excessively increasing the difficulty and quantity of knowledge points. This approach would allow learners to maintain a moderate level of mental effort.

Conclusion

There is a strong need to explore the effects and mechanism of bullet screens on video-based learning from different learning styles perspective. The current study has revealed that bullet screens distract the attention of learners with both cognitive styles, increase the cognitive load of field-dependent learners, and decrease the video learning outcomes of field-dependent learner. Bullet screens do not affect the video learning outcomes of field-independent learner, and have no effect on the learning satisfaction of learners with both cognitive styles. Thus, we have provided further evidence that the effect of disruptive bullet-screens in instructional videos on video learning is moderated by individual differences in cognitive style.

Abbreviations

fMRI

Functional magnetic resonance imaging

MOOC

Massive open online course

PPT

Powerpoint

GEFT

Group embedded figures test

SMI

Senso motoric instruments

ANOVA

Analysis of variance

ROI

Region of interest

CI

Confidence interval

Author contributions

Conceptualization: Mingxuan Zou, Defang Mu, Yinghe Chen; Data acquisition: Mingxuan Zou, Defang Mu; Data analysis: Mingxuan Zou, Defang Mu, Yinghe Chen; Data interpretation: Defang Mu, Yinghe Chen; Drafting: Mingxuan Zou, Defang Mu, Yinghe Chen; Revision: Defang Mu, Yinghe Chen.

Funding

This study was funded by the MOE Project of Key Research Institutes of Humanities and Social Science at Universities (22JJD190001).

Data availability

The datasets used in this study are not publicly available but are available from the first author.

Declarations

Ethics approval and consent to participate

Informed consent was obtained from all participants. All procedures in this study were in accordance with the Declaration of Helsinki and the study protocol was approved by Institutional Review Board (IRB) of Tianjin University of Commerce (approval number: IRB-2023-001).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.