The difference in views of educators and students on Forced Online Distance Education can lead to unintentional side effects

Abstract

The aim of the study was to investigate the different views of educators and students on Forced Online Distance Education during the Corona virus disease 2019 (COVID-19) lock-down. Such differences in views can be a source of misunderstanding, leading to unintended side effects. Online open-ended surveys resulted in 1341 comments received from 210 university educators and 347 students. The coding, based on the principles of Grounded Theory, resulted in 35 concepts, organized into 6 categories. The main findings were that students and educators shared most of the negative and positive views; however, there exist unique views that are not shared between the two groups. The negative views outweigh the positive ones, and educators are more negative than students. The category 'Perceived usefulness' is the most positive and 'Technology' the most negative category. Positive views were attributed to the quality of life, not the quality of the study. The most important contribution of the work to the existing body of knowledge is the comparative analysis of the unconstrained views of students and their educators about Online Learning Environments (OLE) as the workhorse of Forced Online Distance Education (FODE). The results of this study can be helpful for institutional evaluators, since they reveal undesirable side effects that are usually overlooked. The study brings a new, deeper look at Forced Online Distance Education and the non-neutral role of digital technology in it.

Introduction

The year 2020 was an exceptional year for higher education, owing to the outbreak of Corona virus disease 2019 (COVID-19) (Witze, 2020). The response of higher education institutions was far from uniform and ranged from no intervention to closures. In extreme cases, entire educational and even research activities at traditional universities were suspended and transferred online (Crawford et al., 2020). The disadvantage of such an approach, often due to lack of experience and resources, was that the response was not aligned with best practices and principles for online teaching (Crawford et al., 2020) and followed the approach best described as Emergency Remote Teaching (Bozkurt & Sharma, 2020), Emergency Forced Remote Education (Afip et al., 2020) and Forced Online Distance Education (FODE) in its online form. The term FODE combines the well-defined and elaborated term Online Distance Education (e.g., Maguire, 2005) as a combination of content and pedagogy in an online environment with the mostly unwanted pressure to go online. In the words of Bozkurt and Sharma (2020, p. ii) "Online distance education involves more than simply uploading educational content, rather, it is a learning process that provides learners agency, responsibility, flexibility and choice. It is a complex process that requires careful planning, designing and determination of aims to create an effective learning ecology." In the case of COVID -19 lockdown, the transfer of lessons and courses was not carefully planned and organized, and teachers were forced to transfer online what they had on hand with their knowledge and skills to save a study year. FODE puts the concepts of online education on a new foundation and does not characterize students and their educators equally. Because it has not been tested before, FODE can unintentionally generate side effects in addition to those previously identified in online education (Howell, 2001; Mseleku, 2020).

According to the Sternberg theory of Successful Intelligence (2005), educators had the option to (1) adapt to the new environment; (2) adapt the new environment; or (3) change the new environment (using various synchronous and asynchronous online distance learning strategies or a combination of both). On the other hand, students' options were limited to adapting to the new educational environment or to discontinuing the course(s). Therefore, all those involved in studying or lecturing under lock-down conditions were forced to develop coping strategies to address the emerging and novel problems. These coping strategies reflect various personal, cultural and environmental factors; however, it is too early to generalize. Among the published studies on students' coping strategies, Kamaludin et al. (2020) report that students in Malaysia "used maladaptive coping strategies (acceptance and mental disengagement) more than adaptive coping strategies (humanitarian and seeking social support)", and Rogowska et al. (2020) reported that "university students experience extremely high stress and anxiety during quarantine period". Various studies explored students' and/or educators’ attitudes or views towards online education (Dilmaç, 2020; Flores & Gago, 2020; Hebebci et al., 2020; Sepulveda-Escobar & Morrison, 2020), but we found no serious studies comparing the open-ended opinions of students and educators towards online education.

Since students and their educators have been forced into FODE regardless of their abilities and the condition of their equipment, the authors of this study seek to gain first-hand, unbiased insight into the obstacles and benefits of FODE as perceived by students and their educators. Identifying similarities and differences in their views can help to synchronize their expectations and practices with FODE. The research is important because FODE not only seems to be a lifeboat for universities during a COVID crisis, but has the potential to become a new standard, even in situations where the outcomes of such education can be harmful or without measurable benefit. Paraphrasing Martorella, who recognized the technology as a sleeping giant as early as 1997, we can recognize that the FODE was another such a sleeping giant, woken without knowledge of what side effects and damage (besides the benefits) it could cause.

Both the advantages of and the obstacles to distance education have been well researched both in traditional online environments and in hybrid forms of both (e.g. Al-Samarraie et al., 2018; Boelens et al., 2017; Kebritchi et al., 2017; Li & Irby, 2008; Moore et al., 2011; Pearcy, 2014). From a technical point of view, prior to the lock-down, everything that was necessary for the introduction of online distance learning courses was already available, at least on the market. However, there was also a major conceptual difference in the degree to which enrolment was voluntary. Many, if not all, the assumptions of connectivism (Downes, 2010; Goldie, 2016; Siemens, 2005), such as academic autonomy, choice of learning platforms, use of a variety of communication channels and the like, were overridden, and "brick and mortar" educators and their students were put online whether they wished or not, creating new tensions due to new interactions unfamiliar to many. Educator-student interactions in computer-mediated online education were rarely studied, even before the suspension. Nevertheless, all existing studies on interaction report that experiences vary and are strongly influenced by the interactivity of the courses (e.g., Gray, 2016; Pearcy, 2014).

Following the aims of the study, the intellectual framework of our work is mostly derived from a family of general theories (Table 1).

Table 1.

Overview of the theories explored in a search for organizing categories

Theory	Brief description	Constructs	Author(s)
Theory of planned behavior	The theory that assumes that behavioral intentions are the most proximal determinant of human social behavior	Attitude toward the behavior, Subjective norm, Perceived behavioral control, Intention, Behavior	Ajzen, 1991
Theory of self-determination	Theory of human motivation and personality that deals with the inherent growth tendencies and innate psychological needs of humans	Motivation, Self-regulation, Well-being	Ryan & Deci, 2000
Theory of self-efficacy	Theory of how psychological procedures whatever their form, alter the level and strength of self-efficacy	Self-efficacy	Bandura, 1997
Innovation Diffusion Theory	Model that explains how, why, and at what rate new ideas and technologies spread	Relative advantage, Compatibility, Complexity, Trialability, Observability, Adoption	Rogers, 1995
Technology Acceptance Model (TAM)	Information system theory that predicts (models) user acceptance and usage of technology	External variables, Perceived usefulness, Perceived ease of use, Attitude toward using, Behavioral intention to use, Actual system use	Davis, 1989; Davis et al., 1992
Unified theory of acceptance and use of technology (UTAUT)	Information system theory that predicts (models) user intention to use of information systems	Performance expectancy, Effort expectancy, Social influence, Facilitating conditions, Behavioral intention, Use behavior	Venkatesh & Davis, 2000; Venkatesh et al., 2003
General Extended Technology Acceptance Model for e-learning (GETAMEL)	Information system theory that predicts (models) user intention and actual usage of e-learning	Self-efficacy, Subjective norm, Perceived enjoyment, Computer anxiety, Experience, Perceived usefulness, Perceived ease of use, Attitude, Intention to use, Actual use	Abdullah & Ward, 2016
Information systems Continuance Intention Theory	Information system theory that predicts (models) and explain user intention to continue using ICT	Confirmation, Satisfaction, Perceived usefulness, Information system continuance intention	Bhattacherjee, 2001

To summarize these theories: Behavioral intention precedes actual behavior, and satisfaction determines continuance. One of the basic assumptions behind these theories, namely the voluntary nature of acceptance and continued use of technology, was violated because, in our case, students and educators were forced to go online. Knowledge about existence of theoretically well supported constructs (Table 1) was used to establish a framework of Preliminary Organizing Categories (See Table 2). To investigate this forced migration, we used Grounded Theory (Charmaz, 2014) as a method to find constructs that could be used to explain possible discrepancies in the views of students and educators about FODE.

Table 2.

Preliminary Organizing Categories (POCs)

POCs	Description of the category	References
‘Personal’	The category 'Personal' includes concepts related to the individual and his or her experience with and perception of teaching in online learning environments	Bandura, 1997; Ryan & Deci, 2000
‘Social interaction and pedagogy’	The category ‘Social interaction and pedagogy’ is described as a multitude of relationships and interactions between individuals in the online learning environments (i.e. educators and students)	Ajzen, 1991
‘Technology’	The category ‘Technology’ refers to the technological aspect of online learning environments	Rogers, 1995; Davis, 1989; Davis et al., 1992; Venkatesh & Davis, 2000; Venkatesh et al., 2003; Abdullah & Ward, 2016
‘Organizational’	The category ‘Organizational’ deals with work organization on all levels of online learning environments	Igbaria et al., 1996; Igbaria et al., 1997; Thompson et al., 1991
‘Perceived usefulness’	The category ‘Perceived usefulness’ describes the conditions under which respondents perceived the usefulness of online learning environments	Liao et al., 2009; Davis, 1989;
‘Undecided’	The category ‘Undecided’ represents responses that were blank (no opinion) or not decided (neither-neither)

The role of personal digital technologies in distributed environments, as established by FODE, should not be neglected, because without the massive use of information and communication technology, at least all the synchronous distance learning forms and many of the asynchronous ones will not be possible either. Therefore, information and communication technology can by no means be regarded as neutral (Zhao et al., 2004) and affects students and university staff differently. Due to differences in the quality of personal hardware, speed and stability of internet connections, digital skills, home environment, ethical concerns, and others, sending students and educators home was not only a solution to an epidemiological problem, but a potential source of inequalities (Czerepaniak-Walczak, 2020) that do not characterize students and educators equaly. User experiences (Hassenzahl, & Tractinsky, 2006) with digital technology and, for some, unfamiliar applications in computer-mediated communication may differ between students and their educators, an aspect that was expected from the study's findings but is still under-researched.

Aims and scope

The primary objective was to identify the problems and benefits as perceived by students and educators in relation to FODE. The impetus for the study was not only academic curiosity but also the provision of suggestions for the academic community to improve the online form of education, which was a novelty for many educators and their students.

We were interested in the response of students and educators to the novel situation of FODE, in order to optimize the teaching and learning experience. The nature of the study was exploratory. The students and their educators were asked to answer two open-ended questions. Thus, the research questions were:

• What problems were detected by students and their educators in using and teaching¹ (learning²) with online learning environments (OLE)?

• What benefits were detected by students and their educators in using and teaching¹ (learning²) with OLE?

• What are the main differences between students and their educators in using and teaching¹ (learning²) with OLE?

^1,2 The difference between educators and students was in the use of the teaching–learning pair. The educators were asked about teaching and the students about learning.

Based on research question the direction of outcomes was not guessed. The hypotheses and its alternative null hypotheses (not listed) are:

• H₁: There exist differences in quantity and severity of the problems detected by students and their educators in using and teaching¹ (learning²) with online learning environments.

• H₂: There exist differences in quantity and quality of benefits detected by students and their educators in using and teaching¹ (learning²) with OLE.

Methodology

The course of the study can be divided into several stages, as follows:

1. Review of the literature

   1. definition of queries based on research questions and hypothesis
   2. screening of the databases Web of Science, Scopus and Google Scholar
   3. filtering references by predefined keywords
   4. creating intellectual framework and Preliminary Organizing Categories derived from a family of general theories (Table 1)
2. Data collection

   1. the definition and analysis of the environment of the study (see Online learning environment at the University of Maribor)
   2. designing inventories to investigate the use of and attitudes towards online learning environments among educators and students (see Description of instruments).
   3. collection of responses from 347 students and 210 university educators on two open-ended questions (see Participants and collection of responses)
3. Raw data clarification and organization (see Coding and data structure)

   1. elimination of missing data and meaningless comments in both response fields
   2. separation of multiple comments to a single response
4. Coding and categorization (see Coding and data structure)

   1. categorization of responses by their content and also according to the strength of the response
   2. categorization of responses into Preliminary Organizing Categories (POCs).
   3. classification of POCs into 35 final concepts
5. Data analysis and interpretation (see Data analysis)

   1. descriptive data analysis, Chi square and Cohen's h (see Table 3 and Table 4)
   2. comparative analysis between students and educators (see Table 4)
   3. calculation of level of agreement between students and educators (see Fig. 2)

Table 3.

Frequency of responses attributed to one of the six organizational categories as expressed by educators and students and differences presented as effect sizes between them

Organizational Category	NES (%)	NE (%)	NS (%)	Cohen's h	CI
'Organizational'	293 (21.8)	67 (22.8)	226 (77.2)	0.42 (S)	0.31—0.53
'Social interaction and pedagogy'	288 (21.5)	143 (49.6)	145 (50.4)	0.20 (E)	0.09—0.31
'Undecided'	237 (17.7)	130 (54.8)	107 (45.2)	0.19 (E)	0.08—0.30
'Personal'	125 (9.3)	42 (33.6)	83 (66.4)	0.09 (S)	0.01—0.20
'Perceived usefulness'	143 (10.7)	55 (38.5)	88 (61.5)	0.03 (S)	0.01—0.13
'Technology'	254 (18.9)	105 (19.4)	149 (18.6)	0.02 (E)	0.01—0.13

E Educators; S Students; N_E 542; N_S 799; N_ES 1341; CI Confidence intervals. A table is ranked by decreasing effect size (Cohen's h)

Table 4.

Frequency of all the concepts as expressed by educators and students

Category	Concept	NES (%)	NE (%)	NS (%)	U
'Organizational'	Work organization: personal assets	101 (7.5)	10 (1.8)	91 (11.4)	ES
	Work organization: pedagogy	84 (6.3)	19 (3.5)	65 (8.1)	ES
	Work/study from home	47 (3.5)	13 (2.4)	34 (4.3)	ES
	Work organization: learning	25 (1.9)	0 (0)	25 (3.1)	S
	Organizational support	19 (1.4)	16 (3.0)	3 (0.4)	ES
	Implementation Knowledge Assessment/exams	14 (1.0)	6 (1.1)	8 (1.0)	ES
	Work organization: administration	3 (0.2)	3 (0.6)	0 (0)	E
'Social interaction and pedagogy'	Interaction	95 (7.1)	61 (11.3)	34 (4.3)	ES
	Communication	43 (3.2)	23 (4.2)	20 (2.5)	ES
	Collaboration	40 (3.0)	27 (5.0)	13 (1.6)	ES
	Interest, understanding, adaptation of educators	32 (2.4)	0 (0)	32 (4.0)	S
	Attendance	30 (2.2)	18 (3.3)	12 (1.5)	ES
	Lack of educators’ skills	29 (2.2)	0 (0)	29 (3.6)	S
	Knowledge/understanding	12 (0.9)	7 (1.3)	5 (0.6)	ES
	Students’ satisfaction	7 (0.5)	7 (1.3)	0 (0)	E
'Technology'	Web connection	120 (8.9)	28 (5.2)	92 (11.5)	ES
	Software	66 (4.9)	35 (6.5)	31 (3.9)	ES
	Hardware	64 (4.8)	38 (7.0)	26 (3.3)	ES
	Learning materials	2 (0.1)	2 (0.4)	0 (0)	E
	Copyright	1 (0.1)	1 (0.2)	0 (0)	E
	Technology impact	1 (0.1)	1 (0.2)	0 (0)	E
'Perceived usefulness'	… in crisis situations	54 (4.0)	36 (6.6)	18 (2.3)	ES
	… for certain study resources	50 (3.7)	0 (0)	50 (6.3)	S
	… for students	25 (1.9)	10 (1.8)	15 (1.9)	ES
	… for certain types of studies	14 (1.0)	9 (1.7)	5 (0.6)	ES
'Personal'	Previous experiences	33 (2.5)	22 (4.1)	11 (1.4)	ES
	Motivation	22 (1.5)	2 (0.4)	20 (2.5)	ES
	Health	21 (1.6)	2 (0.4)	19 (2.4)	ES
	Concentration	21 (1.6)	0 (0)	21 (2.6)	S
	Personal satisfaction	11 (0.8)	2 (0.4)	9 (1.1)	ES
	Work input	9 (0.8)	9 (1.7)	0 (0)	E
	Privacy	6 (0.4)	5 (0.9)	1 (0.1)	ES
	Own lack of knowledge	2 (0.1)	0 (0)	2 (0.3)	S
'Undecided'	No opinion	129 (9.6)	62 (11.4)	67 (8.4)	ES
	Neither-Neither	108 (8.1)	68 (12.5)	40 (5.0)	ES

E Educators; S Students; N_E 542; N_S 799; N_ES 1341; U Uniqueness of responses. The table is ranked by decreasing number of responses attributed to each concept within each category

Fig. 2.

Comparing educators and students (excluded Concepts with N<10)

Online learning environment at the University of Maribor

The University of Maribor, Slovenia, established a central organizational unit—the Computer Center of the University of Maribor (CCUM)—to ensure the uniform and rapid development and use of information technology (IT). Within the CCUM, the following important events relevant to this paper have occurred over time:

• the central installation of the Moodle learning environment named eUM in 2007;

• the establishment of the Dell DEMO Center for Identity and Access Management in 2013 and the introduction of a unified digital university identity—for staff and students;

• redesigning a secure portal.it.um.si in 2015, where employees and students can use their digital identity to access relevant software.

Distance education at the University of Maribor relies heavily on a learning environment eUM, which is based on the learning management system Moodle, Microsoft Office 365 Apps and MS Teams as a tool for video conferencing. The listed software is accessible free of cost to all students and staff with the university's digital identity. Most faculties also offer free access to course-specific software that is only available to their students. The decision of the university management a few days before the closure was that the learning environment eUM and MS Teams with Office 365 Apps will be used exclusively in synchronous and asynchronous courses as they are maintained with full licensing options by the university IT who also provide online manuals and support. Therefore, all responses can be considered as a reflection on working with these two systems. The programs used for research and supporting software for the devices, such as statistical packages, programming tools, imaging and course-specific software, even if used by students in some courses, are outside the scope of this research.

Description of the instruments

The open-ended questions examined in the recent study were part of larger inventories to examine the use of and attitudes towards online learning environments among educators and students (Ploj Virtič et al., 2021). The two open-ended questions were included in both survey instruments:

• Please tell us where you have found most problems in using and teaching¹ (learning²) with online learning environments (OLE).

• Please tell us where you have seen the benefits of using and teaching¹ (learning²) with online learning environments (OLE).

^1,2 The difference between educators and students was in the use of the teaching–learning pair. Educators were asked about teaching and students about learning.

In online forms, space was reserved for comments and no limiters (e.g. number of characters) were applied.

Participants and collection of responses

According to the 2019 census, the research population consisted of approximately 13,300 students and 900 university employees with teaching positions, including professors, lecturers and teaching assistants from the University of Maribor, Slovenia (hereinafter referred to as educators).

We conducted two separate studies: one among Slovenian-speaking students and the other among their educators during a university closure in the first half of 2020. We succeed in collecting responses from 347 students and 210 university educators.

The online questionnaires were delivered by 1Ka, an open source application for web surveys (https://www.1ka.si/). The link to the student survey was made available to university students through various access channels, such as social networks and faculty mailing lists. The link to the survey for educators was sent to all employees with teaching responsibilities by official e-mail and was sent again a week later as a reminder. Limitations of this kind of data collection include self-election and lack of response from the invisible majority.

The instrument was anonymous, and initiating the response was considered as consent. An opt-out option was recognized in the possibility that no fields were marked as mandatory and that each participant gained no penalties or benefits by commenting. According to the rules of the university, such research, where no sensitive personal data is collected, does not require the consent of an ethical or similar body.

Coding and data structure

All collected responses were saved as a spreadsheet. The spreadsheet was initially cleared of missing data and meaningless comments in both response fields. In cases where comments were provided in one or the other field, the other field was counted as undecided (Table 2). Since some respondents made several comments, these were also separated; therefore, we ended up with 1341 open-ended responses: 799 from students and 542 from educators.

Previously, a decision was accepted that comments would be categorized as present or not present not only according to their content, but also according to the strength of the response. For this purpose, five rankings similar to a 5-point Likert scale were defined. The ranks were: very negative (-2), negative (-1), neutral (0), positive (+ 1) and very positive (+ 2) response. An example of such coding is the difference between the words "difficult" and "very difficult". For statistical analysis and to avoid negative values, the scale was transformed into a range from 1 to 5. Later, it was recognized that neutral answers occurred very rarely, and since they could not be randomly added to the negative or positive comments, they were excluded from the quantitative statistical tests. Also excluded were answers in the category 'Undecided'.

Three researchers carried out the coding based on "word-for-word coding", as a dynamic and reflective process with evaluation and re-evaluation of each response, provided that agreement was reached on the category to which it belongs and its strength. The coding is based pragmatically and technically on the principles of Content Analysis and Grounded Theory, which led us to find new areas (Charmaz, 2014; Isabella, 1990; Suddaby, 2006; Thornberg, 2012), as well as on suggestions from Twining et al. (2017). The comments from students and educators were first categorized separately for both groups into Preliminary Organizing Categories (POCs). The description of the POCs is shown in Table 1.

Later comments provisionally included in POCs were classified into 35 final concepts (Fig. 1; Table 3) (Isabella, 1990). Because it emerged that there was no need to add categories to the POCs, these became the final organizing categories (hereafter categories). Examples of typical comments are provided in the Appendix Table 6.

Fig. 1.

Organizing categories and concepts revealed through the coding process

Data analysis

Once the existence of a concept had been established, all similar comments were clustered in this concept, separately for students and educators. In a process of reflection and lively communication between researchers at the end of the coding, we ended with 35 coding concepts, organized into six categories and ranked on a 5-point scale between very negative (-2) and very positive (+ 2), in one of the pre-established ranks. The results of descriptive data analysis for the organization of the categories are presented in Table 3, and for all categories and concepts in Table 4. The frequency of statements assigned to each concept and rank was used for comparative analysis between students and educators. Given the unequal group sizes, percentages were used as a starting point for comparative analysis. When analyzing the individual concepts, we counted the frequency of responses and calculated their proportions according to educators N_E (%), students N_S (%) and both groups together N_ES (%). The concepts mentioned by educators (E) and students (S) only and by both groups (ES) are shown in column U in Table 4.

The Chi-square test was used to determine statistical significance between frequencies, and Cohen's h was used to determine effect size (Lee, 2016). Cohen's h was the choice for comparing between category proportions (Table 3). Cohen's h was not calculated for individual concepts because the number of comments was too small in most cases to provide values in reasonable confidence intervals (Banno, 2016).

Those concepts with frequencies below ten are presented in the paper but have not been included in the graphic representation (Fig. 2). The level of agreement between students and educators (LA) was calculated as the difference between negative and positive comments, disregarding neutral comments. In cases where the concepts are unique to one or the other group, only the frequencies for this group are given.

We were unable to perform the traditional GAP analysis in the sense of looking for probable cause between some given values and their realization, or between results of our study and results from other published studies. The reason was that in the tradition of Grounded Theory (Charmaz, 2014; Isabella, 1990; Suddaby, 2006; Thornberg, 2012; Twining et al., 2017) theories emerge from the data collected. Despite our best efforts, we were unable to find a similar study with which to compare.

Results

Results are reported as tables and graphs.

Organizing categories and concepts revealed through the coding process

Thirty-five concepts, belonging to six categories were formed (Fig. 1). These concepts and categories form the subject of analysis to explore differences between students and educators.

Differences between organizational categories

The categories differ both in the number of concepts and in the comments on OLE attributed to them by students and educators (Table 3). In terms of frequency of comments, the category 'Organizational' (21.8%) comes first, followed by 'Social interaction and pedagogy' (21.5%), 'Technology' (18.9%), 'Perceived usefulness' (10.6%) and 'Personal' (9.3%), while 17.7% of respondents reported being 'Undecided'. Based on the frequency and excluding the category 'Undecided', three concepts are most prominent: a) Web connection (8.9%), b) Work organization: personal assets (7.5%) and c) Interaction (7.1%), which show the three most problematic areas of OLE. The biggest difference between students and educators occurs in the category 'Organizational', where most comments were made by students. The effect size (h = 0.42) falls into a small effect size and is closer to the medium value threshold (h > 0.5) than to the statistically insignificant margin (h < 0.2). Differences in the number of comments between two categories ('Social interaction and pedagogy'; 'Undecided') fall at the lower threshold of the small effect size, and the differences in three categories ('Personal', 'Perceived usefulness', and 'Technology') can be considered statistically insignificant, which shows that they characterize both groups in the same way. We would like to highlight the category 'Technology', which received high attention in both groups.

Differences between concepts

The comments (positive/negative) from both groups were categorized into 6 categories and 35 concepts. On average, each student provided 2.3 comments and educators 2.6 comments. Although these numbers are close to each other and small in terms of effect size (Cohen's h = 0.38; CI 0.21—0.58), there are differences between several concepts and the comments attributed to each concept. The most diverse categories are 'Social interaction and pedagogy' and 'Personal', with eight concepts each. 'Organizational' (seven concepts), 'Technology' (six concepts), 'Perceived usefulness' (four concepts), and 'Undecided' (two concepts) follow.

When examining the differences between the two groups, it was found that of all the concepts, most are shared, but there were six concepts that were valid only for educators and six only for students, which shows that some of the views are not shared between the groups. The unique concepts mentioned by the educators belong only to the categories 'Personal' (Work input), 'Social interaction and pedagogy' (Student satisfaction), and 'Technology' (Learning materials, Copyright, and Technology impact). The frequency of all the comments attributed to these concepts is less than ten, so we suspect that only a few individuals recognized these as problematic or beneficial, but they go unnoticed by the majority.

The unique concepts mentioned by students belong only to the categories 'Personal' (Concentration and Own lack of knowledge), 'Social interaction and pedagogy' (Interest, understanding, the adaptation of educators, and Lack of educators’ skills), 'Organizational' (Work organization: learning), and 'Perceived usefulness' (For certain study resources). With the exception of the concept ‘Own lack of knowledge’, the frequency of comments attributed to all other concepts is greater than 10. These concepts seem to be important for the majority of students because they relate to the learning process (Concentration, Work organization: learning, Perceived usefulness … for certain study resources) and to their perspective on educators (Interest, understanding, adaptation of educators, and Lack of educator’s skills).

Among the shared concepts, three allow the calculation of Cohen's h within reasonable confidence intervals. Work organization: personal assets (Cohen's h = 1.86; CI = 1.21—2.51) and Web connection (Cohen's h = 1.13; CI = 0.70—1.55) are more prominent among students, and Interaction (Cohen's h = 0.58; CI = 0.16—1.00) is more problematic for educators.

Differences between negative and positive views

The frequency results between positive and negative comments from students and educators are shown in Table 5. To calculate the ratio between positive and negative comments, very positive and positive and very negative and negative comments were summed. From the frequencies of coded comments, an asymmetry in the number of positive and negative comments and their strength between students and educators could be identified (χ2 (3, N = 981) = 137.31, p < 0.00001).

Table 5.

Differences between negative and positive views

	Very negative (%)	Negative (%)	Positive (%)	Very positive (%)	Row Totals	Sum negative (%)	Sum positive (%)	Cohen's h
Students	44 (6.4)	352 (51.2)	263 (38.2)	29 (4.2)	688	396 (57.6)	292 (42.4)	0.30, 0.15 – 0.45
Educators	59 (14.6)	200 (49.4)	124 (30.6)	22 (5.4)	405	259 (63.9)	146 (36.1)	0.57, 0.36 – 0.77
Column Totals	103 (10.5)	552 (56.3)	387 (39.5)	51 (5.2)	1093	655 (66.8)	438 (44.6)
Cohen's h						0.42, 0.26 – 0.58	0.68, 0.48 – 0.88

It is obvious that negative views prevail among both groups. However, the students' comments tend to be less negative (Cohen's h = 0.30; CI 0.15—0.45) than the comments of their educators (Cohen's h = 0.57, CI 0.36—0.77). A comparison of negative views between them shows that educators are more negative than students (Cohen's h = 0.42, CI 0.26—0.58) in negative views and the students are more positive than educators (Cohen's h = 0.68, CI 0.48—0.88) in positive views; however, in all cases the effect sizes are in the small or medium range.

To examine the differences for each concept in more detail, a percentage scale was calculated separately for students and educators. This scale represents the response strength of each factor in relation to the total number of concept responses. The level of agreement (LA) is calculated as the difference between negative and positive comments, disregarding neutral comments. A positive LA means that the percentage of positive comments (very positive and positive) was higher and vice versa.

Figure 2 shows a trend in the transition of concepts, regardless of the category to which they belong, from 100% negative (4 cases) to 100% positive (1 case) and a proportion of negative and positive comments from students and educators. In the discussion, we comment on these according to the category to which they belong.

In analyzing the strength of comments on the negative and positive scale, we identified a trend in the transition of concepts, regardless of the category to which they belong, and a proportion of negative and positive comments from students and educators. We speculated that the positive or negative numbers relate to the extent of problems and benefits associated with each category. However, the correlation r (5) = 0.317; p = 0.603 between the number of concepts and the number of comments attributed to each category does not support this claim.

To further examine the differences between the benefits and problems of students and educators for each category, the level of agreement (LA) was calculated. Depending on the category to which they belong, the following findings were obtained:

'Perceived usefulness'

All four concepts in the category 'Perceived usefulness' have high positive LA. The highest LA pertains to the concept 'Perceived usefulness …for students (LA = 100): both educators (38%) and students (62%) recognized that OLE is positive for students who are otherwise unable (illness, disability, etc.) to attend lectures. This is followed by Perceived usefulness …in a crisis situation (LA = 91), Perceived usefulness …for certain types of study (LA = 88) and Perceived usefulness …for certain study resources (LA = 79), the last one being unique to students. The conclusion is that everyone recognizes some benefit from the transition to OLE. It is worth noting that we did not find any "positive image" for online education under "regular conditions" that was identified by all students (and educators) equally.

'Organizational'

In the category 'Organizational', there are three concepts with positive LA. The highest LA pertains to the concept Work organization: assets (LA = 94), where 86% of the students and 11% of educators agree that because of OLE there are benefits concerning the time and money devoted to transport. Positive LA was also reported for Work/study from home (LA = 57), and Work organization: learning (LA = 54). Slightly negative LA was reported for Work organization: pedagogy (LA = -9); however, the concepts Organizational support (LA = -92) and Implementation knowledge assessment/exams (LA = -100) have high negative LA. In these concepts both students and educators feel dissatisfied with the support they have received and do not know how they will manage to organize or take exams in OLE. Another acknowledged problem is that the positive comments were mainly attributed to quality of life and not to quality of study. Although the positive influence of educational organizations on improving quality of life is limited and can be increased by negotiated timetables and the number of tasks required to complete the course, the study quality is not always the same. The influence of OLE on the quality of educational outcomes can be shared, but the quality of organizational support is entirely on the shoulders of the organization.

'Social interaction and pedagogy'

In the category 'Social interaction and Pedagogy' only the concept Attendance has positive LA (LA = 64). Students (36%) and educators (45%) are of the opinion that more students are present during the lectures in OLE compared to face-to-face teaching. Only 18% of the educators think differently. All other concepts in this category have negative LA: Collaboration (LA = -7), Knowledge/understanding (LA = -23), Communication (LA = -55), Interaction (LA = -70), Interest, understanding, adaptation of educators (LA = -95) and Lack of educators' skills (LA = -100). The last two concepts are unique to students because they believe educators are inflexible, do not understand their perspective, and are unable to adapt to the technology used in OLE. It is interesting to note that in their responses, students associate the age of educators with a lack of skills. The most commonly used expression was "old professors".

'Personal'

In the category 'Personal', there is no concept with positive LA. The concept Personal satisfaction has neutral LA (0), while the other four concepts Health (LA = -41), Previous experience (LA = -82), Concentration (LA = -83), Motivation (LA = -100) have negative LAs. The highest negative LA occurs in Motivation, where 82% of the students and 8% of educators feel unmotivated to learn/work in OLE The concept Concentration, unique to students, shows that 91% of students have difficulty concentrating while attending educational activities on OLE.

'Technology'

All concepts in the category 'Technology' have high negative LA. For the concept Software (LA = -74), 39% of the students and 47% of the educators reported various problems with the software used in OLE. In the concept Hardware (LA = -97), both students and educators reported problems with age and lack of computers/computer equipment (microphones, webcams, etc.). The highest negative LA occurs in the concept Web connection (LA = -100), which also has the highest frequency of entries (N = 120) among all categories. Both students (78%) and educators (22%) reported constant problems with connectivity.

In total (Fig. 2), positive LA is reported for 8 concepts representing 4 categories ('Perceived usefulness', 'Social interaction and pedagogy', 'Organizational', and 'Personal'). One concept is neutral and represents the category 'Personal'. Fourteen concepts have negative LA and represent 4 categories ('Social interaction and pedagogy', 'Technology', 'Organizational' and 'Personal').

Most concepts are not balanced and represent either a high positive or a high negative LA. Only four concepts are roughly balanced: Personal satisfaction (LA = 0), Collaboration (LA = -7), Work organization: pedagogy (LA = -9) and Knowledge/understanding (LA = -23); however, the last one is roughly balanced only in LA. When analyzing the ratio between negative and positive comments from students and educators, we observe that 38% of students reported negative and only 8% positive; on the other hand, 23% of educators reported negative and 31% positive.

Discussion

As a result of the lockdown, FODE has brought, like the awakened sleeping giant (Martorella, 1997), not only solutions for saving a study process interrupted by a pandemic, but also many undesirable side effects resulting from the distinct positions of students and educators in this process. While educators have the choice of adapting to the new environment, adapting the environment or changing the environment, students can only adapt to the environment (Sternberg, 2005). Given this assumption, and with the desire to neutralize potential tensions and misunderstandings between students and their educators, the only conclusion is that studies examining both groups using similar instruments are necessary. We can observe that many researchers and opinion-makers have found education in the new situation challenging. In an extremely short time, they produced numerous papers on the influence of COVID on various aspects of university life. Most studies focus on students (e.g., Chen et al., 2020; Dilmaç, 2020; Flores & Gago, 2020; Sepulveda-Escobar & Morrison, 2020), a few on educators (e.g., Almanthari et al., 2020; Rapanta et al., 2020), and we could only identify one qualitative study (Hebebci, et al., 2020) that focuses on the comparative views of both.

We find that students and educators shared most views; however, there were also unique views (concepts) that were not shared between the groups. Six views unique to students and six unique to educators were identified, reflecting that such concepts are recognized as problematic or beneficial in one group, while remaining unnoticed in the other group (Table 3). In the student group, such views are usually related to the learning process (e.g., concentration) and their perspective on educators, while educators' concerns are related to organization and study materials (e.g., copyright).

The examination of the concepts showed that the views are far from balanced. Both of our hypotheses: (H₁) there exist differences in quantity and severity of the problems detected by students and their educators in using and teaching¹ (learning²) with online learning environments and (H₂) there exist differences in quantity and quality of benefits detected by students and their educators in using and teaching¹ (learning²) with OLE was accepted. Negative views predominate in both groups, and educators are more negative than students, which contradicts the results of the study by Hebebci et al (2020). This finding in itself requires additional comparative studies to examine the source of these differences. We suspect that the differences are related to the quality of living conditions, equipment, working conditions and the like.

When considering organizational categories, only in the category 'Perceived Usefulness' were the comments of both groups mostly positive. In line with the theories, it was once again confirmed that 'Perceived usefulness' is an important construct for explaining online behavior and probably for adaptation to online learning, even under forced circumstances. However, all responses point to exceptional circumstances and not to online education in "normal" times. On the other hand, the comments on all three concepts from the category 'Technology' were mostly negative. For example, Web connection was one of four concepts receiving 100% negative comments. The comments in the other three categories ('Social Interaction and Pedagogy', 'Organizational', and 'Personal') varied from positive to negative depending on the concept; however, the comments were mostly negative. Even when students have no choice, they find some positive aspects that are not based on learning experience or technology, but on Work organization: assets and Attendance concepts. Both are associated with domestic comfort and prosperity. Educators and students share 23 concepts, but there were 6 concepts unique to students and 6 unique to educators. These unique concepts show that one-third of all views are not shared between groups.

To summarize the findings: The most common problems with teaching/learning with OLE detected by students and educators were related to the concepts: Implementation, knowledge assessment/exams, Web connection, Motivation, Hardware, Organizational support, and Concentration. Students identified two unique concepts: Lack of educator's skills and Interest, understanding, adaptation of educators. These concepts also had more than 90% of the negative responses. On the other hand, students and educators recognized benefits in teaching/learning with OLE in the concepts: Perceived usefulness for certain study resources, Perceived usefulness for certain types of study, Perceived usefulness in crisis situation, Work organization: assets and Perceived usefulness for students. Perceived usefulness for certain study resources is unique to students. These concepts also had more than 90% positive responses. There were few concepts that were unique to both educators and students, however, these concepts were specific to each group. The main differences between students and their educators regarding teaching/learning with OLE were in the opportunities to choose the environment and the prevalence of negative views. Although educators had the choice of adapting the environment or changing the environment, their views were more negative than those of students who could only adapt to the environment chosen by their educators. We also found some unique differences between educators and students in the 'Technology' category, where students had more issues with web connections and hardware, while issues with software were shared. Also worth mentioning is the concept of Health, where student responses predominated, which can be explained by the perspective that they spent more time directly present at the computer than educators.

Conclusion

Novel situations such as overnight migration to an online learning environment without careful preparation and planning usually have many unwanted side effects and therefore require strict evaluation. This can be done using familiar methods such as observation and interviewing; however, there is always the possibility that we will fail to collect data that is important to the community. Open-ended comments can highlight the details that are most important at that given moment.

The main findings were that students and educators shared most negative and positive views; however, there are unique views that are not shared between the two groups. The negative views outweigh the positive, and educators are more negative than students. The category 'Perceived usefulness' is the most positive and 'Technology' the most negative category. Positive views were attributed to quality of life, not to the quality of study.

Findings from this study will be very helpful for institutional evaluators, since they will show undesirable side effects that are usually overlooked. In the case of our University, we can conclude that there is room for improvement, especially on the side of the educators. However, the students' problems are related to their domestic comfort and the associated prosperity, over which the university has no control.

The most important contribution of the work to the existing body of knowledge is the comparative analysis of the unconstrained views of students and their educators about Online Learning Environments (OLE) as the workhorse of Forced Online Distance Education (FODE). As it appears, there are several possible futures. The first, optimistic and favored by many, is that sooner or later the university will return to its well-pinned "brick and mortar" courses in a pre-closure fashion. The second, more realistic, is that the courses will be blended, as a mix of online lectures and hands-on activities. The third, probably unrealistic, that all education will be transferred online. Apart from a minority of already fully online tertiary education institutions, the second option is the most likely future form of OLE. To make this future comfortable for students and their teachers, the results of this study can be seen as one of the possible entrances to the labyrinths of possible unexplored paths to the best possible educational experiences. Research is devoted to the evaluation of e-learning from two perspectives: that of the learners and that of the educators. Nowadays, the literature is dominated by studies that usually present the diagnostic results of one group or the other, however, the juxtaposition of the two perspectives is an obvious process and rarely explored. Despite our efforts, we could not find a similar study to compare.

Limitations

This study has several limitations that should be taken into account in order to make the results more objective. The main purpose of this study was to identify the problems and benefits perceived by students and educators in relation to FODE. It is possible that students and educators at other universities might provide different comments. We can only speculate about the transferability of the results to the world population. This is because the study was conducted with data collected only in one institution. Problems, benefits and differences between students and educators could vary depending on the university environment, the systems used, the organizational support and similar factors. Against this background, the study should be repeated with the involvement of populations from other countries. The next limitation associated with data collection is self-election, which can lead to biased views. The probably much greater problem is the lack of responses from the invisible majority of students and educators. For them, we can only speculate that their answers match those of the responders. However, it is impossible to correct this potential error in the study design. Nor did we ask for comments from non-Slovenian speaking students (Erasmus and other exchange students) and educators (visiting lecturers), and we did not try to identify students with special needs.

Abbreviations

FODE

Forced Online Distance Education

COVID-19

Corona virus disease 2019

TAM

Technology Acceptance Model

UTAUT

The Unified Theory of Acceptance and Use of Technology

GETAMEL

General Extended Technology Acceptance Model for e-learning

CCUM

Computer Center of the University of Maribor

OLE

Online Learning Environments

POCs

Preliminary Organizing Categories

LA

Level of Agreement

Appendix

Table 6.

Development of coding categories (Isabella, 1990)

Organizing Categories	Examples	Specific Coding Concepts
‘Personal’	I find it difficult to motivate myself to use online learning environments	Motivation
	frequent headaches and a lot of stress	Health
	initial lack of knowledge of MS Teams	Previous experiences
	anonymity / possibility of recording	Privacy
	more relaxed atmosphere / I don’t like it	Personal satisfaction
	Requires a great deal of time	Work input
	It’s difficult to maintain concentration	Concentration
	Each learning platform requires specific knowledge to use	Own lack of knowledge
‘Social interaction and pedagogy’	impersonal communication / fast response of assistants in the MS Teams environment	Interaction
	poor student collaboration and involvement in discussions	Collaboration
	I don't know if students are present	Attendance
	Online communication cannot replace a live conversation	Communication
	I don’t understand the explanation and find it hard to follow	Knowledge/understanding
	Educators do not know how to adapt and do not have an understanding of our problems	Interest/understanding/adaptations of educators
	Educators are inexpert in program and sometimes it takes half an hour to start lectures	Lack of educators’ skills
	I like distance learning so much that I attend all the lectures	Students’ satisfaction
‘Technology’	poor internet connection	Web connection
	We don't have a computer at home. / My microphone is not working	Hardware
	I had problems loading programs	Software
	Due to the closure of libraries, I do not have access to literature	Learning materials
	The material should be protected from the possibility of printing and reproduction arising from copyright	Copyright
	New tool MS Teams in my pedagogical activities	Technology impact
‘Organizational’	rational use of time	Personal assets
	A more relaxed way of learning, an easier approach to listening to lectures, as no mobilization is required	Learning
	Problems with subjects where I cannot complete all the activities without practical work in the laboratory	Pedagogy
	Tracking and organizing tasks is easier than with paperwork	Administration
	I can teach in a tracksuit / I can cook lunch during lectures	Work/study from home
	There are problems with certain forms of knowledge testing	Implementation of exams online, and organizational support
'Perceived usefulness’	A good solution for crisis situations such as a pandemic	…in crisis situation
	All courses and learning materials are contained in one location, for example in MS Teams	…for students
	Some faculties, especially the social sciences, can easily carry out all forms of teaching in OLE. Especially if lectures and exercises are conducted by videoconference, students are not deprived of the quality of contact hours	…for certain types of studies
	Constant access to explanation and learning materials in one place	…for certain study resources
‘Undecided’	/ or no problems/benefits	Neither-neither
	(blank)	No opinion

Authors contributions

Dolenc K. (D.K.), Šorgo A. (Š.A.), and Ploj Virtič M. (P.V.M.) designed the study and the instrument. D.K. collected data, D.K., A.Š. and P.V.M. coded  and analysed the results, A.Š. developed a theoretical framework, which was discussed and accepted by D.K., A.Š. and P.V.M.. D.K. wrote the draft of the paper which was reviewed and improved by input of A.Š. and P.V.M.

Funding

This work was supported by the Slovenian Research Agency under the core projects: “Information Systems”, grant no. P2-0057 (Šorgo, Andrej) and “Computationally Intensive Complex Systems”, grant no. P1-0403 (Ploj Virtič, Mateja).

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

The authors declare that they have no conflict of interest. The article is an original work of all the authors, and has not been submitted elsewhere, nor is it under consideration for publication in any other journal. No copyrighted material is included in the paper.