The impact of science teaching strategies in the Arabic-speaking countries: A multilevel analysis of TIMSS 2019 data

Abstract

This paper analyses teaching strategies in relation to the science achievement of 4th-grade students using TIMSS 2019 data. It focuses on seven Arabic-speaking countries, which share similar cultures and distinct features of their societies and school systems. It applies multilevel modeling with student- and classroom-level information to decompose variance and understand relationships at these two levels. Overall, the results show the importance of analysis at the classroom level, as half of the science achievement variance is associated with differences between classrooms. Our results indicate that in the Arabic-speaking countries, emphasis on academic success and more active instruction are positively related to student achievement. On the other hand, science investigation methods such as planning and running experiments are negatively related to achievement, although the effects are small in magnitude. Moreover, the results from multilevel models with random coefficients for home resources suggest that higher emphasis on academic success increases differences in student achievement depending on their socioeconomic background.

1. Introduction

Teaching strategies are all those practices and methods teachers may use when instructing students, which are expected to raise knowledge and understanding of science. Moreover, they can also help students in noncognitive outcomes such as problem solving, communication, self-monitoring, and self-reflection [1,2] The topic of teaching strategies is well-studied in the literature, and the most effective strategies from the point of view of meeting students' needs have already been investigated [[3], [4], [5], [6]]. For instance, a meta-analysis by Ref. [7] indicated that strategies based on questioning, inquiry, enhanced context, and collaborative learning strategies are the ones associated the most with higher science achievement in the United States [7].

However, comprehensive research on the effectiveness of science teaching strategies in countries outside of the US and Europe remains limited. This gap is significant because understanding teaching strategies in diverse settings can offer valuable insights into globally applicable methods and context-specific nuances. Furthermore, as educational practices and outcomes may be influenced by local cultures, organization of the school system, or student and teacher attitudes, it is crucial to expand research horizons and investigate effective teaching strategies in different parts of the world. Moreover, while the importance of examining teaching methods and practices in primary school is crucial, there is a lack of comprehensive evidence on the impact of reforms in science teaching [8], as well as missing evidence on the impact of large-scale assessment on the design of the reforms themselves [9].

This research paper aims to examine the associations between teaching strategies in science and educational achievement in the Arabic-speaking countries that participated in the 4th grade assessment of TIMSS 2019. The countries included in the analysis are Bahrain, Kuwait, Morocco, Oman, Qatar, Saudi Arabia, and the United Arab Emirates. These countries share a common cultural background and have similar education systems, characterized by gender separation and centralized structures. The study utilizes the large-scale data from the TIMSS assessment to explore whether the relationships between teaching strategies and science achievement differ among these countries. The TIMSS assessment, which in 2019 reached its 7th edition, provides high quality estimations for student performance, with a framework that guarantees proper representativeness, thus having a significant potential for generalizability of its results. By analyzing the data at the individual student level, the research seeks to uncover any variations or similarities in the associations between teaching strategies and students' performance in science across the participating countries.

The comparative analysis considers the specific teaching strategies employed in science education and their potential impact on students' achievement. The study explores factors such as inquiry-based learning, hands-on activities, teacher-student interaction, use of technology, and other instructional approaches that have been shown to influence educational outcomes. By examining the associations between teaching strategies and science achievement in the Arabic-speaking countries, this research aims to provide insights into the effectiveness of different instructional approaches within these specific contexts. Identifying any variations in these associations across countries can inform educational policymakers and practitioners about the potential benefits and challenges of specific teaching strategies in enhancing science education and improving students' academic performance. Through this comparative analysis, the study contributes to the existing body of knowledge on teaching strategies and their impact on educational achievement in science. The findings have the potential to inform evidence-based educational practices and policy decisions, leading to improved science education outcomes in the Arabic-speaking countries participating in the TIMSS 2019 assessment.

The effectiveness of teaching methods may vary depending on the student characteristics [10]. For instance, problem-based learning has a different association with achievement depending on the skills of students: students with verbal skills below average can benefit more from it [11]. The effects of teaching might also be mediated through characteristics like gender, socioeconomic status, or student well-being. As most studies focus on Western cultures and societies, they do not properly recognize contextual differences in Arab countries. For example, gender could play a different role as boys and girls are often separated in schools, and socioeconomic background is typically less strongly related to achievement in Arab-speaking countries [12].

Against this background, the objective of this study is to investigate the role that teaching strategies and methods, together with class- and student-level indicators, have in influencing science achievements of 4th grade students in the Arab countries. The evidence gathered from the 2019 TIMSS assessment allows us to obtain robust results that can contribute to the evidence on the effectiveness of teaching strategies, and at the same time provide a solid background for the implementation of educational reforms based on evidence. Our results support the claim that classroom-level strategies may be the most effective, with active instruction being positively associated with science scores.

The paper proceeds as follows. The first section reviews the relevant literature on the topic of the most effective teaching strategies in science. The second section presents the data and the third the methodology. The fourth section discusses the results, while the last section concludes.

2. Literature review

It is crucial to understand the most effective strategies in science – a subject based on concepts that may be difficult for students to understand without appropriate support [13]. Nonetheless, there is no consensus on more innovative teaching strategies, such as active instruction [14], and their effectiveness. International assessments show that more innovative teaching strategies, including active learning, may also negatively impact student performance, while more traditional teaching methods positively affect achievement [15]. Inquiry-based teaching is the most impacting innovative teaching strategy in science. Engaging students generally improves comprehension compared to passive learning [16]. Moreover, it has been found that teachers who use very controlling strategies tend to be ineffective in improving student achievement and may even impair their performance [17,18].

A framework for the development of more effective learning has been proposed by Ref. [19]. The authors indicate four main principles of learning, which include the importance of biological and neurological systems, the importance of emotions and identity, the cultural aspects of learning, and the relevance of social interactions [19]. A meta-analysis implemented in the United States showed that enhanced context and collaborative learning strategies are among the most effective in increasing student achievement in science [7]. Interestingly, while still positively impacting achievement, enhanced material strategies and instructional technology strategies were the least effective [7]. For the purpose of our analysis, the framework developed by Ref. [19] allows us to identify different types of teaching strategies that may be effective in influencing the achievements of students, which include strategies of science investigation, active instruction, but also emphasis on academic success. The four main principles of learning outlined in their study can provide more equitable education and increase the engagement of students [19]. Contrary to these findings, our estimations show how innovative science teaching strategies have a negative association with achievements, possibly as a result of inadequate professional development and resources; nonetheless, we find that active instruction and emphasis on academic success are associated with higher science achievements.

Problem-based and project-based learning practices have been shown to be equally effective on the communication and collaboration skills of students when learning science [20]. This has implications, especially for those subdomains which can benefit more from group work, such as physics [20], which relies upon experiments and laboratory inquiries. Teaching strategies that focus on the pedagogical and educative aspects, and not only on content knowledge, have been shown to be more effective in increasing students achievement when controlling for the learning styles of students [21]. This again stresses the importance of incorporating psychological and behavioral aspects in the professional development of teachers and not only strictly focusing on the topics of the subject.

2.1. Innovative teaching strategies and the use of technology

Innovative teaching strategies have been proven to be beneficial for learning science. Effective practices include asking questions [22], using practical examples [23], developing critical thinking [24,25], the use of inquiries and experiments [26,27], and using real-world examples [28]. The use of innovative instructional strategies is shown to improve the educational achievement of students, as it allows them to better understand scientific concepts. It is also important to stress self-efficacy for students, as it improves confidence and liking of the subjects [29].

Science is an interdisciplinary subject that links to other knowledge and real-world issues. A more general approach to real-world issues can increase the effectiveness of teaching practices in science [30], and passing from theoretical to practical issues can be achieved by employing problem based-learning [31]. A problem-based approach that focuses on the most important social and natural issues can increase the comprehension and understanding of scientific concepts by students [32]. This approach also positively impacts educational achievement [[33], [34], [35]]. Importantly [36], have also shown that working on problems improves student attitudes toward science and helps build correct beliefs about the discipline.

When considering how technology can be effective for science teaching, the evidence suggests that it can provide support for teachers and improve their motivation [37], and that its effectiveness may strongly depend on the class context in which technology is actually used [38]. The use of technological tools such as laptops has been found to have a moderate positive impact on learning, in terms of more effective writing and problem-solving [39]. A meta-analysis by Ref. [40] has shown a positive effect of mobile devices on achievement, yet this effect is moderate in size; the authors stress the fact that the use of these devices may also have negative effects, which should be thus considered when increasing the availability of digital tools for classes. Moreover, this difference in student performance due to the use of technology is visible even without changes in teaching strategies between students who do and do not use laptops [39]. Comparing the achievement of science students who do and do not use digital tools shows that the former have significantly better learning but with a significantly moderating effect of teaching strategies. This means that the impact of digital tools is higher when they are used together with developed instruction practices and not independently [41]. While we do not directly include measures of technology use, it is important to stress how technology can support active instruction strategies, and increase the involvement and self-efficacy of students when learning, thus representing an opportunity to innovate and increase the quality of teaching.

2.2. Self-efficacy and teacher professional development

An important aspect related to teaching strategies is the extent to which teachers are confident and prepared enough to actually implement effective teaching strategies. While in our analyses we do not employ direct measures of self-efficacy, which can however be proxied by the use of innovative teaching strategies themselves [42,43], we include measures related to teachers' formal education. Research studies have indicated a positive relationship between teachers' self-efficacy and their knowledge of science content, which can have significant benefits for students’ learning outcomes [44,45].

Teachers who have higher self-efficacy in their abilities to teach science are more likely to possess a deeper understanding of the subject matter and demonstrate effective instructional practices. Furthermore, it has been suggested that targeted training programs for teachers, particularly in areas where they may have lower content knowledge, can have a positive impact on their self-efficacy [45]. [42] conducted a study that emphasized the importance of effective science teaching in narrowing the achievement gap between students at different performance levels. The research findings highlighted the crucial role of motivated and self-effective teachers in promoting positive educational outcomes. They also emphasized the significance of supporting teachers by addressing their beliefs and providing appropriate professional development opportunities to enhance their effectiveness in the classroom [42].

As noted, the self-efficacy of teachers is associated with their knowledge of the teaching content [43]; nevertheless, in order to implement successful teaching strategies, it is important to also train teachers to develop their pedagogical skills, and not just their content knowledge [46]. The importance of self-efficacy has also been stressed for science teachers, in terms of the confidence they have when teaching especially early in their careers [47]. Confidence of teachers in itself has a positive impact on achievement [48,49], thus a lack of it when teaching can lead to poorer outcomes for students. Thus, in addition to developing their content knowledge, teachers need consistent and sound professional development – especially early in their careers – to develop the necessary confidence when teaching science [47]. [50] have shown that science teachers should have a high level of pedagogical content knowledge, focusing on the pedagogical and disciplinary aspects of science teaching, as this can be beneficial for the achievement of students.

Emphasizing the importance of science teaching is crucial for the development of more effective teaching strategies. Research studies have shown that positive attitudes, both among teachers and students, can have a significant impact on the understanding of scientific concepts and improve educational achievement [51]. In the field of science education, the effectiveness of teachers plays a vital role in determining higher educational achievement, even when accounting for student characteristics [52]. Effective teaching practices have been found to be positively associated with lower achievement gap between students with different performance levels [52]. Another relevant indicator for the study of effective strategies is emphasis on academic success, which relates to stressing the importance of education and learning. Emphasis on academic success has been found to increase student performance, specifically in science [[53], [54], [55]]. This is also related to school climate, which has a positive association with achievements [56,57], and which also has a positive effect on emphasis on academic success itself [53]. School climate can also improve teachers' job satisfaction [58], which in turn has a mixed-to-positive impact on student achievements [59]. It has also been found how science achievements are correlated with lower limitations in teaching brought about by student needs [54,60], highlighting the importance of this indicator in analysing effective teaching.

A study by Ref. [61] shows that science teachers usually present the weakest knowledge when considering the links between science, technology, society, and the environment (compared to other science subdomains), thus indicating that teaching practices focusing on these links may be underused in spite their positive impact on student scores. A lack of proper training of science teachers can result in lower confidence when teaching, lower engagement of students, and thus lower achievement [62]. have shown that students can benefit from better-trained teachers who show higher self-efficacy by showing higher confidence when learning science. In general, students can significantly benefit from teachers with a background or professional development in inquiry-based instruction, with a positive effect on science scores, thanks to the increased knowledge of scientific concepts that may be more difficult to grasp without an inquiry-based approach [63].

2.3. Student attitudes

Similarly to teachers’ self-efficacy, student attitudes towards science learning can also be relevant in determining their achievement. In our analysis, we include measures of liking science, confidence in science, and finding the subject clear as taught by the teacher. We include these indicators to test the extent to which student self-efficacy or attitudes can influence their science achievements. The existing literature indicates that the self-efficacy of students is also found to have a positive impact on achievements, although lower in magnitude than the teaching strategy [13]. This indicates the importance of considering student attitudes when designing better instructional methods for science teaching.

[64] have stressed the importance of student's welfare and well-being when designing teaching strategies. In particular, it is important to consider student attitudes and learning preferences to understand which teaching strategies may be the most effective [64]. Helping students develop critical thinking skills is crucial for science teachers and science learning, given the large number of scientific misconceptions that students have and which may not be removed without such critical thinking [65,66]. The authors also stress the importance of designing appropriate curricula including teaching strategies that can help students develop their critical thought and skills [65].

2.4. The present study

Our comprehensive study is dedicated to exploring and analyzing the most effective science teaching strategies employed in Arab countries that participated in the 2019 TIMSS assessment. The region in question exhibits distinct common traits, both culturally and within their educational systems, which are important to underline when considering them. It is important to highlight these shared features, as they may play a significant role in shaping educational practices, as well as their outcomes. Notably, the Arab region has only recently garnered significant attention in the realm of educational research [67]. Furthermore, while in recent years Arab societies have invested in education to foster development and innovation [68], the topic of STEM education specifically remains understudied [69]. This delay in focused research may be attributed to the uniqueness of Arab culture, which may necessitate a distinct methodological approach when conducting investigations into teaching practices [70]. Understanding these aspects, sensitivities and dynamics is crucial for designing effective teaching strategies that can resonate with the local context.

Despite notable investments in education over recent years, the region has faced challenges in achieving its educational reform objectives, particularly in enhancing student learning outcomes [71]. This discrepancy between investment and outcomes may be attributed to the lack of innovation in the implementation of teaching strategies, among which critical thinking [71], as well as self-evaluation and exchanges with peers, also due to school segregation [72]. The absence of robust mechanisms for self-assessment and collaborative learning may be contributing to the challenges faced by the educational systems of the region.

In light of these considerations, our study contributes to the literature on science teaching and learning within the Arab region. We examine the effectiveness of various teaching strategies using evidence gathered from a standardized international assessment (TIMSS 2019), which provides robust and reliable estimations of the competencies of students and in general on schools; through this, we aim at providing policy recommendations for the region that can be properly targeted to the specific countries and cultures that are part of it.

3. Methods and data

3.1. Sample and procedure

In order to examine the effectiveness of teaching strategies, this research paper utilizes data collected from the 2019 Trends in International Mathematics and Science Study (TIMSS) assessment [73]. The TIMSS framework combines the evaluation of mathematics and science teachers in the 4th grade. For the purpose of our analyses, we consider both general teaching strategies that apply to various subjects and science-specific practices as documented in the TIMSS questionnaires. The focus of our analyses is on the Arabic-speaking countries that participated in the TIMSS 2019 assessment. These countries include Bahrain, Kuwait, Morocco, Oman, Qatar, Saudi Arabia, and the United Arab Emirates. It is noteworthy that Arabic was the second most prevalent language used in the TIMSS 2019 assessment, following English. The research examines the teaching strategies employed by educators in these countries, considering both the general pedagogical approaches applicable to all subjects (such as, for instance, emphasis on academic success), and the instructional practices which are specific to science education (such as active instruction in science). By analyzing the TIMSS data, we aim to explore the relationship between these teaching strategies and the academic performance of students in mathematics and science. Our initial sample size, which includes the seven Arab countries that participated in the assessment of 4th grade students in TIMSS 2019, is composed of 60 956 students and 2654 science teachers (see Supplementary information).

We consider several indicators related to science teachers and instruction. At the teacher level, we include the education of the teacher and the degree in science, the years of experience, the gender, and the level of formal education (computed on the ISCED scale). Using item response theory's partial credit model on the ordinal items of the teacher questionnaire, we group teaching strategies based on TIMSS items and what they survey; our aim is to obtain indicators that are representative of teaching practices more in general, without depending on individual items, but at the same time that are diversified enough to capture different teaching techniques and methods.

We proceed as follows. We consider items as provided by the TIMSS questionnaire as the variables for the item response theory's partial credit model, which allows to capture underlying mechanisms given the answers to multiple items at once. In particular, we grouped observing phenomena, demonstrating experiments, planning experiments, conducting experiments, presenting data, interpreting data, using evidence, and doing field work outside the class, as being innovative strategies of science investigation. In addition, we consider relating to student lives, explaining the answers provided by the teacher, bringing interesting material, challenge students to go beyond simple instruction, encouraging class discussion, linking knowledge, encourage the development of the student's problem solving, and encourage to express ideas as indicators of active instruction, being the teaching strategies surveyed by TIMSS as general to both mathematics and science instructions. In all cases, teachers are asked to specify whether they use these strategies “Every or almost every lesson”, “About half the lessons”, “Some lessons”, or “Never”.

The variables related to teaching strategies have been selected and operationalized based on the responses given by teachers. The TIMSS questionnaire included inquiries related to the above-mentioned strategies, which were then considered as the individual items used to construct the indexes. For the purpose of our analysis, we selected and included those variables that relate to either general teaching strategies, or specific science-teaching strategies. The use of Item Response Theory (IRT) allows to combine these items based on their underlying mechanisms, which allowed us to identify the scales that we included in our analysis. In addition to these indexes, we also include emphasis on academic success, safe and orderly school, job satisfaction, and teaching limited by student needs as they are defined by the TIMSS assessment framework. However, these indices were re-estimated using the IRT partial credit model using data from Arabic-speaking countries only. This assures that the measurement of these indices is not affected by responses in other regions.

In the analysis at the student level, several variables were included to capture individual characteristics and attitudes that could potentially influence science achievement. These variables included gender, frequency of absences, exposure to bullying (as measured by the TIMSS index), and various attitude indices such as liking science, confidence in science, and clarity in science instruction. To simplify the analysis, the attitude indices were averaged to create a composite measure of attitudes – in particular, this construct measures the extent to which students are positive about science, in terms of liking the subject, being confident in it, but also finding it clear when taught by the teacher. The home resources for learning index was used as a measure of student family socioeconomic background. This index captured not only conditions for learning after school but also parents’ education and occupation. By using this index, we aimed to compare the effects of teaching strategies across students with low and high socioeconomic status. By considering these individual-level variables, the study aimed to explore how teaching strategies interact with student characteristics and attitudes, and how these factors collectively influence science achievement. This approach provides insights into the differential effects of teaching strategies on students from different socioeconomic backgrounds, allowing for a more comprehensive understanding of the factors that contribute to science achievement disparities.

3.2. Measures

Table 1 provides a description of all variables and descriptive statistics. To facilitate interpretation of the results, all variables have higher values that should correspond to higher achievement. Achievement is measured by the first plausible value for science performance in TIMSS datasets. Educational achievements are standardized in order to have a mean of 500 points and a standard deviation of 100 points in the original sample of TIMSS countries participating in the first study in 1995. For Arab-speaking countries, the average is lower by around 100 points, while the standard deviation is larger – around 120 points, which is more than one standard deviation of the original TIMSS scale). Before the analysis, all continuous explanatory indices were standardized on the available sample to facilitate interpretation of the results. Thus, their average was equal to 0 and standard deviation to 1 in a sample of all teachers for teacher-level indices, and in the sample of all students for student-level indices. After merging with student data the distribution of teacher-level indices slightly changes as shown in the table. For student-level indices, we further decomposed them into class averages to capture the between-class effect in the multilevel model, and into class-average centered variables to capture the within-class effect. Possible missing values were not specifically addressed, being just later absent in the various specifications of the multilevel models.

Table 1.

Description of the variables and weighted summary statistics.

Variable	Description	Mean	St. dev.	Min	Max
Science achievement	Higher values indicate higher science achievement	398.37	119.36	5.00	825.03
Classroom-level variables
Teacher with science-related degree	Teacher has a formal degree in science or science education	0.47	0.50	0	1
Teacher education level	Teacher's formal education (ISCED)	3.67	1.36	1	7
Teacher gender (Male = 0; Female = 1)	Teacher gender	0.62	0.49	0	1
Science investigation	Higher values indicate more frequent usage of science investigation strategies	−0.20	1.04	−1.47	1.89
Emphasis on academic success	Higher values indicate more emphasis on academic success	−0.35	1.00	−1.56	1.86
Teaching not limited by students not ready	Higher values indicate teaching is less limited by student readiness	−0.23	0.80	−0.98	3.37
Index of active instruction	Higher values indicate more active instruction strategies	−0.65	1.18	−3.86	0.95
Average bullying	Higher values indicate lower frequency of bullying	−0.01	0.51	−1.83	1.53
Average attitudes	Higher liking, confidence, and clarity in mathematics	−0.02	0.51	−2.13	1.41
Average home resources	Higher availability of home resources	0.04	0.81	−1.94	2.92
Student-level variables
Within-class effect of bullying	Lower frequency of bullying	0.00	0.86	−4.41	3.10
Within-class effect of attitudes	Higher liking, confidence, and clarity in mathematics	0.00	0.86	−4.43	3.25
Within-class effect of home resources	Higher availability of home resources	0.00	0.59	−2.79	2.86
Student absenteeism	Less frequent absences	3.77	1.59	1	5
Student gender (male = 0; female = 1)	Female student	0.49	0.50	0	1

4. Data analysis

To decompose the effects into classroom- and student-level we apply two-level models with random effects at the classroom level. The basic two-level model can be described by two equations. The first equation describes the model for the student-level where i is an index for students and j is an index for classrooms:

$$Y_{ij}={\beta }_{0j}+{\beta }_{1j}{X}_{ij}+e_{ij}\text{.}$$

In this equation, $Y_{ij}$ (the outcome variable) is the first plausible values of student achievement in science. $X_{ij}$ is a vector of student-level predictors, including student gender and absenteism, and within-class differences in attitudes, bullying and home resources, and $e_{ij}$ is a student-level error.

The second level equation describes classroom-level intercepts:

$${\beta }_{0j}={\gamma }_{00}+{\gamma }_{01}{W}_j+u_{0j}$$

where $W_j$ is a set of classroom-level predictors including teacher characteristcs, teacher strategies and opinions, and classroom averages of bullying, attitudes, and home resources, and $u_{0j}$ is a random error component at the regional-level. We assume that random components are uncorrelated and normally distributed.

This model can be further expanded to capture the effect of teaching strategies on the association between home resources (our measure of student socioeconomic background) and science achievement. In this model, we allow the slope of home resources to randomly vary within classrooms and explain this variation using different teaching strategies. Let us assume that the vector $X_{ij}$ contains only the home resources index and will allow this effect to vary across classrooms. In this case, we can add a third equation explaining the home resources slope and adding a random component that varies at the classroom-level.

$${\beta }_{1j}={\gamma }_{10}+{\gamma }_{11}{W}_j+u_{1j}$$

This model enables the testing of whether the variation of the random components, which explain the coefficients of home resources, significantly deviates from zero. If a significant variation exists, the model proceeds to examine classroom-level predictors that can account for this variation. Building upon the assumptions made in the model with a random intercept, an additional assumption is introduced, namely an unstructured covariance between the random errors at the second level.

We apply the random coefficient model to estimate the interaction between several teaching strategies and home resources. This way we can see if they are associated with weaker or stronger relationship between home resources and science achievement. Positive interaction means that applying more often a particular strategy increases home resources effect, leveraging achievement of students with more resources at home. To the contrary, a negative coefficient would indicate that this strategy has an equalizing effect, leveraging the achievement of students with disadvantaged background.

TIMSS is a complex survey with random sampling of schools, classrooms, and students. We include survey weights to obtain population estimates [74]. To properly apply the weights with multilevel models, we separately use student probability weights and classroom probability weights, with student-level weights scaled so that they sum to the sample size of their corresponding second-level cluster (classroom) see Ref. [75]. In our analyses, we used the first plausible value in science as the dependent variable. We used the Stata software for data preparation and analysis.

When comparing the average achievements across the seven Arab countries, some differences emerge. In particular, the average science achievement computed with the first plausible value equals 494 points in Bahrain, 475 points in the United Arab Emirates, 452 points in Qatar, 436 points in Oman, 404 points in Saudi Arabia, 395 points in Kuwait, and 377 points in Morocco. These estimations are all statistically significantly different from one another with a 95% confidence interval. Nonetheless, while these differences are significant across the countries considered, we retain all of them in the data analysis in order to obtain a comprehensive and robust estimation of the impact of science teaching strategies across a culturally homogeneous area such as the one we use as our unit of analysis.

It is worth stressing that significant differences exist in achievements across countries, and at the same time that country-level differences may determine a different perception of some of the constructs we use in our analyses – for instance student attitudes towards science –, similarly to student gender. Yet, we keep the countries within the same dataset and conduct a single pooled data analysis given that we want to capture how teaching strategies are associated to science achievements in the Arab countries as a whole. We therefore consider the whole area as a single larger entity, in which we implement our data analyses.

5. Results

Table 2 presents the results of the five specifications of the multilevel model. Column (1) shows results for the so-called empty models, providing a reference point for variance analysis. The Intra-Class Correlation coefficient (ICC), added to estimate the variation determined by the class, is 0.49, meaning that nearly half of the overall variation is associated with differences between classrooms. Around half of classroom-level variance can be explained by the model presented in column (2), which includes student-level variables but also classroom averages of bullying, attitudes, and home resources. Student-level variables also explain around 14% of the variance at the individual level. Results for model in column (3), which includes teacher characteristics, like gender and education, show that these characteristics do not explain much of the variance in science achievement. In the analysis presented in Table 2, the last column includes results that incorporate teacher strategies as explanatory variables.

Table 2.

Multilevel regressions with random intercept at the classroom level.

	(1)	(2)	(3)	(4)
Classroom-level variables
Average bullying		17.7***	18.1***	16.0***
Average attitudes		43.6***	44.3***	40.3***
Average home resources		80.3***	81.6***	72.7***
Teacher with science-related degree			4.0	0.4
Teacher education level			2.2	1.8
Teacher gender (Male = 0; Female = 1)			−10.3**	−11.7**
Emphasis on science investigation				−3.1*
Emphasis on academic success				8.5***
Teaching not limited by student				4.9***
Index of active instruction				4.3**
Student-level variables
Within-class effect of bullying		7.3***	7.4***	7.4***
Within-class effect of attitudes		22.8***	22.6***	22.6***
Within-class effect of home resources		22.6***	23.0***	23.0***
Student absenteeism		8.7***	8.9***	8.9***
Student gender (male = 0; female = 1)		5.5***	6.1***	6.1***
Constant	444.9***	364.1***	356.1***	367.1***
Variance (classrooms)	6769.0	3184.8	3054.4	2901.1
Variance (students)	7151.0	6165.0	6091.6	6091.9
ICC	0.486	0.341	0.334	0.323
N. of students	61 733	40 356	35 938	35 938
N. of classes	2683	2589	2259	2259

Source: TIMSS 2019 data. Note: *indicates p < 0.05; **indicates p < 0.01; indicates ***p < 0.001.

The emphasis on academic success and the index measuring active instruction are found to have a positive relationship with science achievement. An increase of one standard deviation in these indices is associated with an improvement of approximately 4–8% of the international standard deviation of student science achievement, or 6–11% of the classroom-level achievement standard deviation. These findings suggest that when teachers place a higher emphasis on academic success and employ more active instruction strategies, students tend to achieve better in science. These teaching strategies appear to have a positive impact on student learning outcomes. Teacher self-reported index of limitations related to student readiness for learning science is also related to achievement, showing that higher readiness is associated with higher achievement.

On the other hand, the emphasis on science investigation using innovative teaching methods (related to experiments and data analysis) is negatively associated with science achievement, although the effect size is relatively small. This means that classrooms where innovative science investigation strategies are emphasized may have slightly lower science achievement compared to classrooms with a lower emphasis on and use of such strategies. This finding suggests how innovative teaching strategies which include the use of technological tools may suffer from inadequate resources, as well as teachers lacking the adequate professional development to implement them effectively.

Most control variables, namely socioeconomic background, attitudes, and bullying, show expected signs. Higher home resources, better attitudes, smaller exposure to bullying, and less frequent absenteeism are all strongly related to higher science achievement at the individual, within-classroom level, but also at the between-classroom level – this means that they have an impact that is not only dependent on the class to which the student belongs to. Interesting are the different effects of gender on achievements depending on whether instructors or learners are considered. In particular, the presence female teachers in the classroom has an average negative impact on scores of 10 points, but with the student-level effect showing higher performance of girls by around 6 score points instead.

The results presented in Table 3 indicate that there are significant differences in the effect of socioeconomic background, as measured by home resources, on science achievement across classrooms. The positive variance in the home resources slope suggests that the impact of socioeconomic background on science achievement varies across classrooms. However, the estimated covariance between intercepts and slopes, which would indicate whether the differences in slopes are associated with differences in average achievement, is not significant. This suggests that the variations in the effect of socioeconomic background on science achievement across classrooms are not related to differences in average achievement. These findings highlight the importance of considering the classroom context when examining the relationship between socioeconomic background and science achievement. The results suggest that classroom-level factors, such as teaching strategies, classroom climate, or instructional practices, may play a role in moderating the impact of socioeconomic background on science achievement. Understanding these contextual factors can provide insights into how to effectively support students from diverse socioeconomic backgrounds and promote equitable educational outcomes.

Table 3.

Multilevel regressions, random coefficient models.

	(1)	(2)	(3)	(4)	(5)
Classroom-level variables
Average bullying	16.0***	16.0***	16.0***	16.0***	16.0***
Average attitudes	40.4***	40.4***	40.4***	40.4***	40.4***
Average home resources	72.9***	72.9***	72.9***	72.9***	72.9***
Teacher with science-related degree	0.4	0.4	0.5	0.4	0.4
Teacher education level	1.8	1.9	1.9	1.8	1.8
Teacher gender (Male = 0; Female = 1)	−11.4**	−11.4**	−11.4**	−11.4**	−11.5**
Emphasis on science investigation	−2.9*	−2.9*	−3.1*	−2.9*	−3.1*
Emphasis on academic success	8.3***	8.5***	8.5***	8.5***	8.5***
Teaching limited by student	4.9***	4.9***	4.9***	4.8***	4.8***
Index of active instruction	4.3**	4.1*	4.3**	4.3**	4.2**
Student-level variables
Within-class effect of bullying	7.4***	7.4***	7.4***	7.4***	7.4***
Within-class effect of attitudes	22.6***	22.6***	22.6***	22.6***	22.6***
Within-class effect of home resources	23.0***	23.2***	22.6***	23.0***	22.9***
Student absenteeism	8.9***	8.9***	8.9***	8.9***	8.9***
Student gender (male = 0; female = 1)	6.1***	6.0***	6.1***	6.0***	6.1***
Interactions explaining the within-classroom slope of home resources
Emphasis on science investigation	3.4***				1.0
Emphasis on academic success		4.5***			2.6*
Teaching not limited by student			4.1***		2.4*
Index of active instruction				2.9**	1.4
Constant	366.4***	366.4***	366.3***	366.3***	366.5***
Variance (slope of home resources)	240.5	219.3	227.9	251.8	202.2
Variance (classrooms)	2908.6	2908.2	2908.6	2909.1	2908.3
Variance (students)	6017.7	6020.4	6019.1	6016.0	6021.9
N. of students	35 938	35 938	35 938	35 938	35 938
N. of classes	2259	2259	2259	2259	2259

Source: TIMSS 2019 data. Note: *indicates p < 0.05; **indicates p < 0.01; indicates ***p < 0.001.

The models presented in columns (1) to (4) of this analysis examine the individual interactions that explain the slope of home resources on science achievement, with each strategy considered individually. Specifically, these models investigate the influence of various indices, namely the emphasis on science investigation, academic success, teacher self-reports of instruction limited by student readiness, and the index of active instruction, on the relationship between home resources and science achievement. The results from these models indicate that all of the interactions between the aforementioned indices and home resources are statistically significant and positive. This implies that higher levels of the indices correspond to a stronger effect of home resources on students' science achievement. In other words, when teachers more frequently utilize investigation strategies, place greater emphasis on academic success, report limitations in student readiness, or employ more active instructional methods, the impact of home resources on learning outcomes in science becomes more pronounced. Yet, different estimations are found when considering these strategies together (see below).

Furthermore, the findings suggest that students with higher socioeconomic status tend to achieve better science results in classrooms where these teaching practices are more prevalent. This implies that the positive influence of home resources on science achievement is amplified in classrooms where teachers prioritize investigation strategies, academic success, address student readiness limitations, and utilize active instructional approaches. These results highlight the importance of considering classroom-level factors and teaching practices when examining the relationship between home resources and science achievement. They suggest that the instructional context, characterized by specific teaching strategies and attitudes, can modify the impact of home resources on students' learning outcomes in science. Understanding these interactions can inform educational policymakers and practitioners in designing effective instructional approaches that maximize the positive effects of home resources on science achievement, particularly for students with higher socioeconomic status.

The results from column (5) of the analysis indicate that when considering all four interactions simultaneously (contrary to the previous individual analyses), only classrooms where teachers place greater emphasis on academic success or report that students are less ready exhibit a stronger effect of home resources on science achievement. Specifically, an increase of one standard deviation in the emphasis on academic success is associated with a 2.6-point increase in the slope of the home resources effect. Considering that the within-class effect of home resources is approximately 23 points, this means that the increase in the home resources effect due to an emphasis on academic success corresponds to around an 11% change. These findings suggest that there is an interaction between socioeconomic background (as indicated by the availability of home resources) and the emphasis on academic success. In other words, a change in the emphasis on academic success can contribute to increasing the inequalities between students that arise from different socioeconomic backgrounds. These results highlight the importance of considering both teaching strategies and socioeconomic factors in understanding the impact of home resources on science achievement. The findings suggest that the emphasis on academic success may amplify the existing disparities in student achievement associated with socioeconomic background. It is crucial for policymakers and educators to be aware of these interactions and strive to implement strategies that promote equity and equal opportunities for all students, regardless of their socioeconomic status.

As in the previous analysis, we also find that more positive student attitudes, less frequent absenteeism, and lower bullying are associated with higher scores in science. While these aspects do not directly relate to teaching strategies, it is important to notice that a generally improved school climate, by which students have more positive attitudes or engage less frequently in bullying (thus worsening school climate) can positively and significantly impact science achievements. Thus, we argue that, while not specifically science-related, students may benefit from teachers who foster positive attitudes and emphasize the need for positive school climate, in cooperation with school principals. Lastly, we again find how gender has contrasting impacts when it is considered for teachers and students, with a negative association with achievements found for females in the former, and a positive one in the latter.

Both our multilevel models show how emphasis on science investigation using innovative teaching strategies has a negative association with achievements. As noted above, this variable includes the strategies of observing phenomena, demonstrating experiments, planning experiments, conducting experiments, presenting data, interpreting data, using evidence, and doing field work outside the class. This could be somewhat counterintuitive, given how the evidence from the existing literature shows that these strategies may instead be beneficial for science achievement. Yet, it is important to notice how in our analytical framework we include both further teaching-related variables, as well as school-level socioeconomic background (proxied by home resources). We argue that innovative teaching strategies, which depend significantly on the school's availability of resources, for instance to run experiments, may exhibit a negative association with science scores due to their observed impact being due to resources availability.

6. Conclusions and policy recommendations

This study aimed to examine the relationship between teaching strategies and science achievement in 4th grade students, using data from the TIMSS 2019 assessment in seven Arabic-speaking countries. The analysis employed multilevel models to explore the variability in science achievement at both the classroom and student levels. The findings indicate that a significant portion of the variation in science achievement can be attributed to differences between classrooms. Approximately half of this variability can be explained by classroom-level factors, including the average levels of home resources, attitudes, and incidents of bullying. Interestingly, teacher characteristics such as education level and specialization did not show a significant association with differences in student achievement. Furthermore, the study assessed the impact of teaching strategies on science achievement using four indices: emphasis on science investigation, emphasis on academic success, and two measures of teacher activity in the classroom. The results revealed that a higher emphasis on academic success and more active instruction were positively associated with student achievement. However, an emphasis on science investigation showed a small negative effect on achievement, though the magnitude of this effect was not substantial. Additionally, the study explored the interaction between home resources and teaching strategies. The results indicated that a higher emphasis on academic success strengthened the relationship between home resources and science achievement. In other words, the difference in achievement between students from homes with varying levels of resources was more pronounced in classrooms with a greater emphasis on academic success.

These findings have important implications for educational policies and practices in the examined Arabic-speaking countries. It suggests that promoting teaching strategies that emphasize academic success and encourage active instruction can potentially enhance science achievement among 4th grade students. The emphasis on academic success, while important for increased student achievements, can also lead to increased psychological challenges, among which anxiety [76], due to the increased competitiveness among students [77]. Therefore, it is crucial to foster a healthy environment when emphasizing the importance of academic success.

Furthermore, efforts to reduce incidents of bullying and address disparities in home resources can contribute to improved educational outcomes. The available evidence underlines the importance of a safe and secure learning environment for students, for which teachers can play a significant role [78,79]. As a meta-analysis by Ref. [80] has shown how instances of bullying are more frequent when students come from a low socioeconomic background. Therefore, it is important to stress the relevance of a safe learning environment to allow students to reach their full potential; in this respect, teachers can play a significant role with their classroom strategies. The creation of such an environment should also consider the importance of socioeconomic background in influencing school climate and bullying for students.

It is worth noting that while female students achieved higher scores on average, classrooms with female teachers tended to have lower science performance. This finding highlights the need for further investigation into the underlying factors contributing to this discrepancy. Overall, this study provides valuable insights into the relationship between teaching strategies, classroom-level factors, and science achievement in 4th grade students. It offers important considerations for policymakers and educators in designing effective interventions and improving science education outcomes in the Arabic-speaking countries examined.

The negative association between innovative teaching strategies and achievements, as noted above, can be due to a lack of economic or educational resources that may allow for their full potential to be reached. In this respect, it is crucial to stress the important role played by thе quality of their implementation. Therefore, an important conclusion that may be drawn from this result is a lack of readiness of thе educational infrastructure to support innovative teaching strategies effectively, both in terms of economic resources and teachers’ professional development. The evidence shows how professional development may have a nuanced impact on achievements depending on how and in which areas it is implemented [81,82], stressing the importance of appropriate and high-quality support for instructors in the implementation of innovative science teaching strategies.

Our study has some limitations that are important to underline. First, while the use of the TIMSS dataset guarantees the reliability and standardization of the results, it also does not allow to account for local cultural aspects that may influence the results. For instance, it has been shown how the way in which Arab science textbooks are developed may prevent students from being fully equipped for science learning [83]. In this respect, investigating how cultural aspects may have an impact on the effectiveness of teaching strategies in the Arab countries. Second, for methodological reasons our analysis was based on the combination of individual items from the TIMSS questionnaire into indexes that identify separate teaching strategies. While this has increased the overall robustness and reliability of the results, it has on the other hand also caused a loss of information related to how single-item teaching strategies may be associated with science achievements. Thus, it could be useful to investigate how the use of individual strategies may be associated with scores. Third, we used data for 4th grade students and therefore obtained results for this particular cohort, yet it could be advisable to also investigate TIMSS data for 8th grade students or PISA data for 15-year-old students could be employed. This would allow to increase the available evidence on how teaching strategies may impact science achievements also for older students.

6.1. Policy recommendations

The results of this study provide valuable insights for policymakers in the region, suggesting specific policy interventions that can enhance science education and improve students’ achievements. Based on the findings, the following policy recommendations can be made:

Promote active instruction strategies. The results indicate that active instruction strategies are associated with higher science achievements. Therefore, it is recommended to encourage and support teachers in implementing more active instructional approaches in the classroom. This can involve providing professional development opportunities, resources, and guidance to teachers on effective strategies that engage students actively in the learning process. As noted previously, an important aspect is nonetheless the quality and field of application of professional development, given that its association with achievements internationally is not straightforward. Therefore, country-level surveys and analyses could help develop appropriate programs for teachers and support them in their implementation of effective teaching strategies.

Emphasize academic success. The study reveals that a higher emphasis on academic success is linked to higher science scores. Policymakers should prioritize creating a culture of academic excellence within schools, emphasizing the importance of academic achievement in science and other subjects. This can be achieved through clear educational goals, curriculum frameworks that promote rigorous standards, and recognition of outstanding academic performance. Yet, attention to psychological well-being should be ensured as well. In this respect, country-level studies could be implemented in order to evaluate the local factors that may affect the psychological well-being of students in relation to emphasis on success.

Address bullying and create safe learning environments. The findings highlight the significant impact of a lower frequency of bullying on science achievements. Policymakers should implement comprehensive anti-bullying policies and programs in schools to create safe and supportive learning environments. This can involve raising awareness, providing training to teachers and staff, and establishing mechanisms for reporting and addressing instances of bullying effectively. In this respect, it is crucial to consider country-level differences in school climate, given that the school area (urban versus rural) or the level of economic development of a given region may significantly affect the phenomenon. Therefore, it is important to tailor initiatives based on the needs emerging from each country and specific areas within each country.

Enhance availability of home resources. The study also identifies the availability of home resources as a significant predictor of higher science achievements. Policymakers should consider initiatives that aim to bridge the resource gap between students from different socioeconomic backgrounds. This can involve providing equitable access to educational resources, such as textbooks, technology, and learning materials, particularly for students from disadvantaged backgrounds. As in the case of school climate, it is crucial to intervene in those areas and those schools which struggle the most with lack of resources and low socioeconomic background. It is important to target the most disadvantaged areas first, to reduce the gaps in economic development and educational opportunities among students within each country.

Foster positive attitudes towards learning science. The study suggests that improving students' attitudes towards learning science can contribute to higher educational achievements. Policymakers should promote inquiry-based and student-centered approaches that foster curiosity, engagement, and a positive mindset towards science. This can be achieved through curriculum reforms, teacher training programs, and creating opportunities for hands-on, experiential learning in science.

By implementing these policy interventions, policymakers can create an environment that supports effective teaching strategies, enhances academic success, addresses bullying, and ensures equitable access to resources. These actions can ultimately improve the learning opportunities and outcomes in science for students in the region, fostering their passion for science and preparing them for future success.

funding

The authors extend their appreciation to the Deputyship for Research & Innovation, Ministry of Education in Saudi Arabia for funding this research work through the project no. (IFKSURG-S106).

Author contributions

Stefano Pagliarani: Writing – review & editing. Abdu AlMufti: Writing – original draft, Formal analysis, Data curation. Bakeel Aldarwani: Writing – original draft, Validation, Methodology, Formal analysis, Data curation. Mohammed Alghamdi: Writing – review & editing, Methodology, Data curation, Conceptualization. Fahad AlShaya: Writing – original draft, Methodology, Formal analysis, Conceptualization. Misfer AlSalouli: Writing – review & editing, Project administration, Methodology, Conceptualization

Data availability statement

The data for this study are available on the TIMSS 2019 international database website at https://timss2019.org/international-database/.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:Misfer AlSalouli reports financial support was provided by King Saud University. Misfer AlSalouli reports a relationship with King Saud University that includes: employment. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article: