Examining the psychometric characteristics of the metacognition questionnaire in teaching: a cross-sectional study

Abstract

Background:

The role of metacognition as a common concept in education is undeniable. One of the challenges in the field of metacognition is to measure the impact of metacognition in teaching with practical tools. Therefore, this study aimed to investigate the psychometric characteristics of the metacognition questionnaire in teaching.

Methods:

In this cross-sectional study, the statistical population of the study included all teachers in 2020 which was selected by an available sampling method of 137 people. The data were collected using a teacher’s metacognition questionnaire (TMI). A Confirmatory, Pearson correlation coefficient was used to analyze the data. For this purpose, the use of SPSS-25 and LISREL software has been used.

Results:

of exploratory and confirmatory factor analysis indicate that. The validity and retest of the total score were calculated to be 0.93 and 0.86, respectively. The credibility of its subscales was also acceptable.

Conclusion:

As a result, it can be stated that the questionnaire has the proper psychometric properties for use in Iranian society and can be used as a valid tool for identifying teachers with teaching problems and teaching methods.

Introduction

Highlights

• The questionnaire has the proper psychometric properties for use in Iranian society.

• Can be used as a valid tool for identifying teachers with teaching problems.

• Can be used as a valid tool for identifying teaching methods.

Metacognition is a term first coined by Flavell in the field of memory. He regarded metacognition as cognition about cognition¹. From the perspective of Flavell (1979), metacognition is divided into two components metacognitive knowledge and metacognitive experience. Metacognitive knowledge is the knowledge that one has acquired about the human mind and its activities. This knowledge, like any other knowledge, may be linguistic, verbal, practical, or process (one knows how …). Flavell divides metacognitive knowledge into three categories of knowledge about individuals, tasks, and strategies. Metacognition can also mean an individual’s awareness of their thinking process and their ability to control this process^2,3. Metacognition is a cognitive model that operates at a higher level and is based on supervision and control⁴.

Successful people are those who plan well, identify and identify specific goals, and plan strategies. The most important advantage of metacognitive knowledge is that it enables the learner to be aware of his or her learning activity moment by moment and how his or her career progresses and to identify both its strengths and weaknesses⁵.

According to Hiver et al. ⁶ effective, metacognitive teachers act. Thomas (2012) states that while numerous researchers have investigated the importance of metacognition, this is not evident in the activities of teachers or teacher training teachers. He also emphasizes that it is unclear to what extent their teachers are metacognitive and insists on the need for further research on teacher metacognition because it can improve the effectiveness of their professional development activities⁷.

Various studies have acknowledged the importance and importance of teaching metacognitive skills to teachers, as well as the impact of this training on classroom teaching and school learning activities but at the same time, they raised some difficulties. Previous research has indicated inadequate studies on teacher education or metacognitive knowledge and skills. One reason for inadequate research on teachers’ cognitive skills may be the belief that cognitive change, if possible, relates to childhood and adolescence and that adults have reached the limit of their cognitive skills. Another reason that researchers are interested in is teachers’ reluctance to evaluate and disclose their cognitive problems that could compromise their educational authority^8–10.

Researchers claim that teachers who are familiar with the concept of metacognition do not have the resources to use it in teaching (time and specialized teaching materials); therefore, believe that this part of the literature on metacognition has a theory-practice gap: academic studies highlight the value of metacognition in learning, But the effort required to transfer this knowledge to regular classes is seldom made. For metacognition to make its way into mainstream education, policymakers need to make changes in the curriculum and teacher training that facilitate it^7,11 and concerning Zohar Research, (1999–2006) highlights the importance of teacher metacognitive knowledge; But teachers are having difficulty acquiring specific knowledge and metacognitive teaching in the field of developing scientific thinking. He similarly points to the research of Leo and his colleagues who challenge teachers about their metacognitive knowledge of higher-level thinking that is important to facilitate the transfer of this knowledge to their educational activities^12–14.

In the new metacognitive studies, there are two independent but interrelated elements; one is metacognitive knowledge and the other is metacognitive control^2,15. Metacognition in teaching can lead to effective teaching and successful student learning¹⁶.

Jiang, Ma, and Gao (2016) also developed a Teacher Metacognition Inventory (TMI) scale with a factor analysis approach. The TMI scale consists of the factors of teacher metacognitive experiences, metacognitive knowledge about pedagogy, metacognitive knowledge about the self, teacher metacognitive planning, and teacher metacognitive monitoring.

Teacher recognition and advertising of metacognition in teaching can lead to improved education. Therefore, validation of a suitable tool for the evaluation of metacognition in teaching is essential.

The TMI scale is a modified version of the Metacognition Inventory questionnaire. The teacher Metacognition Inventory Questionnaire consisted of 42 items on six scales (teacher metacognitive experiences, metacognitive knowledge about pedagogy, teacher metacognitive planning, and teacher metacognitive monitoring) validated by (Jiang, Ma & Gao, 2016) on 430 teachers and reported high validity¹⁷. According to the mentioned materials, this study aims to examine the psychometric characteristics of the metacognition questionnaire in teaching.

Method

The statistical population studied in the present study is all teachers in 2020. Given that our statistical population is large and the list of teachers was not available, 137 people were selected through available sampling. The method of implementation in the research is that in the first stage, a list of schools in different areas of (Ardabil-Iran) was prepared and a school was selected from each area of (Ardabil-Iran) and then a questionnaire was administered by the teachers of those schools. The work has been reported in line with the STROCSS criteria¹⁸.

Research tools

Teacher Metacognitive inventory (TMI)

This questionnaire is a modified version of the metacognition assessment questionnaire. The teacher Metacognition Assessment Questionnaire consists of 42 items in six scales (teacher metacognition experience, teacher metacognition knowledge about upbringing, teacher metacognition reflection, and teacher metacognition monitoring) which is standardized and validated by Jiang, Ma, and Gao (2016) on 430 teachers. Have reported high validity. Answers are based on a five-point Likert scale from strongly agree to strongly disagree. The validity coefficient of the questionnaire through the general formula of Cronbach’s alpha is 0.895¹⁷.

Metacognition questionnaire short form (MCQ-30)

This 30-item self-report was designed by Wells in 1997 to measure several characteristic metacognitive elements, some of which play a central role in the metacognitive model of psychological disorder¹⁹. This scale is based on the self-regulatory executive function model for emotional disorders and the metacognitive pattern of anxiety disorder. In the present study, Cronbach’s alpha results for the entire questionnaire were 0.91 and 0.8% ،, with positive scores about concern at 0.86, cognitive awareness or self-awareness at 0.81, memory or cognitive confidence at 0.80, and the need for Thought skills at 0.71²⁰.

Statistical analysis

To analyze the data, factor analysis, confirmatory factor analysis, through LISREL software, and Pearson correlation, SPSS-25 were used.

Ethical considerations

In this study, to comply with ethical principles, the satisfaction of all subjects was obtained to participate in the research. To maintain confidentiality, sufficient explanations were given to the subjects after assuring them of the confidentiality of the information.

Results

In this section, first, the data obtained are described and then data and inferential statistics are analyzed using appropriate tests

According to Table 1, descriptive statistics include sample size, mean, standard error mean, Median, standard deviation, variance, skewness for the total score, and each of the subscales of teachers’ metacognition in teaching, including teacher metacognition experience scale, and metacognition knowledge. The teacher is presented with upbringing, the reflection of teacher metacognition, and monitoring of teacher metacognition.

Table 1.

Descriptive statistics of the teacher metacognition scale and its subscales

	Teacher metacognition experience	Metacognitive knowledge about research	Metacognitive reflection	Teachers’ metacognitive knowledge	Teacher metacognition program	Teacher metacognition monitoring	Total score
N	137	137	137	137	137	137	137
X	20.21	16.38	29.26	16.51	12.94	22.38	11.707
S.E.M	0.26	0.18	0.30	0.18	0.24	0.53	1.13
MD	21.00	16.00	29.00	17.00	13.00	21.00	11.008
S	3.14	2.12	3.61	2.18	2.82	6.22	13.33
S2	9.90	4.50	13.06	4.78	7.95	38.81	17.797
Kurtosis	−0.19	−0.28	−0.13	0.02	1.73	2.69	0.78
Skewness	-0.31	0.04	0.64	0.38	4.34	7.77	2.77

S.E.M indicates standard error of the mean; MD, Median.

As can be seen in Table 2, the average of the materials varies in the range between 3.39 and 4.66. The minimum score that the subjects could get is a score of one and the maximum is 5. The average of Article 2 (I’m worried I can not control the speed of class teaching well) is 3.39. This average indicates that teachers have shown this item at a lower level. Also, the average of Articles 4, 22, 24, and 28 are equal to 4.45, 4.58, 4.53, and 4.66, respectively. These averages indicate that teachers have shown this syndrome at a high level.

Table 2.

Mean and standard deviation of TMI scale scores in each item

Number	Question content	X	SD
1	I always worry about student boredom in the classroom.	4.18	0.98
2	I’m worried about not being able to teach fast enough in class	3.39	1.34
3	I get anxious when I fail to teach.	3.66	1.29
4	I feel relaxed when I finish teaching in class.	4.45	0/97
5	I feel relaxed when I offer a satisfying training program.	4.54	0.90
6	I know for a fact that drama can create abstract knowledge.	4.19	0.73
7	I know the method of religious education can make students think.	4.03	1.21
8	I know that group discussions are not appropriate in cases where time is short for teaching.	3.80	0.96
9	I know well that interacting with students can keep them focused.	4.36	0.99
10	I evaluate the appropriateness of my teaching goals after each lesson.	4.31	0.87
11	After teaching, I think about whether the lesson planner is right or not.	3.85	0.99
12	After each lesson, I asked myself how I accomplished my learning goals.	4.26	0.73
13	After each lesson, I ask myself if I have explored other teaching methods.	3.88	1.01
14	I think about the teaching effect after each lesson.	4.36	0.73
15	After each lesson, I think about whether my teaching performance is good or not.	4.30	0.84
16	After each lesson, I check to what extent I have achieved my teaching goals.	4.31	0.74
17	I have a good knowledge of the concepts, principles, and methods of the subjects I teach.	4.16	0.94
18	I can quickly adjust my circumstances before starting the lesson.	4.23	0.83
19	I know very well why I have certain advantages in teaching.	4.01	0.74
20	I am aware of my weaknesses in teaching.	4.11	0.91
21	I prepare for unexpected situations that may arise in the classroom.	4.12	0.94
22	I always set a specific educational goal for each lesson.	4.58	1.37
23	I design a special training program for each lesson.	4.28	1.38
24	I noticed the change in my feelings in class.	4.53	1.29
25	Periodically, I review my teaching progress to see if it meets my expectations.	4.44	1.38
26	I ask myself, what do I do while teaching?	4.43	1.42
27	Periodically, while teaching, I ask myself if my teaching methods are appropriate.	4.34	1.43
28	I regularly check that students understand the material while I am teaching.	4.66	1.32

TMI, Teacher Metacognition Inventory.

Main findings in terms of research questions

Question 1: Is the teachers’ metacognition test in teaching (TMI) valid enough?

The internal consistency method was used to assess validity. The results are examined below. Considering that the test answers are determined as five-choice or five-point. To estimate the internal consistency of the test in this study, Cronbach’s alpha method was used.

As the results of Table 3 show, the test validity coefficient for the whole test is 0.84, and for the subtests of teacher metacognition experience, teacher metacognition knowledge about upbringing, the reflection of teacher metacognition, teacher metacognition monitoring, respectively 0.76, / 71 0.81, 0.78 and 0.79 were obtained. Measurement standard error for the whole test was 0.26 and subtests of teacher metacognition experience, teacher metacognition knowledge about upbringing, the reflection of teacher metacognition, and monitoring of teacher metacognition 0.26, 0.18, 0.30, 0.18, respectively. 0.24 and 0.35 were obtained which show that the standard deviation of the distribution of error scores in the above tests is low. As a result, small values ​​of measurement error indicate high reliability and almost low error. The criterion for judging error values ​​is the ratio of the number of test data and subtests.

Table 3.

Calculation of internal consistency (Cronbach’s alpha) and test measurement standard error

Scales and subscales	Cronbach’s alpha coefficients	Standard error measurement (SEM)
The whole test	0.84	0.26
Teacher metacognition experience	0.76	0.18
Metacognitive knowledge about breeding	0.71	0.30
Teachers’ metacognitive knowledge	0.81	0.18
Teacher metacognition program	0.78	0.24
Teacher metacognition monitoring	0.79	0.53

Question 2: Does the teachers’ metacognition test in teaching TMI have sufficient validity?

To answer the second research question as to whether the TMI test is sufficiently valid?

Simultaneous and structural validity methods were used to assess validity. The results are examined below. In this study, the Wells Metacognition Questionnaire was used to assess concurrent validity, the results of which are presented in Table 4. Pearson correlation test was used to calculate.

Table 4.

Correlation between the TMI scale and its subscales with wales metacognition scale

Scale	1	2	3	4	5	6	7
TMI
Teacher metacognition experience	0.48**
Metacognitive knowledge about breeding	0.52**	0.39**
Metacognitive reflection	0.57**	0.19**	0.31**
Teachers’ metacognitive knowledge	0.64**	0.29**	0.32**	0.55**
Teacher metacognition program	0.74**	0.10**	0.16**	0.13**	0.31**
Teacher metacognition monitoring	0.81**	0.13**	0.20**	0.20**	0.31**	0.84**
Metacognition of Wales	0.44**	0.36**	0.34**	0.71**	0.62**	0.86**	0.54**

TMI, Teacher Metacognition Inventory.

^**P <0.01.

As shown in Table 4, the correlation of the TMI scale with the Wells Metacognition Scale is 0.44. Also, the correlation of the six TMI subscales with the Wells Metacognition Scale is 0.36, 0.34, 0.71, 0.62, 0.86, and 0.54, respectively. These correlations indicate the high convergent validity of the TMI scale.

To estimate the validity of the test structure, two methods were used: one is the method of calculating the correlation coefficient of the test and subtests with each other and the other is the factor analysis, which we describe, respectively. To estimate the construct validity, the method of calculating the correlation coefficient between the test and subtests has been used. Theoretically, the test should have a high correlation with the subtests and the subtests should have a lower correlation with each other.

As Table 5 shows, the correlation coefficient of the TMI scale with six subtests of teacher metacognition experience, teacher metacognition knowledge about upbringing, teacher metacognition reflection, and teacher metacognition monitoring were 0.48, 0.52, 0.57, and 64, respectively. 0, 0.74, and 0.81 were obtained which show a high correlation and all subscales are significant. The TMI scale has a desirable and acceptable construct validity due to its correlation with its constituent subscales.

Table 5.

The correlation coefficient of the teacher’s metacognition scale with its subscales

Scale	1	2	3	4	5	6
MCQ
Teacher metacognition experience	0.48**
Metacognitive knowledge about breeding	0.52**	0.39**
Metacognitive reflection	0.57**	0.19**	0.31**
Teachers’ metacognitive knowledge	0.64**	0.29**	0.32**	0.55**
Teacher metacognition program	0.74**	0.10**	0.16**	0.13**	0.31**
Teacher metacognition monitoring	0.81**	0.13**	0.20**	0.20**	0.31**	0.84**

^**P <0.01.

MCQ, metacognition questionnaire

The construct validity of the teacher metacognition scale in teaching was assessed by confirmatory factor analysis using Lisrel 8.7. From the maximum probability method for estimating the model and from chi-square indices (χ²), chi-square-to-freedom ratio index (χ²/df), fit-fit index (GFI), adaptive fit-fit index (AGFI), index Comparative fit (CFI), root mean square root approximation (RMSEA) and mean square root (RMR) were used to fit the model.

As Table 6 and (Figure 1). shows, If the chi-square is not statistically significant, it indicates that the fit is very good, but this index is often significant in samples larger than 100 and therefore is not a suitable indicator to measure the fit of the model if the chi-square ratio of the degree of freedom is less than 3 indicates a very desirable fit. If CFI, AGFI, and GFI indices are greater than 0.90 and RMSEA and RMR indices are less than 0.05, it indicates a very desirable and very suitable fit and less than 0.08 indicates a desirable and appropriate fit. As a result, CFI, AGFI, and GFI indices indicate a very good fit and RMSEA and RMR indices indicate a good fit and are based on the ratio of χ² to the degree of freedom of normal fit.

Table 6.

The result of the calculated indices for the confirmatory factor analysis of the teacher metacognition scale.

$x^2$	$x^2/\mathit{df}$	GFI	AGFI	CFI	RMSEA	RMR
636.21	1.89	0.90	0.89	0.90	0.08	0.05

AGFI, adaptive fit-fit index; CFI, index comparative fit; GFI, fit-fit index; RMR, mean square root; RMSEA, root mean square root approximation.

Figure 1.

Standard factor analysis model.

Discussion and conclusion

Cronbach’s alpha method was used to estimate the validity coefficient of the test and the retest method was used to examine the internal consistency of the test and subtests. The validity coefficient in this study was acceptable for the whole test and subtests. The correlation coefficient obtained from the retest method showed that the test has relatively good accuracy for measuring the desired domains. The results were satisfactory and satisfactory compared to the results reported in the main form of the questionnaire. Since the statistical population of the present study consists of teachers. Therefore, the above results are to be expected because it is expected that teachers will not have much difficulty in comprehension and abstraction skills.

The teacher metacognition questionnaire in teaching was examined in terms of content validity and its translated form was approved by experts in this field; in the opinion of experts, its formal and logical validity was confirmed and was deemed suitable for implementation.

Also, the present study is consistent with the original research conducted by (Jiang, Ma, and Gao, 2016)¹⁴. Also, the main questionnaire has high validity and reliability (0.895).

The results of factor analysis in the subscales of the questionnaire showed that it has good validity. In general, considering the analytical results of the teacher metacognition factor, it has sufficient validity. The results obtained are to a large extent consistent with the theoretical framework of the main form of the convergence test, and the results of confirmatory factor analysis indicated this claim.

Conclusion

As a result, it can be stated that the questionnaire has the proper psychometric properties for use in Iranian society and can be used as a valid tool for identifying teachers with teaching problems and teaching methods.

Limitations

Although the sampling adequacy index was desirable and enough care was taken in selecting the research community, which was teachers with different cultures, it seems that a very large sample had to be selected. This test has been studied only in (XXX) schools with the teachers of this city, therefore the normative and statistical results of this study can be used only in (XXX) and the results of its application in other cities and schools should be interpreted with caution.

Suggestions

Due to the importance of standard and valid tools, it is suggested that the test be standardized at a much wider level in different cultural and social contexts. Due to the appropriate psychometric properties of the teacher metacognition scale, it is suggested to use this scale to improve teaching.

Ethical approval

This study is approved by the Research Vice-Chancellor of Islamic Azad University of Ardabil. The registration was a unique identifying number (http://thesis.iauardabil.ac.ir/Public/Thesis_Defense/Thesis_Defense.aspx?Date=1397/06/18).

Consent

Written informed consent was obtained from the participants. A copy of the written consent form is available for review by the editor of this journal upon request.

Source of funding

The current study received no funding.

Author contribution

All authors contributed to the design and implementation of the study.

Conflicts of interest disclosure

The author declares no conflict of interest.

Research registration unique identifying number (UIN)

This is not an interventional human or animal study. Our study is not an intervention or clinical trial, it is a survey of teachers and an observational study, so it is not registered. The Iranian Clinical Trial Registration Center only registers interventional articles due to the compatibility of the IRCT database with the WHO portal and does not register observational articles. However, it has been registered on the website of Ardabil Azad University and has been approved by the Research Vice-Chancellor of Ardabil Azad University. http://thesis.iauardabil.ac.ir/Public/Thesis_Defense/Thesis_Defense.aspx?Date=1397/06/18.

Guarantor

All authors accept full responsibility for the study.

Data availability

Data are available from authors on request.

Provenance and peer review

Not commissioned, externally peer-reviewed.