Designing and Validation of Self-Assessment Tool of Professional Competency and Its Psychometrics in the Administrators of Preschool Educational Centers

Abstract

Background:

The role of preschool administrators in raising children is increasing day by day, so the purpose of this study was to detect the validity and reliability of the professional competency questionnaire and its psychometric assessment in preschool administrators.

Methods:

This descriptive study was conducted on 380 preschool administrators with a minimum age of 25 and a maximum of 65 year participated in 2018. The sampling method was stratified from four regions of north, south, west and east of Tehran, Iran. Rosenberg self-esteem questionnaire was used for convergent validity. Cronbach's alpha coefficient was used to examine the internal consistency. Face validity, content validity and structural validity were also calculated. The data collection tool was a researcher-made questionnaire.

Results:

The questionnaire of professional competency of preschool administrators was appropriate in terms of face validity. Indicators related to content validity were appropriate in terms of relevance, transparency and necessity in this study. Cronbach's alpha coefficient was 0.98. The correlation between the total score of the managers’ professional competency test and self-esteem was equal to 0.08 with a significance level of 0.58.

Conclusion:

The professional competency questionnaire of preschool administrators had good validity and reliability to assess the level of professional competency of preschool administrators. Researchers can use this questionnaire to assess the professional competency of preschool administrators.

Introduction

Management, guidance and control of a specific activity to perform it in a predetermined context (1). Improving or developing the management of an organizational structure to perform the assigned tasks, leading and motivating employees are other management functions (2). Therefore, organizations try to attract, by training, and try to keep successful administrators in promoting the organization (3). Administrators are people who influence the future of the organization and their competence and decisions are effective in managing the organization and its successful achievements (4). Therefore, in order to achieve the desired results and efficient management of the organization, administrators must be aware of new challenges such as advances in communications and information technology, increasing competition due to globalization and the strategic importance of management perspective and reveal the potential talents of individuals (5).

One of the management sectors that has a profound and influential effect on social, political and economic systems is management in education (6). Efficient management prevents incurring excessive costs, improving service quality and facilitating workforce planning (7). Educational administrators are responsible for the financial situation, staff and courses of their schools, as well as for guiding and supervising teachers during their educational activities (8). Preschool management needs professional expertise and training in this field, and administrators must advance it by setting their own goals and using their own philosophy and policies (9). Preschool course is the first social environment that a child experiences after home (10). In this period, according to the facilities and requirements of each country, children aged three to six years are studied (11). At these ages, children go through important and sensitive stages of their lives in terms of personality, social, physical, emotional and educational, linguistics, and children in preschool environments have a greater improvement in learning and meeting the needs of the next generation (12,13). In recent decades, the enrollment of children in preschool has increased in most countries of the world (14). Countries with moderate global per capita incomes are increasingly valuing preschool education to prepare their children for life (15).

Today, working with children, preschool teacher and parents at different levels of preschool (care centers and kindergartens) requires careful and scientific supervision, Monitoring and management (16). Preschool administrators are responsible for maintaining, protecting, nurturing, hygiene and preventing school violence as one of the main problems in school security (17). Therefore, preschool administrators must also have competencies. Competencies are measurable characteristics of an individual that are related to success in work (18) and can be defined as the basic characteristics of an individual that are related to superior performance in a job position (19,20).

Various studies in professional competency have defined competencies as effective attributes, skills, behaviors and values for the organization (21,22) without restricting this definition to a specific group or organization. In this regard, organizations have been designing and applying the competency model for more than three decades to explain the basic criteria of a particular job (23). Competency-based management is an integrated process that puts people in an appropriate organizational position in accordance with their abilities, capabilities and work skills (23) and expands professional competencies and special abilities to be effective in job performance (24).

The identification of competencies in society has led to the creation of a competency model as a characteristic for a particular situation (25). Awareness of the professional competency of administrators can help in understanding the competence and factors related to the effective performance of leadership in schools (26).

Khajei (27) in a research has considered the most important competence of educational administrators in three parts: technical, human and perceptual. Some researchers reached the conclusion that administrators have more moral-personality competence than other competencies (28–30). There was a positive correlation between managers’ competence and their managerial background (31,32).

There was a significant relationship between the field of study of administrators and students’ academic achievement (33). However, another study between the level of education and the performance of administrators has not shown a relationship (34).

So far no tool has been developed to check the qualifications of the administrators of preschool educational centers in a comprehensive and integrated manner, although people in their research have used researcher-made tools to evaluation the competence of school administrators and preschool centers, but none of these tools have examined different aspects of competence, so the lack of tools Suitable for assessing the professional competency of preschool administrators is one of the main problems in the education system of the country (34–36).

Therefore, we aimed to be used as a possible solution to identify the skills of the administrators of these centers in Iran by validating and relying on the professional competency tools of pre-school administrators.

Materials and Methods

The statistical population includes all preschool administrators in 2018. To sample them, first in coordination with the Welfare Organization of Tehran Province, questionnaires provided to administrators in person, but according to the dispersion of preschools, sampling these people was done in a classify manner from four regions north, south, west, East with a sample size of 380 qualified preschool administrators.

Tools

Preschool administrators professional competency Self-Assessment Questionnaire: A researcher-made questionnaire for preschool administrators professional competency self-assessment includes 11 scales (knowledge and training skills, organizational skills, staff management, technology, educational planning, safety and compliance, Communication and relationship management, parent and family support, financial management, professional participation and personal management) based on the Likert scale (total score) from 1 (complete mastery), 2 (strong), 3 (moderate), 4 (Medium to low), and 5 (weak).

The items of each spectrum are prepared based on indexing and operationalizing managerial competence. In addition, the validity of this tool is based on content validity. In other words, after reviewing the topics and consulting with experts, the researchers found that the indicators included in the questionnaire questions were exactly related to the concept in each competency, and the sum of the scores of individuals in each comparison shows how competency the individual is desired. To set up this questionnaire, a set of different items has been prepared for assessment. Then, based on the votes of the three judges, appropriate items were selected to assess the ability of administrators in each field. With a total of selected items, a questionnaire consisting of 120 items has been prepared. This questionnaire has been pre-tested in a sample of 25 preschool administrators. After this test, the results obtained were analyzed by SPSS version 20 software (IBM Corp., Armonk, NY, USA) and finally, by removing the redundant items to achieve the appropriate level of validity, a questionnaire with 76 questions was prepared. Cronbach's alpha coefficient was 0.98.

Rozenberg Self-Esteem: The Rosenberg Self-Esteem Scale was an English edition of Rosenberg in 1965. The scale includes ten self-report items that positively express general feelings of worth or acceptance. Moreover, each statement of this scale contains two scales (I agree, I disagree). Then we got the algebraic sum of the total scores. A score of +10 indicates very high self-esteem and a score of −10 indicates very low self-esteem.

The relationship between individual and collective self-esteem in a sample of 82 students with P>0/01, r=0/34. Cronbach's alpha coefficient in a study of female students was 0.93 and in the reliability of the retest test was r=0/85 (37).

The study is extracted from the student dissertation and has been approved by the ethics committee of the University of Social Welfare and Rehabilitation Sciences with the code IR.USWR.REC.1396.89.

Data Analysis

A simple logistic Rush model and version 3.76 of Winestep software were used to analyze the data in the framework of the Rush approach and to estimate the psychometric properties of the professional competency test of preschool administrators.

This software offers two types of MNSQ and Zstd statistics to check the model fit and the question fit. The range of MNSQ statistics is from zero to infinity and its expected value is 1. Therefore, due to the high volume of the sample used in this study and also considering the recommendation to investigate the fit of the question with the model, the MNSQ statistic was used estimated and reported in two forms of external and internal fit. A value less than 2 indicates the fit of the model with the data. Therefore, in the present study, the internal fit statistic was used to examine the fit of the question. The acceptable range for this statistic is from 0.7 to 1.3.

To check the appropriateness of the questions, two methods of question fit statistics and question Performance Difference Analysis (DIF) were used. To check the fit of the questions, two indicators outfit MNSQ and infit MNSQ were used. For grading scales, (0.6–1.4) is suggested as the standard distance for MNSQ. Questions for which the outfit MNSQ or infit MNSQ is out of range are considered questions that do not fit the model well. DIF analysis is an analysis that examines the same performance of questions in different groups. DIF analysis identifies questions that, after controlling for ability, appear to be very difficult for some groups and very easy for others (38). In this study, DIF analysis with different backgrounds, different education as well as age groups were used.

To classify the questionnaire questions based on the rationality of the structures, the structural validity of the questionnaire was used through factor analysis Principal Axis Factoring (PAF) method. Reliability was performed by internal consistency method of Cronbach's alpha index. To evaluate the adequacy of the model, Kaiser-Mayer-Olkin (KMO) index and Bartlett test results were used.

Results

This study was performed on 387 preschool administrators, of which 2.9% were male and 97.1% were female. Most of them (35.7%) were between 30 and 40 yr old, and in terms of work experience, people with 5–10 yr of work experience had the highest frequency (32.82%) and most participants (64/08%). had a bachelor's degree (Table 1). The results of exploratory factor analysis showed that the value of KMO statistic is equal to 0.97 which is higher than 0.7 and shows that the data are suitable for factor analysis. Bartlett test was used to evaluate the significance of the correlation matrix of the research data. Bartlett statistic was 28244.23, freedom was 2850 and significance level was 0.001. Factor analysis can be done. By performing factor analysis on the scale questions, a general factor with an eigenvalue of 40.36 and an explanatory variance of 53.11 was obtained.

Table 1:

Frequency distribution of participants

Education	Frequency	Percentage	Work Experience	Frequency	Percentage	Age	Frequency	Percentage
Diploma	23	5.94	More than 20 yr	49	12.66	20–30	64	16.5
Associate Degree	18	4.65	15–20 yr	97	25.06	30–40	138	35.7
Masters	248	64.08	5–10 yr	127	32.82	40–50	107	27.6
Masters	89	23.00	Less than 5 yr	97	25.06	50 yr and up	44	11.4
Doctorate	6	1.55	No answer	17	4.40	No answer	34	8.8
No answer	3	0.78	Total	387	100/0	Total	387	100/0
Total	387	100.0

To examine the internal consistency, Cronbach's alpha was calculated and the Cronbach's alpha coefficient was equal to 0.98. The acceptable value of Cronbach's alpha was between 0.70 and 0.95 and according to Cronbach's alpha in the present study, the questionnaire had a very good internal reliability.

The person's credit in the professional competency test of preschool administrators was equal to 0.98 and Separation was equal to 6.93. Moreover, the validity of the question was equal to 0.96 and the separation was equal to 5.10.

According to the fit indices and the knowledge and difficulty function of the questions, the fit of each answer option, which was from 1 to 5, was also tested through two external and internal fit indices. The professional competency test of pre-school administrators has a range of 5 options and the probability of choosing options 4 and 5 is more frequent among most participants, and the external and internal fit indices also confirmed this (Table 2).

Table 2:

Response options fitting indicators

	Options	1	2	3	4	5
MNSQ	INFIT	1.31	0.83	0.92	0.87	1.05
	OUTFIT	1.46	0.85	1.01	0.87	1.11

Fig. 1 shows the specific curve of the question, the test awareness function, and the questionability map according to the same scaling.

Fig. 1:

Question-specific curve, test awareness function, and question-ability map

The specific bend of the test in Fig. 1-A during the estimated abilities indicates that test awareness peaks at the zero ability point, and people at this level of ability have the least standard estimation error. In other words, the accuracy of the measurement at this level of ability is at its highest. Fig. 1-B shows the test awareness function along the scale from lowest to highest score. As mentioned above, the closer the scores are to zero, the lower the estimation error. According to Fig. 1-C, the average questions and difficulties are different from each other. Although this difference is not large, but the average difficulty of the questions is higher than the abilities.

The correlation between the total score of administrators’ professional competency test and self-esteem test was equal to 0.08 with a significance level of 0.58, which shows that there is no relationship between the total score of this test and self-esteem and indicates that these two tests Different structures are measured so divergent validity is established.

The average of people with more than 20 years of work experience is higher than other people and people with less than 5 years of work experience have the lowest score in the test. The results obtained through Levin test showed the assumption of homogeneity of variance. The results of analysis of variance between people with different experience (P=0.001, F_3.366 = 8.37) showed that there was a difference between groups with different work experience, which means that people with different work experience in the qualification test. Professional administrators receive different scores, and this difference is significant (Table 3).

Table 3:

Descriptive statistics, results of homogeneity test of variance and results of analysis of variance of people with different work experience

Work experience	Frequency	Percentage	M	S.D	df	FLevin	P	F	P
More than 20 yr	49	12.66	297.14	60.49	3	0.98	0.40	8.37	0.001
15–20 yr	97	25.06	288.43	64.88	366
5–10 yr	127	32.82	283.76	65.78
Less than 5 yr	97	25.06	250.38	67.01
No answer	17	4.40	-	-
Total	387	100.00	278.01	67.13

Note: M: Average; S.D: Standard deviation

People with doctoral degrees got the highest average score and people with diploma degrees got the lowest average score. Homogeneity of variance was tested by Levin test, and the results showed that the assumption of homogeneity is valid. The results of analysis of variance between people with different education (P=0.001, F_4.379=5.30) showed that the difference between them is significant, meaning that people with different education get different scores in the professional competency test of administrators (Table 4).

Table 4:

Descriptive statistics, results of homogeneity test of variance and results of analysis of variance of people with different education

Education	Frequency	Percentage	M	S.D	df	FLevin	P	F	P
Diploma	23	5.94	241.65	73.08	4	2.71	0.13	5.30	0.001
Associate Degree	18	4.65	247.28	74.72	379
Masters	248	64.08	274.35	69.24
Masters	89	23.00	297.70	54.02
Doctorate	6	1.55	314.83	43.18
No answer	3	0.78	-	-
Total	387	100.00	277.16	67.71

Note: M: Average; S.D: Standard deviation

People aged 40 to 50 yr have the highest average score and people aged 20 to 30 yr have the lowest average score. The results of Levin test also showed that the assumption of homogeneity of variance is established. Moreover, the results of analysis of variance between people of different ages (P=0.01, F_3.349=8.37) showed that there is a difference between groups with different ages, which means that people with different ages get different scores in the professional competency test of administrators and this difference is significant (Table 5).

Table 5:

Descriptive statistics, results of homogeneity test of variance, and results of analysis of variance of people of different ages

Age (yr)	Frequency	Percentage	M	S.D	df	FLevin	P	F	P
30–20	64	16.5	256.69	61.70	3	2.02	0.11	4.13	0.01
40–30	138	35.7	270.96	73.13	349
50–40	107	27.6	290.26	60.90
50 yr or older	44	11.4	288.14	68.28
No answer	34	8.8	-	-
Total	387	100.00	276.36	67.88

Discussion

The findings of the present study confirmed the psychometric properties of the professional competency self-assessment questionnaire of pre-school administrators. Cronbach's alpha coefficient was 0.98. The acceptable value of Cronbach's alpha is between 0.70 and 0.95 and considering that Cronbach's alpha in the present study is higher than this value, the questionnaire had a very good internal reliability. The content validity of the self-assessment questionnaire of professional competency of preschool administrators was confirmed based on Judgment of experts. Divergent validity of the self-assessment professional competency questionnaire of pre-school administrators was calculated through the simultaneous implementation of the self-esteem scale. The correlation between the total score of the managers’ professional competency test and self-esteem was 0.08 with a significance level of 0.58, which shows that there is no relationship between the total score of this test and self-esteem. The lack of correlation between the two tests indicates that these two tests measure different structures, so divergent validity is established.

Moreover, the reliability and validity of the items of the professional competency self-assessment questionnaire with limitations. Those limitations become more visible especially in the context of examining the types of validity of a scale, which is an ongoing process. Therefore, in order to complete the validation process of the self-assessment questionnaire of professional competency of pre-school administrators, more research is needed. Research on the psychometric properties of the preschool administrators’ Professional Competency Self-Assessment Questionnaire on various examples are other steps that need to be taken.

The mean of all subjects was 277.47 and the standard deviation was 67.56. Moreover, the average weight for all people was 3.65, which is very close to the desired range of 3.67, so the questions measure the desired structure well with appropriate items.

The situation of primary school administrators in terms of having professional managerial competencies and skills to the desired level. Moreover, there is a significant positive relationship between the level of managers’ professional competencies and managerial skills and job commitment and attachment in teachers (35). In this study there was a positive correlation between managers’ competence and their managerial background. There was a positive correlation between the competence of managers and their managerial history (31, 32). However, some other studies have not shown a relationship between the years of study and the performance of administrators (33, 34).

Conclusion

Finally, administrators with less work experience were less professionally qualified. Therefore, those in charge use in-service training to improve the professional competency of preschool administrators. Regarding the managerial competence of preschool administrators in Tehran which is low, the selection of administrators in terms of managerial competence be done more carefully. Organizations that are supported and supervised by preschool centers, by concluding contracting with educational and academic institutions, encourage administrators to raise the level of knowledge and skills needed.

Journalism Ethics considerations

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.