Evaluation of psychometric properties of scales measuring student academic satisfaction: A Systematic review

Abstract

BACKGROUND:

Student satisfaction has an impact on student motivation, recruitment of new students, and retention of existing students. Hence, it is important for researchers and academic institutes to assess student academic satisfaction by valid and reliable scales. The aim of this study was to rigorously assess methodological quality and psychometric properties of scales measuring student academic satisfaction.

METHODS:

In this systematic review, databases including Scopus, PubMed, ProQuest, ScienceDirect, and Web of Science, and two Persian databases were searched using relevant keywords such as academic satisfaction, student satisfaction, university satisfaction, campus satisfaction, academic life experience, validation, and psychometric and factor analysis from 1970 to December 2018. Considering eligibility criteria, studies were selected after titles and abstracts screening. The methodological quality assessment was performed by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist and the Terwee quality criteria.

RESULTS:

Of 814 retrieved articles, 13 studies were included in the study. Based on the COSMIN checklist, structural validity (84%), content validity (53%), and hypothesis testing (53%) were the most reported properties. One study reported cross-cultural validity, one for criterion validity, and none reported measurement error.

CONCLUSION:

The results of our study showed that in spite of ≥48 years of development in student satisfaction scales; however, each scale has at least one “poor” psychometric property based on the COSMIN checklist. Quality appraisal of scales is necessary after developing and performing psychometric process.

Introduction

Students are the most important and main output of the universities, and it is necessary to identify what is important to students.[1] In this regard, to improve the quality of academic services, and adopt appropriate educational policies for students, continuous monitoring of student satisfaction is imperative. According to dynamic education environment, the results of student satisfaction help higher education institutions to remain in competitive situations.[2,3,4]

Satisfaction is customer's pleasure resulted from services provided by the organization.[5] In academic setting, student satisfaction definition was referred to Oliver and DeSarbo “the favorability of a student's subjective evaluation of the various outcomes and experiences associated with education.”[6] Academic satisfaction is also defined as “Short-term attitude that results from the evaluation of student experiences with the education service received,” and this attitude has an impact on student motivation, recruitment of new students, and retention of existing students.[7]

Student satisfaction is correlated with some academic outcomes.[8] Previous studies reported the relationship between student satisfaction, retention in the field of the study, and academic achievement.[3,9,10] Furthermore, evidence showed that student satisfaction leads to academic success that improves student academic motivation; in other words, student with higher academic satisfaction has higher motivation and try more for top grades.[9]

Student academic satisfaction is a multifaceted concept;[11] Lent et al. reported a social cognitive model in engineering students that academic satisfaction would be directly predicted by self-efficacy, outcome expectations, environmental supports, and perceived goal progress.[12] Alves and Raposo tested a conceptual model of student satisfaction in higher education and found that variables image, expectation, quality, and value influence in student satisfaction that the consequences of them were student loyalty and word of mouth from student to student.[13]

Studies have shown that there are significant differences among academic satisfaction of students from different academic level, field of study, country of study, and time of satisfaction assessment. Student satisfaction of postgraduate students was different from undergraduate students, because of maturity, academic ability, their experience and expectations of their educational experience.[14] Regarding the field of study, field of nursing needs more interest and competency,[5] and evidence showed that they had different academic satisfaction levels compared to other students.[9] It should be noted that there are significant differences in the educational system and student satisfaction in each society.[15] Furthermore, as the concept of academic satisfaction depends on the educational structure of universities, it is expected that this concept changes overtime.

According to these factors, there are various scales that have been developed in university student; some are general[16,17,18,19,20,21,22,23] and some are for specific groups such as nursing students,[2,24,25] international students,[26] and sport students.[27] Regarding multidimensional nature of academic satisfaction concept, scales are different in dimensions, and some scales do not cover all the dimensions of academic satisfaction. Hence, it becomes necessary to conduct a systematic review for evaluating the psychometric properties of scales that measuring academic satisfaction for proper selecting and better using of them in academic setting. The aims of this systematic review are to:

1. Identify scales that investigate university student academic satisfaction

2. Assess the methodological quality of included studies

3. Analyze the psychometric properties of the scales.

Methods

A systematic review of studies that evaluate the psychometric properties of academic satisfaction scales was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

Search strategy

Electronic databases were searched including Scopus, PubMed, ScienceDirect, and Web of Science, two Persian databases such as SID (https://www.sid.ir/) and MAGIRAN (http://www.magiran.com/) and finally, Google Scholar as a search engine from 1970 to 16 December 2018. Furthermore, ProQuest database was searched to identify relevant theses. Reference lists of all identified articles were also hand searched.

Keywords used in the search were as follows: academic satisfaction, student satisfaction, university satisfaction, campus satisfaction, academic life experience, validation, psychometric, and factor analysis. Persian meaning of “Student Satisfaction” and “Academic Satisfaction” was searched in Persian databases. Keywords used in the search for the different databases are provided in Table 1.

Table 1.

Keywords used in the search for the different databases

Databases	Search string
PubMed	(“Factor analysis” OR validation OR psychometric*) AND (“academic satisfaction”[TIAB] OR “student satisfaction”[TIAB] OR “university satisfaction”[TIAB] OR “campus satisfaction”[TIAB] OR “academic life experience”[TIAB])
Scopus	(TITLE-ABS-KEY [“factor analysis” OR validation OR psychometric*] AND TITLE-ABS-KEY [“academic satisfaction” OR “student satisfaction” OR “university satisfaction” OR “campus satisfaction” OR “academic life experience”])
ISI	TOPIC: (“factor analysis” OR Validation OR psychometric*) AND TOPIC: (“academic satisfaction” OR “student satisfaction” OR “university satisfaction” OR “campus satisfaction” OR “academic life experience”)
ProQuest	ti (validation OR psychometric) AND ti (student satisfaction OR academic satisfaction OR university satisfaction)
ScienceDirect	Psychometric* AND (“student satisfaction” OR “academic satisfaction” OR “university satisfaction”)

Eligibility criteria and selection procedure

Published articles in English and Persian that describe the scales’ psychometric properties/validation process/cross-cultural evaluation of student satisfaction about academic career in university student were included in this study. Articles with irrelevant subjects (student satisfaction about specific teaching method or training courses), language other than English or Persian, structural equation model or model testing articles, review/systematic review articles, and conference articles were excluded from the study. EndNote (version X8; Thomson Reuters, New York, NY, USA) was used to initially screen for duplicated results. Two authors independently involved screening titles and abstracts in the first stage. Full texts of included articles were assessed carefully for eligibility. Any discrepancy between authors was resolved through joint discussions.

Data extraction

Data extraction was independently conducted by two researchers (one statistical expert and one expert in concept of the study). A data extraction sheet included: first author name, publication year, name of scale, country, target population (students’ major), face validity, content validity, construct validity (sample size, factor extraction method, rotation methods, selection of the number of factors, name of factors, and total variance), and reliability (consistency: Cronbach's alpha coefficient, stability: Spearman's correlation coefficient, and interclass correlation (ICC) coefficient).

Quality assessment

Two researchers assessed the full texts of articles for methodological quality on the basis of the checklist proposed by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN). The COSMIN checklist assesses different psychometric properties (A = internal consistency, B = reliability, C = measurement error, D = content validity, E = structural validity, F = hypothesis testing, G = cross-cultural validity, H = criterion validity, and I = responsiveness). To analyze the results obtained, a four-point COSMIN score was used. Each item was classified as “excellent” when there was appropriate methodology, “good” when there was insufficient relevant information, but an adequate level of quality was reached, and “fair” when the applied methodology was questionable and “poor” when there was evidence that the methodological process was not correct. A methodological quality score per box is obtained by taking the lowest rating of any item in a box (“worst score counts”).[28] The quality criteria for measurement properties were analyzed according to the criteria of Terwee study.[29] Inter-reviewer consensus was evaluated according to the Cohen's Kappa value. Any discrepancies were resolved through discussion and consensus.

Data synthesis

Because the overall analysis of psychometric properties is not possible, a narrative analysis was carried out based on the characteristics of the included articles.

Results

Study characteristics

As shown in the PRISMA flow chart [Figure 1], 814 articles (42 articles from Persian database + 772 articles from English language databases) were found in the initial search. After excluding duplicated and irrelevant studies, 13 studies remained.[2,16,17,18,19,20,21,22,23,24,25,26,27]

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart

Included studies were published from the year 1970 to 2017, and majority of them were in the year 2012 (n = 4). One study was doctoral thesis[19] and other was peer-review original articles that published in journals. Only one study was published in the Persian language.[21] Half of the studies (n = 7) were conducted in the USA,[16,17,19,20,22,23,24] followed by Iran (n = 1), Canada (n = 1), Brazil (n = 1), China (n = 1), India (n = 1), and Pakistan (n = 1). Majority of studies focused on college students (n = 5), undergraduate students (n = 5), and three articles were conducted on nursing students [Table 2].

Table 2.

Psychometric characteristics of included studies

First author (year)	Scale	Country	Target population	Face validity	Content validity	Construct Validity					Reliability
						Sample size	Factor extraction method (rotation)	Selection of the number of factors	Name of factor	Total variance (%)	Consistency	Stability
Liu (2017)	Professional Sport University Student Satisfaction Survey Scale	China	Sport Management Student	psychology experts (n=5)	58-item 6-dimension 7-point Likert scale	2715	EFA (Oblique) CFA Maximum Likelihood KMO=0.97 Bartlett’s test	Scree Plot Eigen value Kaiser criterion	F1: Teaching and learning F2: Logistics services F3: Internship and career F4: Learning and scientific research environment F5: Academic and cultural life F6: Student management and guidance	68.54	Total α= 0.98 F1: α= 0.95 F2: α= 0.91 F3: α= 0.95 F4: α= 0.93 F5: α= 0.92 F6: α= 0.92	NR
Chadha (2017)	International Students’ Satisfaction with Higher Education INHedPERF (Indian Higher Education Performance Model)	India	international students Sciences or Technology stream e.g. B.Sc., Engineering, IT	NR	45-item 9-dimension 5-point Likert scale	251	PCA (Varimax) CFA KMO=0.91 Bartlett’s test	NR	F1: Faculty F2: Admin Staff Support F3: Campus facilities & Upkeep F4: Understanding F5: Cost F6: Course Conduct F7: Resource Adequacy F8: Utilities Support F9: Safety & Security	63.33	Total α= 0.95 F1: α= 0.89 F2: α= 0.87 F3: α= 0.75 F4: α= 0.72 F5: α= 0.68 F6: α= 0.76 F7: α= 0.73 F8: α= 0.64 F9: α= 0.59	NR
Hirsch (2016)	cross-culturally adapt and validate the Nursing Student Satisfaction Scale (NSSS)	Brazil	Undergraduate Nursing Student	Conducted by expert panel (n=4)	22-item 3-dimension 5-point Likert scale CVI=0.83 30 nursing students from masters and/or PhD	123	PCA (Varimax) KMO=0.88	factor loadings (>.400); and degree of subjectivity	F1: curriculum and teaching F2: professional social interaction F3: learning environment	59.54	Total α= 0.93	NR
Torkzade (2014)	Student’s Academic Satisfaction scale	Iran	Undergraduate Student	-	59-item 6-dimension 5-point Likert scale	266	EFA, CFA	NR	F1: academic F2: Teachers F3: classmates F4: promotion and educational progress F5: educational evaluation F6: educational environment	NR	F1: α= 0.89 F2: α= 0.92 F3: α= 0.94 F4: α= 0.92 F5: α= 0.91 F6: α= 0.90	NR
Zhai (2012)	Community College Student Satisfaction	USA	College Student	Expert panel and student	60-item 11-dimension 5-point Likert scale expert panel	558	PCA	eigenvalues scree plot factor loading greater correlation with other factors conceptual meaningfulness	F1:curriculum and instruction F2:counseling/Advising F3:facilities F4:campus climate F5:orientation F6:academic development F7:personal development F8:availability of classes F9:financial aid and fees F10:admissions and course registration F11: follow up of academic progress	46.9	Total α= 0.96 F1: α= 0.90 F2: α= 0.92 F3: α= 0.82 F4: α= 0.81 F5: α= 0.84 F6: α= 0.83 F7: α= 0.82 F8: α= 0.77 F9: α= 0.72 F10: α= 0.73 F11: α= 0.75	NR
Dennison (2012)	Undergraduate Nursing Student Academic Satisfaction Scale (UNSASS)	Canada	Undergraduate Nursing Student	Conducted by nursing students (n=22)	48-item 4-dimension 5-point Likert scale CVI=0.83	313	EFA-PCA (Varimax)	Eigenvalue, scree plot, factor loading scores	F1:In-class Teaching F2:Clinical Teaching F3:The Program F4:Support and Resources	50.12	Total α= 0.96 F1: α=0.92 F2: α=0.91 F3: α=0.91 F4: α=0.74	Gutman coefficient: 0.9 between-forms correlation coefficient: 0.82 overall ICC=0.88 (2-Week)
Hussain (2012)	Student University Satisfaction Scale (SUSS)	Pakistan	Graduate and master student	Focus group by students (n=15)	32-item 6-dimension 3-point Likert scale Expert panel	73	NR	NR	F1:Learning Facilities F2:Curriculum F3:Teaching and Learning F4:University Climate F5:Administrative Facilities F6:Policies and Procedures	-	Total α= 0.91	NR
Chen (2012)	Nursing Student Satisfaction Scale (NSSS)	USA	Associate in Science in Nursing or Associate Degree in Nursing	Conducted by students (n=18)	31-item 4-dimension 6-point rating scale Expert panel	303	PCA (Varimax)	Eigenvalue, scree plot, conceptual consideration	31 items: F1: Curriculum F2: Faculty F3: Social interaction F4: Environment	57.6	Total (30 Item) α= 0.93 F1: α=0.85 F2: α=0.87 F3: α=0.88 F4: α=0.86	-
Zullig (2005)	Brief Multidimensional Students’ Life Satisfaction Scale	USA	college students	-	40-item 5-dimension Criterion-related validity with Health Related Quality of Life Scale	522	principal axis factor analysis & Known groups analysis	Scree Plot Eigen value	F1:Family F2: Friendships F3:School F4: Self F5:Enviroment	NR	Total α= 0.78	NR
Juliatre (1996)	Student Satisfaction Inventory (SSI)	USA	college student	student	82-item 11-dimension 7-point Likert scale Expert panel	4974	maximum likelihood and principal components factor analyses (oblique)	Eigen value	F1:Campus Climate F2: Campus Organizations and Activities F3:Responsiveness to Diverse Populations F4:Curriculum and Instruction F5:Financial Aid/Billing F6:Campus Support Services F7:Academic Advising F8:Resident Life F9: Student Acclimation F10:Safety and Security F11: Faculty Effectiveness	58.5	Total α= 0.98 F1: α= 0.94 F2: α= 0.79 F3: α= 0.89 F4: α= 0.82 F5: α= 0.85 F6: α= 0.85 F7: α= 0.98 F8: α= 0.70 F9: α= 0.90 F10: α= 0.70 F11: α= 0.83	3-week R=0.84
Derry (1978)	Academic program (program evaluation survey (PES))	USA	Undergraduate and graduate Student	-	24-item 3-dimension 5-point Likert scale	2752 (under graduate) + 1108 (graduate)	Simple common factor analysis (varimax and aoblique)	eigenvalues scree plot	F1: student perception of value in program F2: student satisfaction with instruction F3: student satisfaction with faculty mentorship and overall satisfaction	NR	undergraduate F1: α= 0.80 F2: α= 0.72 F3: α= 0.80 Graduate F1: α= 0.80 F2: α= 0.75 F3: α= 0.82	NR
Starr (1971)	College student satisfaction questionnaire CSSQ	USA	College student	NR	70-item 5-dimention	3121	NR	NR	F1: Working condition F2: Compensation F3: Quality of education F4: Social life F5: Recognition	NR	Reliability coefficient=0.94
Betz (1970)	College student satisfaction questionnaire CSSQ	USA	College student	NR	92-item 6-dimension 5-point Likert scale	463	Inter scale correlation	NR	F1: policy and procedure F2: Working condition F3: Compensation F4: Quality of education F5: Social life F6: Recognition	NR	α= 0.85-0.91	NR

NR= Not Reported

Psychometric properties

All studies measured student satisfaction concept. Regard to the study design, one article was cross-cultural evaluation study,[25] and others were studied about psychometric properties.[16,17,20,21,23]

Number of scale items and dimensions of included studies were various. Minimum item number was 22[25] and maximum was 92.[16] Minimum number of dimensions were three in two studies[17,25] and two studies had 11 dimensions.[19,22]

All studies tested for the internal consistency, two for the test–retest reliability,[2,19] two for the criterion validity,[19,23] and ten for the construct validity.

Internal consistency was conducted by calculating Cronbach's alpha in all studies. ICC and split-half Spearman–Brown coefficient were reported for stability in reliability. Criterion-related validity with Health-Related Quality of Life Scale (HRQOL-14) was used as criterion scales for criterion validity. Majority of studies had construct validity by principal components factor or principal axis factor analysis (n = 7), exploratory factor analysis (n = 3), confirmatory factor analysis (CFA) (n = 3), and other methods such as known group, inter-scale correlation, and simple common factor analysis. Scales’ item explained 46.9%[22]–68.54%[27] of the total variance and some studies did not report it. Other psychometric characteristics of included studies are summarized in Table 2.

Quality assessment

The results of COSMIN quality assessment of 13 included articles are given in Table 3. None of these articles had “Excellent” quality in all psychometric properties.

Table 3.

COSMIN quality assessment

First author (year)	COSMIN BOXES
	BOX A Internal consistency	BOX B Reliability	BOX C Measurement error	BOX D Content validity	BOX E Structural validity	BOX F Hypothesis testing	BOX G Cross-cultural validity	BOX H Criterion validity
Liu (2017)	Excellent	Poor	Poor	Excellent	Excellent	Excellent	-	-
Chadha (2017)	Fair	Poor	Poor	Good	Excellent	Good	-	-
Hirsch (2016)	Good	Poor	Poor	Excellent	Excellent	Excellent	Good	-
Torkzadeh (2014)	Excellent	Poor	Poor	Good	Excellent	Good	-	-
Zhai (2012)	Fair	Poor	Poor	Excellent	Excellent	Excellent	-	-
Dennison (2012)	Fair	Excellent	Poor	Excellent	Excellent	Excellent	-	-
Hussain (2012)	Good	Poor	Poor	Excellent	Poor	Poor	-	-
Chen (2012)	Excellent	Poor	Poor	Excellent	Excellent	Excellent	-	-
Zullig (2005)	Good	Poor	Poor	Good	Excellent	Good	-	Fair
Juillerat (1996)	Fair	Excellent	Poor	Excellent	Excellent	Excellent	-	-
Derry (1978)	Excellent	Poor	Poor	Good	Excellent	Excellent	-	-
Starr (1971)	Good	Poor	Poor	Good	Poor	Poor	-	-
Betz (1970)	Good	Poor	Poor	Good	Excellent	Good	-	-

BOX A – Internal consistency

Internal consistency is measured to determine the degree of the interrelatedness among the items on the scale.[30] Quality criteria about internal consistency are adequate sample size (seven per items and >100) AND Cronbach's alpha (s) calculated per dimension AND Cronbach's alpha (s) between 0.70 and 0.95.[29] The COSMIN scores for four studies were “Excellent” and five studies were evaluated as “good” because did not calculate alpha for each dimension/subscale separately[16,18,20,23,25] or did not have adequate sample size.[18] Four studies scored as “fair” for Cronbach's alpha (s) <0.70 or >0.95.[2,19,22,26]

BOX B – Reliability

According to the COSMIN checklist, reliability is the extent to which scores have not changed and are the same for repeated measurement under several conditions, for example, overtime (test–retest), by different persons on the same occasion (inter-rater), or by the same persons (i.e., raters or responders) on different occasions (intra-rater). Quality criteria of reliability are ICC or weighted Kappa ≥0.70.[29] Two studies reported reliability criteria[2,19] and were evaluated as “Excellent,” and other studies assessed as “poor” because did not mention ICC or Kappa value for scales.

BOX C – Measurement error

The systematic and random error of a score that is not attributed to true changes in the construct is considered as measurement error. Measurement errors of all studies were not reported.

BOX D – Content validity

In COSMIN checklist, the content validity is defined as “the degree to which the content of scale is an adequate reflection of the construct to be measured.” Criteria for quality are a clear description of the measurement aim, the target population, the concepts that are being measured, and the item selection AND target population AND investigators or experts were involved in item selection. Six studies that did not mention who involved in item selection and content validity were evaluated as “good,”[16,17,20,21,23,26] and others were “Excellent.”

BOX E – Structural validity

Based on the COSMIN checklist, the structural validity is the degree to which the scores of scale are an adequate reflection of the dimensionality of the construct. Studies that perform exploratory or CFA have quality criteria. In this respect, two articles did not report factor analysis[18,20] and were evaluated as “fair.”

BOX F – Hypothesis testing

According to the COSMIN checklist, hypothesis testing is the same of construct validity. Quality criteria about this aspect are specific hypotheses were formulated, AND at least 75% of the results are in accordance with these hypotheses. Two studies did not report construct validity and were scored as “poor,”[18,20] four did not report enough results and were scored as “good,“[16,21,23,26] and seven studies mentioned construct validity with complete details and were scored as “excellent.”[2,17,19,22,24,25,27]

BOX G – Cross-cultural

According to the COSMIN checklist, cross-cultural is the degree to which the performance of the items on a translated or culturally adapted scale is an adequate reflection of the performance of the items of the original version of the scale. Quality criteria are describing translation process, translating item forward and backward, and independently, adequate sample size, pre-testing the scale, and performing CFA. One study was cross-cultural design[25] and was categorized as “good” because it did not report CFA.

BOX H – Criterion validity

Criterion validity is the degree to which the scores of scale are an adequate reflection of a “gold standard.” Quality criteria for criterion validity are convincing arguments that gold standard is “gold” AND correlation with gold standard is >0.70.[29]

One study[23] performed criterion validity and hypothesized that total score of the Brief Multidimensional Students’ Life Satisfaction Scale (BMSLSS) and HRQOL-14 was negatively correlated. Since HRQOL-14 was not gold standard for student satisfaction, and in correlation between scales (BMSLSS and HRQOL-14) was not >0.70, this study was scored as “fair” in criterion validity.

Categories related to responsiveness were not analyzed, because there were no results related to that.

Discussion

This systematic review identified that the psychometric properties of 13 scales measuring academic student satisfaction. Based on the COSMIN checklist, these scales did not score “Excellent” quality in all psychometric properties. In other words, there is no robust and valid single scale for the measurement of student satisfaction.

In this systematic review, the studies were conducted in different field of study, academic level in different publication time and countries. Although the word nursing as a keyword was not used, three scales were developed for nursing students. This may show the results of the importance of nursing student satisfaction and its impact on the patient care. Some studies were specific for undergraduate students or college students, but others were general. The findings showed that the number of psychometric evaluation publications has significantly increased in the year 2012, while the first published study was in 1970. Regarding the country of publication, the majority of studies were conducted in the USA.

It should be noted that the scale for nursing student satisfaction[2,24,25] had better quality and addressed essential psychometric properties. Four scales were validated for undergraduate students.[2,17,21,25] These scales had good quality, but two of them did not report total variance. In terms of time of publication, newly published articles had more quality scores. This could be followed by the use of journals writing tool guideline and new statistical methods for psychometrics evaluation of scales. Regard to country of the study, it should be noted that first study that was found about student satisfaction scale was conducted in the USA followed by five other studies in the year 1970 until 2012. Although the time of publication had influence on quality, it is not comparable.

In general, dimensions of scales could be categorized into four themes such as curriculum, facilities, campus, and relationship. Dimensions about teaching approach were categorized into the curriculum and were the largest proportion of total explained variance of student satisfaction in some studies.[22,24,25] Campus facilities,[26] resources,[2,26] and administrative and learning facilities[18] were mentioned as facility subscales.[22] Some dimensions of scales were related to campus such as campus climate,[19,22] campus organization,[19] management,[27] university climate,[18] financial and fee/cost,[19,22,26] and other dimensions about environment.[21,23,24,25,27] The relationship between students,[21] admin/staff support,[26] social interaction,[24] and professional social interaction[25] was reported in some scales.

The goal of factor extraction is to maximize explained variance, but since parsimony of scale is important, the aim is to balance two goals using as few factors, as it is adequate in explaining a high proportion of variance.[31] Regardless of the factor extraction method, explained variance in half of the included studies was ≥50%. Maximum total explained variance was 68.54% for Liu et al.[27] study with 58 items and 6 factors. Furthermore, minimum variance explained in Zhai et al.[22] article with 60 items, and 11 factors were 46.9%.

The COSMIN checklist was used in this systematic review, which is the only standard tool for quality assessment of studies on psychometric properties of scales. The overall quality score was not used in quality assessment of scale, because psychometric properties are not equally important.[29] A low-quality assessment of a scale does not imply that it is unsuitable. Some studies did not state enough information in the article clearly, so it is difficult to assess their quality. All studies have reported internal consistency as reliability, but in some studies, there was no information about other essential properties. Most scales had lack of face validity, stability, measurement error, and responsiveness evaluation, and thus future studies must consider these properties when attempting to validate scales.

Although included studies did not discuss measurement error, the highest methodological quality was the “Professional Sport University Student Satisfaction Survey Scale” in Liu et al. study[27] that in four boxes of COSMIN checklist scored as “Excellent,” one box “Good,” and one box “Fair.”

Conclusion

This systematic review provides an overview of 13 scales that measuring student satisfaction in university context. Based on the COSMIN checklist, each study has at least “poor” quality in one box. Results of this study help researchers, managers of educational institutions, and other decision-makers to identify appropriate scales with regard to quality and psychometric properties of them to make accurate assessments of students’ academic satisfaction. All of this would help to identify areas for improvement of academic education and make better decisions for students and other stakeholders. Furthermore, it should be noted that quality appraisal of scales is necessary after developing, and future research should pay equal attention to quality of development and validation.

Financial support and sponsorship

Iran University of Medical Sciences.

Conflicts of interest

There are no conflicts of interest.