Students’ approaches to learning (SALs): Validation and psychometric properties of a tool measurement

Abstract

BACKGROUND:

Deep learning is an important outcome of the higher education and is mostly determined by students’ approaches to learning (SALs). The latest version of the Study Process Questionnaire (SPQ) is one of the most used instruments assessing SALs. Many studies from various contexts have either validated or used this famous tool. But none of them—to the best of our knowledge—stem from the Moroccan tertiary context. The current study fills this gap by first: Getting a local translation of the questionnaire following the standardized methodological process and secondly to update the validity and psychometric properties of the construct.

MATERIALS AND METHODS:

Arabic back translation was performed. Data were collected among tertiary scientific students. Descriptive statistics, Cronbach's coefficient alpha, and confirmatory factor analysis were carried out under SPSS version 22.

RESULTS:

A strong fit of the dichotomic construct (deep and surface) was found, whereas the hierarchical models were disappointing.

CONCLUSIONS:

Following the standards of the psychometrics’ validation, this Arabic version could be used only in first-order factor model to evaluate the deep and surface approach within tertiary education in Moroccan context.

Introduction

As in many countries of the world, the reform of the Moroccan higher education system has placed the learner at the heart of all educational activities and has focused on deep learning. Naturally, these changes in the higher education context and in relationship to the quality of learning and raise two questions: First, what motivates students to learn (motives)? Second, what do they do to accomplish their learning tasks (Strategies)? The motives are defined by the intention or the objectives, which push the students to learn. On the contrary, strategies are the methods used by students to learn a subject. The two questions combined lead to the concept of student approaches to learning (SALs).[1,2] The latter were proved as one of the main factors influencing the quality of learning.[3] Many inventories were created to evaluate SALs, but none, as the best of our knowledge, is known to be developed or validated in the Moroccan context.

SALs measurement and the R-SPQ-2F's relevance

Considerable evidence has been found to support the dichotomy division of SALs in general to surface approach (SA) and deep approach (DA). The latter is thought to lead to greater academic success.[4] Learners who take a DA have an intrinsic motivation and are always looking to integrate the new information with the acquired one to make sense. They seek to satisfy an intrinsic interest in the course and would likely employ large and deep reading. On the other side, students who opt for a surface approach only attempt to reproduce what has been learned and combine passive assimilation of information with the minimum of understanding.[5]

In this investigation, we opt to use one of the most used instruments: the Revised Two-Factor (R-SPQ-2F) developed by Biggs et al.[6] The instrument presents many benefits including the assessment of the student learning being used in its turn as feedback in promoting different teaching environments and increasing learning outcomes.[6]

The questionnaire (R-SPQ-2F) was born and refined after the adaptation of his predecessor the 42 items questionnaire, namely, the SPQ (Study process questionnaire) by Biggs.[7] Since then, the R-SPQ-2F has known great importance and worldwide use[8,9] and in different areas (e.g., medical,[10] health sciences,[11,12] engineering,[5] sciences students)[9,13] and was one of the most cited.[8]

Given the changes in the higher education in the world and in the students’ profiles from that of nearly two decades ago and given the sensitivity to contextual and personnel factors, the need of updating the validation of such instruments is steel dominant. Many instruments have been created to assess SALs. The increased attention to the adaptation of the R-SPQ-2F in cross-cultural environment was motivated by the global need to elevate university students to a deep approach to learning.[14] Few of these works investigated the subject in Arabic-speaking contexts with Arabic translated versions (e.g.,[10,15,16]), but none in the Moroccan setting. This study was therefore conducted for examining psychometric properties of an Arabic version of the R-SPQ-2F within the Moroccan tertiary context. Consequently, we will expect to gain a first local tool to assess the deep/surface learning, which is the aim of a larger project in Moroccan higher education.

Materials and Methods

Study design and setting

The present study as a cross-sectional study was conducted to evaluate the psychometric properties of the translated Arabic version of the R-SPQ-2F[6] among Moroccan scientific students.

Study participants and sampling

The present study is a cross-sectional study conducted among 300 participants (162 are males and 138 are females), all between 18 and 23 years. Inclusion criteria consisted of being all enrolled in a Faculty of Sciences (Fez, Morocco) and equally distributed between the three different study levels (i.e. Year 1, Year 2, and Year 3) of License (Bachelor) of Fundament Studies. They were randomly and voluntary invited to participate in this study. The foreign students were then excluded as they came from a different cultural context.

Data collection tool and technique

The 20-item R-SPQ-2F originally developed by Biggs et al.,[6] is evenly divided into two main dimensions or scales: DA and SA. Each of them contains two subscales motif and strategy with 5 items each, namely:

Deep motif (DM) consisting of items 1, 5, 9, 13, and 17 and deep strategy (DS) with items 2, 6, 10, 14, and 18. DM and DS make up the overall deep approach (DA).

Surface motif (SM) formed with items 3, 7, 11, 15, and 19 and surface strategy (SS) presented with items 4, 8, 12, 16, and 20. Both form the surface approach (SA).

The scoring uses a 5-point Likert scale that ranges from (A) (This item is never or just true for me to (E)) (This element is always true for me). The final score is calculated by adding the different Likert ratings corresponding to items of each scale or subscale. This way, DA score = DM score + DS score, and SA score = SM score + SS.

Regarding the content validity, the recommended and commonly used method of translating such research instruments into cross-cultural studies is the “back translation.”[17] As the first step, the first author with an Arabic-speaking colleague teaching English translated the questionnaire into Arabic.[18] This version was then administered individually to two colleagues who were fluent in Arabic language and had a high level of English, to compare it with the original version. The first Arabic version was synthesized then. This was translated again into English by an Arabic-speaking teacher colleague living in the United States who had no idea of the copies of the instrument in English or the one translated in Arabic (Blind Translation).[17,18] The original, the back-translated, and the Arabic versions were reviewed by a judging committee formed of both translators and the first author to identify gaps in meaning, remove discrepancies, and make adjustments. Thus, we all agree to adopt the functional equivalence meaning[17] commonly used in Moroccan teaching context to translate some words such as “class,” “topic,” “study,” and “a course” according to their context in the questionnaire to give a strong meaning to the translation[19] and, finally, add some modifications.[17] In this perspective, for example, the word “Novel” in the 9^th item was replaced by “my favorite play game” since there is a decreasing interest in reading novels by the arrival of technologies. Finally, the work resulted to an agreement on a preliminary Arabic version of the R-SPQ-2F questionnaire in the Moroccan context just before the pretest. The last was conducted on a sample of 20 students to deal with final translation-related problems.

Ethical consideration

After having obtained an informed consent, the students completed voluntary and anonymously the R-SPQ-2F. Also, the permission from the faculty administration to conduct this study was signed.

Statistical analysis

In his review, Clinton-McHarg et al.[20] considered as main psychometric properties of implementation in the health domain: Face and content validity, the Cronbach alpha of >0.70 and test–retest for internal consistency, exploratory factor analysis (EFA), or confirmatory factor analysis (CFA). Measures did not take all of these and depended on the concept under consideration and the specific context.[9] In our case, to assess good reliability and internal consistency, Cronbach alphas should be at 0.7 and higher as a cutoff point[21] and particularly with dichotomous response scales as confirmed in Clinton-McHarg review.[20] The test–retest stability check in over time would not be suitable for a tool such as R-SPQ-2F, where responses depend on individuals and context.[9] To validate the construct of the translated version, we opted for CFA only. Indeed, Burnett and Dart[22] argue that the CFA fit at the optimum the data to the model tested and it is more preferable than an EFA to assess the validity of the existing instruments. Confirmatory analysis seeks to replicate and extend the findings obtained in the first phase (founders works) with a different sample of participants, raters, or context.[23] In sum, the objective of the CFA was to determine whether the instrument performed at least as reliably for this population of students as it did in previous analyses that form the basis for the justification of its usage in education research.[6] Our aim was directed and focused to confirm an existing construct, if it occurs, not to explore another. This confirmation was the first step before different uses of the instrument in our larger research. As our findings were indices-fit-based, many authors, for example, Kline,[24] recommended that at least the following goodness-fit-indices should be given, namely, the model Chi-square (χ2), RMSEA, the Standardized Root Mean Square Residual (SRMR), and CFI. Moreover, Hooper et al.[25] stated: “Due to the restrictiveness of the model Chi-square, researchers have sought alternative indices to assess model fit.” In practice, Chi-square is influenced by many factors and is not considered a very useful adjustment index by most researchers.[24,26] Earlier, it was indicated that in theory, the χ2 fit index indicates a good model fit if the χ2 is not significant, while if the sample is large, the χ2 test will show that the data are significantly different from those expected on a given theory.[24] Therefore, the estimation of the model fit should be beyond the χ2 and also be indicated by other model fit indices. So, following Kline,[24] suggestions that yet in practice the model fits with the data when the RMSEA at least is ≤ 0.08, the SRMR also does not exceed 0.08 and the CFI at least is greater than 0.90. Recommended values adopted by the original authors were for SRMR ≤ 0.08 and CFI coming close to 0.95. To reach our objective, three complementary models were tested [see Figure 1].

Figure 1.

Simplified representation of the constructs of the models tested. Oval forms represent latent factors. Each factor is represented by an acronym of two letters: Main scales DA = Deep Approach, SA = Surface Approach and subscales, DM = Deep Motive, DS = Deep Strategy, SM = Surface Motive, SS = Surface Strategy. Rectangles represent indicators (items). Each rectangle represents separately and individually each of items belonging to the specific factor in the model

Results

The inclusion criteria to validate the tool started by giving descriptive statistics, the internal consistency (reliability), the CFA analysis, and cross-cultural comparisons of fit indices.

Descriptive statistics

The R-SPQ-2F scores reached an average of 40, 61 for SA and 29, 01 for DA. At the item level, the scores were ranged from 3, 7 (SD = 2, 9) to 4, 1 (SD = 3.50) and from 2.4 (SD = 4.5) to 3, 6 (SD = 4.20) for the two scales DA and SA, respectively. In addition, a significant positive Pearson's correlation was r = -0.45, (P =0.000) shows that both scales are strongly and inversely interrelated.

Reliability

Regarding our chosen cutoff point of 0.70 and above,[20,21] to indicate good reliability, the four subscales presented a marginal interne consistency. Subscale coefficients were ranged from 0.61 for the surface strategy to 0.67 for Motif Strategy. Alpha coefficients for the two main scales, deep approach and surface approach, showed good reliability with values of 0.82 and 0.78, respectively. Table 1 shows these results compared to other studies.

Table 1.

Cronbach’s alpha scores in the current study compared to other scores in cross-cultural contexts

Author(s)	R-SQP-2F Version	Cultural context	Main Scales		Subscales
			DA	SA	DM	DS	SM	SS
Biggs et al.,[6]	English Original	Hong Kong	0.73	0.64	0.62	0.63	0.72	0.57
Lake et Boyd.,[28]	English	Australian	0.86	0.81
Martinelli and Raykov,[27]	Spanish	Malta	0.73	0.75	0.49	0.63	0.56	0.61
Gustin et al.,[29]	French	France	0.70	0.64	0.61	0.47	0.49	0.48
Shaik et al.,[10]	English	kingdom of Saudi Arabia	0.737	0.746
McLaughlin and Durrant,[16]	Arabic	United Arab Emirates	0.79	0.60	0.63	0.66	0.44	0.47
Khine and Afari[15]	Arabic	United Arab Emirates	0.81	0.76
Vergara- Hernandez et al.,[12]	Spanish	Colombia	0.836	0.895
This study	Arabic	Morocco	0.82	0.78	0.65	0.67	0.66	0.61

DA=Deep Approach, SA=Surface Approach, Deep Motive=DM, DS=Deep Strategy, SM=Surface Motive, SS=Surface Strategy

Confirmatory factor analysis CFA

As illustrated in Figure 1, three models were built. The data analysis was fit-indices based on a maximum likelihood estimation. Note that the comparison between Models A, B, and C by means of the χ² differences test at P < 0.05 indicated that Model A fitted the data better than Model B and model C better than B, which do not agree the results below.

Model A

As a first step, we examined the common model which looks at the main two first-order factors (deep and surface) as latent variables. Each of them is measured by its ten corresponding items. To save space, only the factors and items loadings and correlations for the first model were reported here [Figure 2].

Figure 2.

Model A: estimates of the factors’ correlation and the loading items

The loadings were all aligned with the findings supra and were above the standards of 0.30. The CFA fit indices, illustrated in the Table 2 (compared to others for the same model A), results were as follows: CFI = 0.901, SRMR = 0.048, and RMSEA = 0.065. All of the values indicated a good fit with the data sample.

Table 2.

Comparison of the fit indices of the two first-order scales model (Model A) among cross-cultural contexts

Author(s)	R-SPQ-2F version	RMSEA	SRMR	CFI
Kapinga Mutatayi et al.,[3]	Congolese	Not reported	0.047	0.952
Vaughan,[31]	Australian	0.020	0.073	0.989
Vergara- Hernandez et al.[12]	Spanish	0.075	Not reported	0.833
Khine and Afari[15]	Arabic	0.048	0.081	0.91
This study	Arabic	0.065	0.048	0.90

RMSEA=Root Mean Square Error of approximation, SRMR=Standardized root mean square residual, CFI=Comparative Fit Index

Model B

This model was built at the item level to test the existence of the four subscales. The first-order latent factors were the four sub-dimensions correlated between them and measured by the five respect items as indicators [see Figure 1]. The findings indicated that Model B fit values felled short of the conventional values that are as follows: RMSEA value was 0.081 (>0.080), SRMR was 0.078 (close to 0.08), and CFI was of 0.88 (<0.90). The results in Table 3 (with comparison given) proved very marginally the existence of the subscales construct.

Table 3.

Comparison of fit indices of the four first-order subscales model (Model B) among cross-cultural contexts

Author(s)	R-SPQ-2F version	RMSEA	SRMR	CFI
Biggs et al.,[6]	Chinese	Not reported	0.058	0.904
Justicia et al.,[32]	Spanish	0.07	0.12	0.91
Gustin et al.,[29]	French	0,084	Not reported	0,74
Kapinga Mutatayi et al.,[3]	Congolese	Not reported	0.0897	0.639
Khine and Afari,[15]	Arabic	0.058	0.096	0.87
Lopez aguado et Gutierrez-Provecho[30]	Spanish	0.060	0.887	0.861
This study	Arabic	0.081	0.076	0.88

RMSEA=Root Mean Square Error of approximation, SRMR=Standardized root mean square residual, CFI=Comparative Fit Index

Model C

Model C represented the instrument structure originally posited by the founders (Biggs, 1987) but not tested explicitly in the last revision (Biggs et al., 2001). This model was built to see the full adjustment of the dimensionality of the questionnaire. It was the model test of a complex hierarchical second-order construct at the item level. In this model, we tested the combination of second-order scales and first-order subscales as latent variables (2 latent scales) X (4 latent subscales) X (20 observed items). The four first-order factors were correlated to their respects items as indicators (observable variables) [cf. Figure 1]. The output indices showed a CFI of 0.67, an SRMR of 0.102, and an RMSEA of 0.91. All of them did not meet conventional standards.

Discussion

This study aimed to validate the psychometric properties of the translated Arabic version of the R-SPQ-2F in the Moroccan settings. First of all, we have to take in mind the cross-cultural sensitivity of the R-SPQ-2F. The two main factors reliability findings were aligned with those found in previous studies (e.g.,[12,27]) and especially those having used the Arabic version (e.g.,[15]). The Alphas widely supported the dichotomous goodness of the deep and the surface scales. In contrast, at the subscale level there is a great discrepancy among the results. Regarding the three models tested, the first one, i.e., the model A was built to support one of the main concerns of this research, which is the validation of the R-SPQ-2F generalized dichotomous structure: surface and deep approaches. Model A is the most accepted model and proved by many authors (e.g.,[10,12,14,30]) as such as with two learning approaches the DA and the SA with ten items each as originally posited. This model was validated by Biggs et al.,[6] by taking the sum of the four subscales items as indicators instead of individual items. Some Arabic contexts[15] supported those findings but others did not (e.g.,[16]). Other Western findings confirm the fit model but with removing items as showed in the Australian osteopathy population.[31] The results found in the second model test (B) matched the Justicia et al.,[32] previous findings. Other studies in different contexts found the model fitted in good to a quite reasonable way with the data, namely, in Arabic,[15] and Australian,[6] contexts, respectively, but not in Congolese,[3] or French[29] (based on RMSEA and CFI only) settings.

Finally, the model C has not been satisfactory at all and by far was rejected. The explanation given may be related to the complexity of the model. The fewer parameter estimated the more fitted model is[26] (e.g., Model A VS Model C). In the same way, Biggs et al.[6] argued the marginally results-of-fit indices (SRMR and CFI), and yet just for a less hierarchical model (model B), were due to the complexity of the model. The model was by far rejected in Norwegian[14] and Spanish[30] studies, even more the latter used nine indices in CFA among absolute, comparative, and parsimonious ones.

The results indicated, as a whole, that the latent construct at the two-factor level with two approaches (deep approach and surface approach) was better represented than the other structures.

Moreover, we should mention that the indices adopted to estimate the goodness-of-fit, either by their number and/or by their cutoff points are surrounded by considerable controversy[33]. The case can affect the finding interpretations among studies. To assess good reliability and internal consistency, we followed Cronbach alphas at 0.7 and higher as a cutoff point,[21] since many studies tented to a lower values (<0.7). With regard to the CFA, we had chosen the minimum required conventional standards according to Kline,[24]: two absolute fit indices: RMSEA and SRMR (at acceptable value ≤0.08) and one comparative index fit: the CFI (<0.90). Vergara-Hernández et al.,[12] used, for the same model A, three adjustment criteria with higher cut-off points: a RMSEA ≤0.06, a CFI, and a TLI (Tucker–Lewis index) both above 0.95 referring to a very good fit. But only with regard to this RMSEA and the CFI (0.075 and 0.083, respectively), we can see the non-fit model without resorting to the last, the TLI. Leung et al.[34] did not report the RMSEA (cutoff point taken was ≤0,05) for fear of having it high with a small sample. Some researches (e.g.,[3]) studied models fit only under CFA and based only on two indexes (SRMR and CFI). Others like Justicia et al.[32] suggested that using other indices can improve the models fit compared to others. Moreover, Newson[26] reported that by opting for certain indices such as parsimonious fit indices, the fit measures are to penalize less parsimonious models. He added that the adoption of a well fitted and simple alternative model (such as model A) is favored instead of seeking the fit of other models via other indices (such as Model C).

Second, there is a great discrepancy between researchers on the cross-cultural validation and adaptation of the R-SPQ-2F. On the one hand, some were limited to validate the entire original structure as defined by the developers or to a lesser extent the validation of the instrument at major scales only: the deep and surface approaches.[3,9,12,30,31] On the other hand, other researches went to review modified versions of the R-SPQ-2F giving rise to a multitude deriving structures in terms of scales or in terms of number of items.[16,31,35] The cultural context, the validation methodology, i.e., translation mode (if used), statistics (CFA or EFA or both) and parameters (choice and numbers), and the sample size are some of the factors that have affected this discrepancy.

Limitation and recommendation

The strength of this research is that it is an original work in Moroccan context and was conducted with a sample size (N = 300) surpassing the minimum required (N = 200) to conduct CFA and with the application of powerful statistical techniques. The only limitation was that participants were from the same university sampling. As a learning tool, we recommend, for the generalizability, that the study should be repeated over other tertiary contexts.

Conclusion

As final conclusion, following the standards of the psychometric properties’ validation applied to the same tool in different fields (medical, dental, nursing, sciences, engineering etc.), Also, our investigation took into strong consideration the experience and validation methods of the R-SPQ-2F developers resulted in comparable results at least for similar models (i.e., Model A). The confirmatory factor analysis was conducted and the measure of internal reliability of the R-SPQ-2F structure supported by far only the two main scales’ structure (surface approach and deep approach) as shown in Model A. In practice, the quality of learning is assessed by the way the students approach globally their learning tasks: in-depth or on the surface. In this perspective, we retain the assertion of the designers themselves, which consider a preferable use of the questionnaire within the scope of only two main approaches: surface approach and deep approach as they are most relevant to its routine use by teachers. Our evidence is added to an accumulation of others evidences that suggests that a two-factor model (deep and surface approaches) is sufficient to describe the underlying constructs of the R-SPQ-2 F. At this level, our goal is achieved, and we can conclude that this first Arabic version of the R-SPQ-2F can be used with confidence to evaluate the students’ learning approaches in Moroccan scientific tertiary environments. Note that a subsequent work was taken in quantitative and qualitative ways (not reported here), explored the R-SPQ-2F in greater detail among this population, addressed the limitations in the present study letting preliminary evidence of it use in other settings.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.