Perception of Learning Environment among Undergraduate Medical Students in Two Different Medical Schools through DREEM and JHLES Questionnaire

Abstract

Introduction

Assessment of learning environment is essential to assess the acceptability of the curriculum among students. Several tools are available to assess undergraduate medical students’ perception of learning environment. Dundee Ready Education Environment Measure (DREEM) questionnaire is the most commonly used tool. Here, we have used both the widely used DREEM questionnaire and a relatively new questionnaire Johns Hopkins Learning Environment Scale (JHLES).

Aim

Assessment of students’ perception of learning environment of two eastern Indian medical schools using DREEM and JHLES questionnaire.

Materials and Methods

In this cross-sectional questionnaire based study, 200 students from Nil Ratan Sircar Medical College (NRSMC) and 78 students from College of Medicine and Sagore Dutta Hospital (CMSDH) of 5^th semester batch duly completed the two questionnaires, DREEM scale and JHLES tool. The DREEM questionnaire has 50 questions arranged in five domains. The JHLES questionnaire has 28 questions arranged in seven domains. Comparison of scores between the two colleges was done by unpaired t-test.

Results

There were altogether 100 female and 178 male participants with mean age of 20.46±0.67.

There were no significant difference between the overall DREEM score (p=0.81) and the JHLES scores (p=0.10) obtained from NRSMC and that obtained from CMSDH. Analysis of individual domain scores on DREEM scale revealed that there were no significant differences in domain scores for the two medical schools except for Students’ Perception of Atmosphere (SPA) score (p=0.0086). JHLES revealed significant differences in terms of engagement, inclusion and safety, and physical space (p<0.001). The DREEM and JHLES results revealed positive correlation (r=0.59).

Conclusion

Both DREEM and JHLES scores revealed comparable results from two schools with positive correlation between DREEM and JHLES tools, however some areas with low scores require modification especially the domain assessing Students’ Academic Self-Perception (SASP) and Students’ Social Self Perceptions (SSSP).

Introduction

Learning Environment (LE) or Educational Environment (EE) is one of the major factors that play a major role in shaping the future of the undergraduate medical students. Experience in the medical school can influence a student’s knowledge, attitude, and practicing pattern in later years of life [1].

Learning or educational environment is not only limited to the environment of the classroom, wards, library or the medical school as a whole, but also includes the teacher-student relationship, co-operation among classmates, attitude of senior students [1,2]. Even, curriculum is also an essential part of the learning or educational environment. Thus, evaluation of learning environment is very much relevant for the medical educators whose ultimate goal is to improve students’ performance within the college and after passing the course as doctors. Now, it is quite challenging to make the learning environment student friendly but keeping the quality and standard of education satisfactory [1–3]. Another challenge for medical educator is appropriate assessment of existing learning environment. There are several tools available to assess students’ perception of the learning environment [4].

The most widely used tool to assess students’ perception of learning environment is the DREEM questionnaire [1,2,5–7]. There are several other tools like Accreditation Council for Graduate Medical Education (ACGME), Learning Environment Assessment (LEA), Medical School Environment Questionnaire (MSEQ), Course Valuing Inventory (CVI), etc., (almost 15) described in various publications.

However, none of them have strong evidence supporting their validity [8]. JHLES, developed and validated by the faculty members of the John Hopkins University School of Medicine, is a newer LE tool with promising evidences to support its validity [9–11].

In this study, we used the already widely used DREEM tool and the newer JHLES to assess students’ perception of learning environment simultaneously in two Government medical colleges enrolled in the same university, West Bengal University of Health Sciences (WBUHS) in West Bengal. We used both DREEM and JHLES to increase the validity of assessment of students’ perception.

Materials and Methods

It was a cross-sectional and questionnaire based study and the students participated in this study anonymously. Permission from the institutional ethics committee was obtained before conducting the study.

All the undergraduate medical students of the 5^th semester batch (academic year 2015-16) of the two medical colleges (NRSMC and CMSDH) were informed about the study and informed consents were obtained from individual willing students (all of them were adults). The study was conducted simultaneously in the two medical colleges in the month of August, 2016 and the students completed the two questionnaires at the same time in a lecture class. Altogether 223 students from NRSMC (out of total 255 students) and 86 students from CMSDH (out of total 100 students) participated in the study.

DREEM tool: There are altogether 50 items in the DREEM questionnaire assessing five domains on Students’ Perceptions of Learning (SPL), Students’ Perceptions of Teachers (SPT), Students’ Academic Self-Perception (SASP), Students’ Perception of Atmosphere (SPA), and Students’ Social Self-Perceptions (SSSP) [1–7].

All the 50 statements were given to each of the students and they were asked to rate each of the statements based on a 5-point based Likert scale (scoring was done as follows, ‘0’=strongly disagree, ‘1’=disagree, ‘2’=unsure, ‘3’=agree, and ‘4’=strongly agree).

The scores for each of the domain was as follows, for SPL maximum score was 48 (12 items), for SPT maximum score was 44 (11 items), SASP maximum score was 32 (8 items), SPA maximum score was 48 (12 items), and SSSP maximum score was 28 (7 items) [1–7].

Reverse scoring was done for the nine negative items (item numbers 4, 8, 9, 17, 25, 35, 39, 48 and 50) in the DREEM questionnaire as for them a score of ‘4’ means strongly disagree and a score of ‘0’ means strongly agree. The maximum score that could be obtained was 200 with scores lying between 0-50 rated as “very poor”, 51-100 rated as “many problems”, 101-150 rated as “more positive than negative”, and 151-200 rated as “excellent”.

During final analysis, mean score for any item > 3.5 indicated true positive points whereas mean score for any item < 2 indicated problem areas and mean scores lying between 2 and 3.5 indicate those areas of the environment that can be improved.

JHLES: The JHLES has overall 28 items under 7 (seven) domains namely: community of peers, faculty relationships, academic climate, meaningful engagement, mentorship, acceptance and safety, and physical space. The students were asked to rate each of the item from 1 to 5 (higher scores indicated better perception of LE). The maximum score that can be obtained is 140 while minimum possible score was 28 [9–11].

The students were asked to assess their perception of LE on two forms (DREEM tool and JHLES) at the same time during a routine lecture class. Two medical teachers from the two medical schools supervised the whole procedure. Students filled in the forms at their respective medical schools. A total time of one hour was allotted and the collected forms were assessed by a third medical teacher not attached to the two medical schools. Incomplete forms were rejected; forms with responses to all the 50 items in DREEM questionnaire and all the 28 items in the JHLES questionnaire were considered.

Statistical Analysis

After collection, the data were compiled on MS Office Excel word spreadsheet (Microsoft Excel 2010 version). Mean and standard deviation for each of the domains were calculated (5 domains in DREEM and 7 domains in JHLES) along with total scores for both DREEM and JHLES.

Domain scores were compared across individual two medical schools using one-way Analysis of Variance (ANOVA0 and for comparison between two schools unpaired t-tests were used.

Results

Demographic profile of the participants: After obtaining the forms (DREEM and JHLES) from all the individuals, 200 forms out of 223 forms from NRSMC and 78 forms out of 86 forms from CMSDH were considered (remaining forms were discarded because of incomplete response, torn forms, ineligible writing, and wrong scoring) for final analysis.

The mean age (in years) of all the participants was 20.46 (SD 0.67); from NRSMC, the mean age was 20.4 (SD 0.85) and that for the participants from CMSDH was 20.64 (SD 0.79) [Table/Fig-1]. Altogether, there were 100 female participants, 35.71% (75 from NRSMC and 25 from CMSDH) and 178 male participants, 64.28% (125 from NRSMC and 53 from CMSDH). Among the participants 170 students were staying at hostel (61.15%) whereas the remaining (108) students stayed at home (38.84%) [Table/Fig-1].

[Table/Fig-1]:

Demographic details of the participating students.

Variables	NRSMC (n=223)	CMSDH (n=86)	Total (n=309)
Number of completed forms	200	78	278
Age in years (mean±SD)	20.4±0.85	20.64±0.79	20.46±0.67
Gender (Female/Male)	75/125	25/53
Accommodation Hostel	120	50
Accommodation: Home	80	28

Analysis of DREEM Results

There were no significant difference (p=0.81) between the overall DREEM score obtained from NRSMC (119.64±13.97) and that obtained from CMSDH (119.11±21.19) [Table/Fig-2].

[Table/Fig-2]:

Analysis of DREEM score. The p<0.05 marked with*

Total Score (Mean±SD)	Domain Score	NRSMC (Mean±SD) n=200	CMSDH (Mean±SD) n=78	p-Value
NRSMC 119.64±13.97 CMSDH 119.11±21.19 p=0.8065	SPL (48)	28.0995±6.11	29.375±7.2624	0.1366
	SPT (44)	30.41±5.99	29.56±4.69	0.2565
	SASP (32)	19.711±4.400	20.248±3.54	0.3351
	SPA (48)	27.45±5.088	25.65±5.27	0.0086 *
	SSSP (28)	14.47±3.69	14.675±3.6932	0.6749

Comparison of domain scores between two schools were done by unpaired t test, *-significant

Analysis of individual domain scores revealed that except for the SPA score (27.45±5.088 for NRSMC students and 25.65±5.27 for CMSDH students, p=0.0086), there were no significant differences among other domain scores as perceived by the students of both NRSMC and CMSDH (p-values were 0.1366, 0.2565, 0.3351, and 0.6749, for SPL, SPT, SASP, and SSSP, respectively) [Table/Fig-2].

Students from both the colleges gave the highest ratings to SPT (students’ perception of teachers) domain, scores being 30.41 (SD 5.99) and 29.56 (SD 4.69) from NRSMC and CMSDH, respectively [Table/Fig-2]. Again, students of NRSMC and CMSDH assigned the lowest scores to the SSSP domain (students’ perception to social self-perception) 14.47 (SD 3.69) and 14.67 (SD 3.69), respectively [Table/Fig-2]. The second lowest scores were given by both the students of NRSMC and CMSDH to the domain SASP (students’ perception to academic self perception), 19.711±4.400 and 20.248±3.54, respectively [Table/Fig-2].

Analysis of JHLES Results

The overall JHLES results revealed no significant difference (p=0.10) among scores obtained from NRSMC and CMSDH students, the scores being 86.2±14.94 and 82.86±16.77, respectively [Table/Fig-3].

[Table/Fig-3]:

Analysis of JHLES questionnaire. p<0.05 marked with*

Total Score (Mean±SD)	Domains (28-140)	NRSMC Mean (SD)	CMSDH Mean (SD)	p-value
NRSMC: 86.21±14.94 CMSDH: 82.86±16.77 p=0.1041	Peers (6-30)	16.621(3.59)	17.34(5.41)	0.1955
	Faculty (6-30)	23.819 (3.41)	23.07(2.4)	0.0739
	Academy (5-25)	22.001 (3.33)	22.066 (1.7829)	0.8689
	Engagement (4-20)	10.24(1.79)	9.39 (1.68)	0.0003*
	Mentorship (2-10)	5.72(1.58)	5.59 (1.52)	0.5301
	Inclusion and safety (3-15)	8.79(2.07)	5.58 (1.81)	0.0001*
	Physical space (2-10)	6.1(1.33)	4.49(1.32)	0.0001*

Comparison of domain scores between two schools were done by unpaired t-test; *-significant

Analysis of individual domain scores revealed significant difference between only three domains “engagement” (p=0.0003), “inclusion and safety” (p=0.0001), and “physical space” (p=0.0001) [Table/Fig-3].

Analysis of other domain scores revealed no significant difference between scores given by the students of NRSMC and CMSDH. The scoring pattern were similar by students of both the colleges, as the same three domains (“peer”, “faculty”, and “academy”) received the maximum scores by the students of both the colleges The scores are as follows 16.621±3.59 and 17.34±5.41 for peer domain, 23.819±3.41 and 23.07±2.4 for faculty domain and 22.001±3.33 and 22.066±1.7829, by the students of NRSMC and CMSDH, respectively for academy domain [Table/Fig-3].

Similarly, minimum scorings were received by the domains “mentorship”, “inclusion and safety”, and “physical space” from students of both NRSMC and CMSDH, although interestingly, there were significant differences (p=0.0001, in the “inclusion and safety” and “physical space” domain scores) among the scorings from these two domains between the two colleges [Table/Fig-3].

Maximum scorings by both the NRSMC and CMSDH students were in “faculty” domain (23.819±3.41, and 23.07±2.4, respectively). However, minimum scorings by NRSMC and CMSDH students were different; it was “mentorship” domain (5.72±1.58) for NRSMC and “physical space” domain for CMSDH (4.49±1.32) [Table/Fig-3].

The total scores of DREEM and JHLES are correlated with one another, although correlation between scores were higher among students of NRSMC (r=0.59) in comparison to those among CMSDH students (r=0.59).

Discussion

Assessment of students’ perception of educational environment is very important for curriculum development and also to evaluate the acceptability of the existing one among the students.

DREEM Scale is a widely used scale used for this purpose but JHLES is a comparatively newer assessing tool [1–7]. Until now, only two studies have used JHLES to assess the students’ perception of LE. The first study conducted by Robert B Sochet and his colleagues was the pilot study conducted with the aim of validating this new measuring tool JHLES [9]. The study was conducted among the students from different years at the John Hopkins University School of Medicine. This study successfully established the validity of JHLES as a measuring tool for assessment of students’ perception of LE.

The second study conducted by Sean Tackett and his colleagues using the JHLES measuring tool and the widely used DREEM scale for assessment and comparison of the students’ perception LE among three medical schools following the same curriculum [10]. The researchers concluded that similar type of positive LE can be established among different schools separated by great distance (located in different countries) through comparing the results using JHLES and DREEM scale data. The results revealed strong correlation between DREEM scale and JHLES scores.

In our country, several studies have been conducted to assess the undergraduate medical students’ perception of LE using the widely used DREEM scale; however, none of the above mentioned studies used the newer JHLES tool. To the best of our knowledge, this is the first study conducted in India to assess and compare the undergraduate medical students’ perception of LE using both the DREEM scale and the JHLES tool in two different medical schools following the same curriculum (enrolled under the same university).

Analysing the results we have found that, there were no significant difference in terms of total DREEM score (p=0.81) and JHLES (p=0.10) between the two medical colleges, NRSMC and CMSDH. Again, analysis of individual domain scores for DREEM scale revealed that there were no significant difference among different domain scores for the two medical colleges except for SPA scores (p=0.0086). One of the promising findings from the analysis of DREEM score is that students of both colleges have found both the learning (SPL) and teachers (SPT) of positive quality (“more positive perception” for SPL and “moving towards the right direction”, SPT). Like the other studies conducted in India, our students also showed positive attitude except for SSSP which indicated requirement of improvement of social self perception which is similar to other studies conducted in India [2,5]. The global score (“more positive than negative) obtained from analysing the DREEM scores was similar to those obtained in other studies [2,5].

The same results are reflected in the results of JHLES tool analysis. It is found that both the domain scores for “faculty” and “academy” have highest scores signifying students’ positive perception particularly towards these two components of the LE.

Analysis of SPA and SASP domain scores of the DREEM questionnaire from both the colleges indicated positive outcome. However, the domain score SSSP for both the colleges were more on the negative side (interpreted as “not a nice place”). The possible reasons for low scoring in this domain might be the stress associated with MBBS course. Another possible reason is with time the students become more accustomed with the study pattern, examination pattern, and thus, can handle the stress better, as the students assessed in this study are from 2^nd year they are relatively new to the course and to the stress associated with it.

This pattern of students’ perception is also reflected in JHLES scores as both the “inclusion and safety” and “physical space” domain scores are particularly low in the both the colleges. Low score in the “inclusion and safety” domain might be because of the recent surge in the unpleasant incidents of harassment of junior doctors. In comparison to other studies, which were conducted in the Western countries [9] or in a medical school following the curriculum of a western university [10,11], the total score using the JHLES tool was somewhat lower. Most probably the difference is due to the different study settings. As it is obvious that, there would be some differences between a developing country (like ours) and a developed country in terms of curriculum, study setting and infrastructure of the institutions, however in terms of faculty and peer the difference is not so prominent [9,10].

Limitation

The most important limitation of our study was that it is a cross-sectional study, analysis has been done only once. It would be better to expose the same batch of students to DREEM scale and JHLES questionnaires at every year as in this way we will get a more real picture regarding the students’ perception to LE of a particular medical college and also to a particular curriculum.

Conclusion

From our study, we can conclude that the undergraduate medical students’ perception to LE in two different medical schools following the same curriculum is comparable and some general measures like involvement of students’ during curriculum development, inclusion of stress relieving measures in the curriculum, encouragement of extra curricular activities, provision of adequate security measures, etc., might be helpful to improve the students’ perception of the LE.

Financial or Other Competing Interests

None.