Validating the Medical Students' Stressor Questionnaire (MSSQ) from a Sri Lankan medical faculty

Abstract

Objectives

The Medical Students' Stressor Questionnaire (MSSQ) is a specific tool to assess the degree of stress in medical students. However, this tool has not been validated in Sri Lanka. Therefore, this study contextually adapted the MSSQ and investigated its validity in the local context.

Methods

A total of 603 medical students, in various phases of their undergraduate training, from the Faculty of Medicine at the University of Colombo, participated in the study. Students who have completed their studies in the faculty for at least six months were included. The self-administered questionnaire was contextually adapted to the local setting and was approved by subject and language experts. Responses were analysed for construct validity (including exploratory factor analysis to estimate factor structure of the scale), sample adequacy, and internal consistency. Data were analysed using the SPSS statistical package.

Results

The mean age of the study cohort was 23.3 ± SD 2.0 years, while 258 (42.8%) were males. The MSSQ scale significantly correlated with the General Health Questionnaire (p<0.001), indicating a strong concurrent validity. The exploratory factor analysis showed that items were loaded appropriately on five new factors, and the Kaiser-Meyer-Olkin measure (0.954) and Bartlett's test of sphericity (p<0.001) showed excellent sample adequacy. The internal consistency of the MSSQ overall (α = 0.95), and each of the new factors: (α > 0.82 for the first four factors and α = 0.54 for the fifth) were satisfactory. The test-retest reliability was high (Pearson's r = 0.918, p<0.001).

Conclusions

The contextually adapted MSSQ is a valid and reliable instrument that can be used in the assessment of stress among medical students.

Introduction

Stress levels are known to be high among medical students during their training¹ and has a negative impact on the academic performance and wellbeing of medical students.² Students perceive that their psychological well-being is affected by the workload in medical school.³ Furthermore, stress has been associated with negative effects such as poor relationships with colleagues, poor decision making, sleep disturbances, substance abuse, and suicidal ideations.^4,5

Previous studies in Sri Lanka have used non-specific tools such as the 12-item General Health Questionnaire (GHQ-12) and Perceived Stress scale to assess stress among medical students. Different scales measuring psychiatric morbidity have been validated to the Sri Lankan population previously.^6,7 The Depression, Anxiety and Stress Scale has been translated into Sinhala and validated among University students.⁸ GHQ-12 has been validated in Sri Lanka as a screening tool to detect minor and non-psychotic psychiatric conditions.⁹

In previous studies, various tools have been used to assess stress levels among medical students. The Perceived Stress Scale-14 has been used in previous studies^10,11 and has been shown to have acceptable composite reliability (greater than 0.80). Furthermore, all standardised factor loadings were statistically significant, ranging from .514 to .806.¹² The Copenhagen Burnout Inventory is another tool which has been used previously and showed good test–retest reliability (>90%) and internal consistency (Cronbach's alpha: 0.92) among medical students.^13,14

The Medical Student Stressor Questionnaire (MSSQ) was first developed in Malaysia, and includes 40 aspects which are specific to medical students. It has been validated in countries such as Malaysia,¹⁵ the Netherlands,¹⁶ Romania,¹⁷ Nepal¹⁸ and India.^19,20 This questionnaire has also been translated to the Romanian language,¹⁷ and it was stated in the systematic review performed by Yusoff et al²¹ that the instrument has probably been translated into other languages as well. However, the relevant data is inaccessible to be referenced as the investigators have failed to disseminate the findings.

The reliability analysis of the MSSQ by Yusoff et al¹¹ showed a total Cronbach's alpha value of 0.95 and all 40 items had an item correlation value of more than 0.3, hence all the preliminary 40 items were included in the MSSQ. The items were loaded appropriately into the six pre-determined hypothetical groups as their factor loading values were more than 0.3. The MSSQ identifies sources of stress specific to medical students such as academic stressors, interpersonal stressors, teaching and learning stressors, social stressors, drive/desire stressors, and group activity stressors.¹⁹ The MSSQ has been used to study and quantify stress among medical students in many countries and a recent systematic review confirmed the overall validity of the instrument across multiple settings. Furthermore, it has been used to validate other similar questionnaires on stress among medical students.²³ Although the MSSQ's validity has been assessed in regional countries such as India, it has not been previously validated in Sri Lanka.²⁰

The MSSQ was chosen as an important tool to be validated in the local setting because it specifically addresses the environmental and psychometric factors among medial students in the context of undergraduate medical training. Moreover, a valid tool is important for future studies among medical students to identify the level of stress and for future changes in the medical curriculum. The objective of this study was to adapt and assess the validity and reliability of the MSSQ using a group of medical students from the Faculty of Medicine, University of Colombo.

Materials and Methods

This study was conducted at the Faculty of Medicine, University of Colombo (Sri Lanka) from February to April 2020. A cross-sectional study design was adapted, similar to the previous validation study on the MSSQ²⁴(11). Students from the first to the final year were invited to participate in the study. Students who have been part of the undergraduate curriculum for at least six months from preclinical and clinical years were included as others were new to the medical faculty. Students who had just completed the final year exam were excluded as they have completed the curriculum, and this may be a confounding factor. Finally, a total of 603 Sri Lankan medical students who consented to participate were included in the study. The questionnaire was administered to medical students representing all five years in the Faculty of Medicine, University of Colombo. The questionnaire was self-administered in English, which is the language of the original version. Students were also administered the General health Questionnaire (GHQ-12)²⁵ at the same time. For the assessment of test-retest reliability, the MSSQ was administered to a group of 52 students twice, two weeks apart.

The MSSQ is available free for non-commercial research. It is a self-reported, self-scoring instrument with 40 items. The students rate the intensity of stress caused by each item on a scale of 0–4 (causing no stress to causing extreme stress). The items are grouped into six categories; academic related stressors (ARS), intrapersonal and interpersonal related stressors (IRS), teaching and learning-related stressors (TLRS), social related stressors (SRS), drive and desire related stressors (DRS), and group activities related stressors (GARS). The original MSSQ is scored by obtaining the mean values for each of the six categories and the scores are rated as mild (0.00–1.00), moderate (1.01–2.00), high (2.01–3.00) and severe (3.01–4.00).

The MSSQ was contextually adapted by two independent investigators and the following modifications were made. In the original questionnaire, ‘quota system in examinations’ was changed to ‘ranking system based on merit’ and ‘conflict with personnel(s)’ to ‘conflict with faculty staff(s)’ to suit the local setting. Face validity of the modified questionnaire was assessed by a panel of two subject experts and two language experts. Concurrent validity was assessed using the GHQ. The GHQ is a screening instrument used in primary care, general medical practice, and community surveys to detect minor psychiatric conditions and stress and has been validated for Sri Lanka.^26,27

Statistical analysis was done using SPSS version 20.0 and SPSS-AMOS programs.

Confirmatory factor analysis (CFA) was performed to assess the validity of the factor structure based on the original construct of the instrument. Goodness of fit of CFA was measured by the following indices: Root Mean Square Error of Approximation (RMSEA) smaller than 0.05, Comparative Fit Index (CFI) greater than 0.90, Tucker–Lewis coefficient (TLI) greater than 0.90, and ratio of the Chi-square value to its degrees of freedom (Chi²/df) of less than 2.5. An exploratory factor analysis was planned in case of poor fit.

Exploratory factor analysis was conducted using the principal component analysis with Varimax rotation. The Kaiser-Meyer-Olkin (KMO) test and Bartlett's test of sphericity were used to determine the sampling adequacy. The sample was considered adequate when the KMO value was more than 0.5 and the Bartlett's test was significant. Internal consistency of the modified questionnaire was measured using Cronbach's alpha. The Pearson's coefficient was used to assess the correlation between the MSSQ and GHQ. A p-value of less than 0.05 was considered statistically significant. The Pearson's coefficient was used to assess the test-retest reliability using a small group of students who were administered the questionnaire two weeks apart.

Results

The mean age of the sample was 23.3$\pm SD2.0$ years and 258 (42.8%) were males. The majority were Sinhalese (n = 509, 84.4%), there were 53 Tamils (8.8%), 26 Muslims (4.3%), and others (2.5%, n = 15).

Background and demographic details of participants are summarised in Table 1.

Table 1.

Participants' background information.

Variables	Frequency	Percentage
Sex
Female	345	57.2%
Male	258	42.8%
Ethnicity
Sinhalese	509	84.4%
Tamil	53	8.8%
Muslim	26	4.3%
Bhutanese	11	1.8
Burger	4	0.7%
Religion
Buddhism	486	80.6%
Hinduism	48	8.0%
Christianity	40	6.6%
Islam	23	3.8%
Other	6	1.0%
Year of Medical school
1st year	95	15.8%
2nd year	95	15.8%
3rd year	200	33.2%
4th year	103	17.1%
Final/5th year	110	18.2%

Construct validity

Confirmatory factor analysis was conducted on the total sample of 603. The CFA's goodness of fit was evaluated by the following indices, as described in the methodology: Root Mean Square Error of Approximation (RMSEA) was 0.078, Comparative Fit Index (CFI) was 0.78, Tucker–Lewis coefficient (TLI) was 0.764 and ratio of the Chi-square value to its degrees of freedom (chi2/df) was 4.630. Therefore, this CFA showed poor fit indicating that the original construct of a six-factor scale of the MSSQ did not fit with the study population.

Therefore, exploratory factor analysis was performed to determine the construct validity of the MSSQ. The Kaiser-Meyer-Olkin test (KMO), which is a measure of sampling adequacy, was 0.954 which indicated an adequate sample size, and Bartlett's test of sphericity was highly significant (p<0.001). The total number of components were extracted using Principal component analysis and Varimax rotation. The items were loaded into five new factors based on the scree plot (Figure 1) that explained a cumulative variance of 53.8%. The loading of each item to the new factors was based on the factor loadings and the theoretical construct. The construct was similar to the Italian MSSQ validation study²⁸ and therefore, a similar factor construct was adapted. Factor structure and internal consistency of the MSSQ is given in Table 2.

Figure 1.

Scree plot.

Table 2.

Factor structure and internal consistency of the MSSQ.

	Factor loadings	Cronbach's Alpha if Item Deleted
Academic Related Stressor (ARS)
1. Tests/Examinations	.486	.919
4. Merit ranking system in finals	.467	.918
7. Need to do well (self-expectation)	.480	.918
10. Heavy workload	.626	.918
12. Falling behind in reading schedule	.551	.918
17. Not enough medical skill practice	.481	.918
18. Lack of time for family and friends	.444	.918
19. Highly competitive learning context	.621	.918
23. Having difficulty understanding the context	.491	.918
25. Getting poor marks	.618	.918
26. Poor motivation to learn	.526	.918
27. Lack of time to review what has been learnt	.624	.918
30. Inability to answer the question from the teachers	.520	.918
33. Large amount of content to be learnt	.657	.918
36. Unjustified grading/ranking process	.589	.917
Staff and Students Related Stressors (SSRS)
3. Conflicts with other students	.588	.919
5. Verbal or physical abuse by other student(s)	.654	.918
8. Not enough study material	.329	.918
9. Conflict with faculty staff	.609	.918
14. Lack of guidance from teacher(s)	.523	.918
28. Verbal or physical abuse by teacher(s)	.667	.918
29. Frequent interruption of my work by others	.486	.918
31. Conflict with teacher(s)	.698	.918
35. Not enough feedback from teacher(s)	.583	.918
38. Working with computers	.350	.919
Intrapersonal and Expectations Related Stressors (IERS)
6. Parental wish for you to study medicine	.342	.919
15. Feeling of incompetence	.593	.917
16. Uncertainty of what is expected of me	.570	.918
32. Unwillingness to study medicine	.426	.918
34. Need to do well (imposed by others)	.528	.918
40. Family responsibility	.326	.918
Teaching and Learning Related Stressors (TLRS)
11. Participation in class discussion	.630	.918
13. Participation in class presentation	.575	.919
20. Teacher- lack of teaching skills	.548	.918
22. Inappropriate assignments	.486	.918
37. Lack of recognition for work done	.566	.917
38. Working with computers	.350	.919
Patients Related Stressors (PRS)
2. Talking to patients about personal problems	.415	.919
21. Unable to answer questions from patients	.494	.918
24. Facing illness or death of the patients	.458	.919

The MSSQ's total scale correlated highly with the GHQ (r = 0.632, p<0.001) as well as the 5 individual subscales (ARS: r = 0.628, p<0.001; SSRS: r = 0.481, p<0.001; IERS: r = 0.583, p<0.001; TLRS: r = 0.580, p<0.001; PRS: r = 0.254 p<0.001) indicating good concurrent validity.

Reliability

The MSSQ showed high internal consistency with a Cronbach's alpha of 0.95. Cronbach's alpha values of the subscales ranged from 0.54 to 0.90 showing acceptable internal consistency. When individual items were removed, the Cronbach's alpha ranged from 0.917 to 0.919 showing that all items contributed to the scale adequately.

Table 3 shows the inter-factor correlation which indicates the discriminant ability of an item. Inter-factor correlations showed that Academic Related Stressors- Factor I (ARS), Staff and Student Related Stressors-Factor II (TLRS), Intrapersonal and Expectation Related Stressors- Factor III- (IRS) and Teaching and Leaning Related Stressors- Factor VI (TLRS) had a higher discriminant ability. Patient Related Stressors- Factor V (PRS) had lower correlation with other items indicating lesser discriminant ability.

Table 3.

Factor Correlation Matrix.

	ARS	SSRS	IERS	TLRS	PRS
ARS	1.000	.708	.791	.735	.461
SSRS	.708	1.000	.656	.719	.476
IERS	.791	.656	1.000	.688	.361
TLRS	.735	.719	.688	1.000	.468
PRS	.416	.476	.361	.468	1.000

Theoretical constructs; ARS = Academic Related Stressor- Factor I, SSRS = Staff and student Related Stressor- Factor II, IERS = Intrapersonal and Expectations Related Stressors- Factor III, TLRS = Teaching and Learning Related Stressors - Factor IV, PRS = Patient Related Stressors- Factor V. All correlations were statistically significant (p<0.001).

The corrected-item total correlation and the inter item correlation value was more than 0.3 for 39 items. Item 2, ‘Talking to patients about personal problems’ had a corrected-item total correlation of 0.265. However, there was no considerable change in Cronbach's alpha with the deletion of any item, indicating acceptable reliability. Therefore, all 40 items were included in the questionnaire. Cronbach's alpha values of each stressor group are shown in Table 4.

Table 4.

Internal Consistency of new subscales of the MSSQ.

Stressor Groups	Number of Items	Cronbach's alpha
Academic Related Stressors (ARS)	15	.900
Staff and Student Related Stressors (SSRS)	10	.834
Intrapersonal and Expectation Related Stressors (IERS)	6	.823
Teaching and Learning Related Stressors (SRS)	6	.832
Patient Related Stressors (PRS)	3	.540

Test-retest reliability

The MSSQ was administered to a group of 52 students twice, two weeks apart. Pearson's correlation coefficient for the overall score was 0.918 (p<0.001), showing a strong test-retest reliability. Similarly, the correlation coefficient for new individual factors were also satisfactory (ARS: r = 0.891, p<0.001; SSRS: r = 0.908, p<0.001; IERS: r = 0.865, p<0.001; TLRS: r = 0.788, p<0.001; PRS: r = 0.862 p<0.001).

Discussion

The contextually adapted version of the MSSQ with a revised factor construct had good internal consistency with an overall Cronbach's alpha of 0.95 and values ranging from 0.54 to 0.90 for the subscales. Significant correlation with the GHQ suggests good concurrent validity. Factor analysis showed that the items loaded onto five new factors which explained a cumulative variance of 53.8%. The high test-retest reliability suggests that the errors in measurement are less likely to be influenced by changes in the individuals' responses over time (r = 0.918, p<0.001).

A study done in Kolkata, India in 2013 assessed the validity of the MSSQ among 81 medical students.²⁰ In that study, the Cronbach's alpha coefficient was greater than 0.8 for the overall score and the academic-related stressor domain. Other domains such as intrapersonal and interpersonal-related stressors, teaching and learning-related stressors and group activities-related stressors showed a Cronbach's alpha value ranging from 0.5–0.8. However, the social-related stressor and drive and desire-related stressors were less than 0.5.²⁰

In the study done by Dagani et al²⁸ in Italy, the CFA performed showed poor fit with the original construct of the MSSQ scale, similar to our study (RMSEA = 0.079 (90% CI: 0.020, 0.102); CFI = 0.757 TLI = 0.724; Chi²/df = 7.064; p < 0.001). This variation may be explained by the differences in cultural background and medical curriculum (methods of teaching and evaluation) in different countries.

Studies have shown a relatively high prevalence of stress in medical students, ranging from 30% to 50%.¹⁹ Stress is the body's nonspecific response or reaction to demands made on it, or to disturbing events in the environment.¹⁹ A critical issue regarding stress among medical students is its effects on learning. Stress can be either facilitating towards learning, which is called ‘favourable stress’, or it can be deleterious towards learning which is called ‘unfavourable stress’.²⁹ The perceived level of stress may vary among different students depending on their personality, previous experience, coping strategies, and cultural background.³⁰ This questionnaire has used a model known as the person-environment model which is useful in understanding stress among students.³¹ According to the model, stress in medical students is caused by multifactorial reasons which are interrelated. This questionnaire will be helpful to identify the stressors in separate domains which is useful in determining the source of stressors among medical students. It is valuable not only to researchers, but also to medical students who can self-evaluate their level of stress and the stressors thereby aiding early intervention.

The contextually adapted MSSQ with new factor construct is a reliable instrument that can be used in the local setting to study stressors among medical students.

Limitations of the study

The study was conducted among the students of a single medical faculty. More accurate results could be obtained via multicentre studies involving other medical faculties to obtain more accurate results in the Sri Lankan context. Further, the study was based on a single measurement. A series of measurements during the undergraduate training period would provide better insight on the validity of the scoring system. Finally, this study was conducted based on the English version of the questionnaire. Future studies may focus on translations to the local languages to obtain a more culturally reliable tool.

Conclusion

This study showed that the contextually adapted MSSQ with new factor construct had acceptable psychometric properties. It is a valid and reliable instrument that can be used in the local setting for assessment of stress among medical students.

Recommendation

The contextually adapted MSSQ has good validity and reliability in terms of internal consistency and test-retest reliability. It will be useful to assess stress among medical students in the Sri Lankan setting.

Source of funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interest

The authors have no conflict of interest to declare.

Ethical approval

Ethical clearance was obtained from the Ethics Review Committee of the Faculty of Medicine, University of Colombo. All participants gave informed written consent before participating in this study.

Consent

All participants gave informed consent to the work. The authors declare that the paper has not been previously published or under review.

Authors' contribution

UJ, KL, AA, AJ and VD conceived and designed the study. UJ, KL, AA and AJ conducted research, provided research materials, and collected and organised data. UJ, KL, AA, AJ and VD analysed and interpreted the data. UJ, KL, AA and AJ wrote the initial and final drafts of the article and provided logistic support. VD performed the critical revisions in the manuscript. All authors have critically reviewed and approved the final draft and are responsible for the content and similarity index of the manuscript.