The framework of Systematic Assessment for Resilience (SAR): development and validation

Abstract

Background

Burnout and depression among health professions education (HPE) students continue to rise, leading to unwanted effects that ultimately jeopardise optimal medical care and patient health. Promoting the resilience of medical students is one solution to this issue. Several interventions have been implemented to foster resilience, but they focus on aspects other than the primary cause: the assessment system. The purpose of this study is to develop a framework to promote resilience in assessment planning and practice.

Methods

We followed the guidelines suggested by Whetten for constructing a theoretical model for framework development. There were four phases in the model development. In the first phase, different literature review methods were used, and additional students’ perspectives were collected through focus group discussions. Then, using the data, we constructed the theoretical model in the second phase. In the third phase, we validated the newly developed model and its related guidelines. Finally, we performed response process validation of the model with a group of medical teachers.

Results

The developed systematic assessment resilience framework (SAR) promotes four constructs: self-control, management, engagement, and growth, through five phases of assessment: assessment experience, assessment direction, assessment preparation, examiner focus, and student reflection. Each phase contains a number of practical guidelines to promote resilience. We rigorously triangulated each approach with its theoretical foundations and evaluated it on the basis of its content and process. The model showed high levels of content and face validity.

Conclusions

The SAR model offers a novel guideline for fostering resilience through assessment planning and practice. It includes a number of attainable and practical guidelines for enhancing resilience. In addition, it opens a new horizon for HPE students’ future use of this framework in the new normal condition (post COVID 19).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12909-023-04177-5.

Background

The study of medicine is demanding and puts a significant strain on the mental and physical health of medical students, who perceive medical education as an anxiety-inducing and stressful field of study [1]. This perception has been mirrored by the high level of stress in medical students. Several systematic reviews and metanalyses [2–4], as well as local [5, 6] and multicenter studies [7, 8] have found a significant prevalence of stress among medical students ranging from 21 to 56%. Consequently, burnout and depression among medical students have increased [4]. Other negative effects of stress on medical students have also been reported, including poor clinical performance, poor decision-making, poor peer interaction, interpersonal conflict, academic dishonesty, and sleep problems [4, 9]. Stress has also been associated with suicide, alcoholism, and drug abuse [10–13]. These negative effects eventually jeopardise optimal medical care and impact patient health negatively [14, 15]. While medical students have identified a number of stressors, research indicates that examinations are the most frequently reported sources of stress [14, 16–21]. As a result, there is a growing body of research on how to improve the mental health of medical students and promote what is currently known as resilience.

Resilience is a psychological construct that refers to the characteristics needed to adapt to the dynamic changes of life and maintain mental well-being [22]. The topic of resilience is of interest in many disciplines, including developmental psychology, sociology, education and, in particular, health professions education (HPE) [23–27]. In psychology, resilience generally refers to the status of an individual who is adapting to significant adversity while maintaining good mental and physical well-being [28]. Alva [29] defined academically resilient students as those ‘who sustain high levels of achievement motivation and performance despite the presence of stressful events and conditions that place them at risk of doing poorly in school and ultimately dropping out of school’ [29]. Further, academically resilient students are able to maintain mental agility and continue growing and developing despite academic and life setbacks [30–33]. In a recent meta-analysis of resilience constructs across 21 resilience scales, Wadi et al. [34] identified four primary resilience characteristics: 1) control: maintaining composure and control in the face of stressful adversity; 2) involvement: being committed to overcoming adversity; 3) resourcefulness: using available resources for appropriate solutions to overcome adversity; and 4) growth: continuing to grow and rebounding stronger from adversity. These four constructs provide a solid foundation for the implementation of interventions fostering resilience.

Studies on HPE have shown that resilience is positively correlated with compassion, satisfaction, and patient care and negatively associated with burnout, secondary stress, anxiety, intolerance to ambiguity, and poor communication [35]. Numerous health institutions have implemented interventions based on these studies [15, 36, 37]. Common intervention guidelines include training workshops focussed on psychosocial skills, such as mindfulness, Stress Management and Resilience Training (SMART), and narrative and simulation training [38–41]. Although these interventions have been shown to have some positive effects [42], they lack a solid theoretical foundation and do not focus on assessment, the primary source of student stress. Resilience theory must be integrated with the assessment context to provide a solid basis to guide intervention strategies and maintain the quality of assessment. Interestingly, a recent paper explored the intersection between resilience and curricula. The authors presented a principle-based approach to curriculum design to foster resilience as an integral part of the curriculum in higher education [43]. Although this approach sheds light on the philosophical approach to building a curriculum to create resilient graduates, it does not address the exact link between assessment and students’ resilience. To the best of our knowledge, no study has incorporated resilience into the assessment process. Accordingly, the purpose of this study was to develop a framework to promote resilience in assessment planning and practice. In this way, assessment would serve as a source of resilience and promote the development of resilience-improving characteristics among students.

Methods

The researchers followed the guidelines for developing a theoretical model proposed by Whetten [44]. These guidelines include four essential questions for model development: (1) What are the constructs/factors that should be considered to explain the model? (2) How are these constructs/factors related to each other? (3) Why is the proposed relationship represented by this portray? (4) What are the implications of this model for research and practice? Each question refers to a developmental phase of the theoretical model. Accordingly, the researchers developed the model in these four phases. This involved establishing the foundation (literature review and focus group discussion) and triangulating the findings [45] through a content validation and response process. Figure 1 presents a flow chart summarising the study phases, questions, research areas, methods, and outcomes of each phase.

Fig. 1.

Flow chart summarizing study phases, questions, research areas, methods, and outcomes of each phase

Phase I

Identifying resilience constructs

In this phase, we aimed to answer the following question: ‘What are the constructs/factors that should be considered in developing the model?’ Accordingly, we first identified the research areas related to the model development: academic resilience, assessment, and test anxiety. Then, we examined narrative and scoping reviews and focus group discussions to find evidence in these areas.

Narrative reviews

We conducted three concurrent narrative reviews to collect sufficient scientific research for the model synthesis [46]. The first review identified the theoretical foundations and factors influencing academic resilience. The second review delineated the theoretical foundations of assessment in HPE. The third review identified the theories behind test anxiety. The articles included in this review were compiled into a table of evidence synthesis [47] (Appendix I) to extract key information regarding the theoretical foundations of academic resilience, test anxiety, and assessment systems and match them with the four resilience constructs [34].

Scoping review

Four of the authors (MW, MSBY, AFA, and NZ) conducted a scoping review to identify factors affecting test anxiety [48] following Arksey and O'Malley [49] stages of scoping reviews. Six electronic databases were used: PubMed, CINAHL, PsychINFO, ERIC (through EBSCOHST), SCOPUS, and ProQuest. The Preferred Reporting of Items for Systematic Reviews and Meta- Analyses (PRISMA) Statement [50] was followed to report the scoping review steps. Based on the factors identified in each study, codes from all studies were compiled to generate subthemes and overarching themes [48]. Appendix II contains a detailed description of the scoping review.

Focus group discussion (FGD)

Four of the authors (MW, MSBY, AFA, and NZ) conducted an FGD to elicit students’ knowledge, perspectives, and attitudes regarding test anxiety (TA) and their coping guidelines [51]. Appendix III contains a detailed description of the FGD steps and procedures.

Phase II

Relating the identified constructs and their factors with each other

Phase 2 answered Whetten’s second question: ‘How are these constructs/factors related to each other?, and based on the notion of triangulation [45], the authors performed three subsequent steps:

• I. Based on the output of the scoping review and FGD, the factors decreasing TA were qualitatively analysed to generate guidelines for decreasing TA. Initially, similar factors were grouped together. Then, a suitable guideline was constructed capturing these factors. The initial guideline statements were revised to remove redundant statements and merge similar guidelines into a single statement. This iterative process was done until consensus was reached among the authors.

• II. Guided by the assessment cycle [52] and the sociotechnical model of assessment [53], the guidelines were thematically categorised into five phases of assessment. Each phase was named and defined.

• III. To evaluate the content validity of these guidelines, the authors utilised a structured tool called the Content Validity Index (CVI) [54, 55], which measures the proportional agreement when two or more expert panels independently evaluate the relevance of a model’s contents to the domain of interest. Ten experts, who were medical education and student assessment specialists, were invited to join the panels [54]. A four-point Likert scale was created in a Google form and sent to the experts via email. They were asked to evaluate the relevance of each guideline to its corresponding category (phase of assessment) [56]. An answer of 1 indicated the guideline was irrelevant, whereas an answer of 4 indicated that the item was extremely relevant.

• IV. Three indices were calculated: item/guideline-level CVIs (I-CVIs), scale-level CVIs (S-CVI) over the universal agreement method (S-CVI/UA), and the average calculation method (S-CVI/Ave) [54, 55, 57]. In I-CVIs, the relevance of each guideline to its related phase of assessment was evaluated by experts. Using a dichotomous rating of relevance, experts’ ratings of 1 or 2, indicating non-relevance, were counted as 0, while ratings of 3 and 4, indicating relevancy, were counted as 1 [57]. In S-CVI/UA, the proportion of guidelines receiving a rating of 3 or 4 (relevant) from all expert panels was calculated. In S-CVI/Ave, the mean I-CVI score for all guidelines was summed [57]. Appendix V contains the full content validity protocol.

• V. Based on these indices and the experts’ recommendations, five guidelines were removed, so the final set included 19 guidelines.

Phase III

Sorting and ranking guidelines by experts

This phase answered Whetten’s third question: ‘Why is the proposed relationship represented by this portray?’ The same experts who were invited for content validation were asked to sort and rank each guideline into the appropriate four resilience constructs [58]. Using the checkboxes grid on the Google form, all of the guidelines were listed in one column, and the resilience constructs were placed at the top of the four adjacent columns. A column headed ‘not applicable’ was added for any guidelines that were irrelevant to the resilience constructs.

The responses were analysed using Microsoft Excel’s sorting and ranking functions. For each resilience construct, an Excel-based graphical representation was created based on the consensus of 50% of the experts if they categorised this particular guideline under a specific resilience construct.

Finally, using the Draw.io website, a conceptual map was created to link the guidelines-based assessment phases and resilience constructs to the theoretical foundation of phase I.

Phase IV

Implication for practice

In phase IV, the authors aimed to answer Whetten’s final question: ‘What are the implications of this model for research and practice?’ We evaluated the guidelines from the users’ perspective based on the response process method [59]. Twenty [20] participants were invited via email [60]. Apart from the invitation, the email contained a description of the research objectives and the validation process. A link to a video describing the application of SAR in assessment practice was also included. A response process form was attached to the email. The medical teachers were asked to review all guidelines and rate each of them based on its clarity and comprehensibility using a four-point scale (1 = not clear and comprehensible, 2 = somewhat clear and comprehensible, 3 = clear and comprehensible, 4 = very clear and comprehensible). Additionally, the participants were asked to provide written feedback on open-ended questions about the feasibility and applicability of the guidelines. Appendix VI contains the full content validity protocol.

We calculated three FVI indices: item/guideline FVIs (I-FVIs), scale/model FVIs using the universal agreement method (S-FVI/UA), and scale/model FVIs using the average method (S-FVI/UA) (Ave). First, all ratings were converted to a dichotomous scale: not clear (ratings of 1 and 2) and clear (ratings of 3 and 4). Then, we calculated the percentage of medical teachers who gave each guideline a ‘clear’ rating (I-FVIs). The proportion of guidelines receiving a rating of 3 or 4 (clear) from all medical teachers was then determined in S-FVI/UA. The average score of all I-FVIs for all guidelines was calculated in S-FVI/Ave. Based on these indices and the recommendations of the medical educators, two guidelines were eliminated, resulting in a final set of 17 guidelines.

The final configuration of the framework

After all these phases, Microsoft Visio® was used to reshape the framework for the systematic assessment of resilience to make it more comprehensible and straightforward to implement. The four resilience constructs were placed in relation to the assessment phases, and the final list of phase-related guidelines was also placed.

Results

Phase I

The narrative reviews yielded relevant theoretical foundations in three areas: academic resilience, assessment, and test anxiety. These foundations were tabulated, and each relevant implication(s), through which resilience could be promoted, was identified (Table 1). In the first three columns, research areas, theoretical foundations/frameworks, and the implications of each study were presented, respectively. Guided by the integrated resilience model [34], every implication was matched to its suitable four resilience constructs: control, involvement, resourcefulness, and growth [34]. The matching was based on the agreement of the authors.

Table 1.

Matching the implication of the identified theoretical frameworks with resilience constructs

			Resilience constructs [34]
			Control	Involvement	Resourceful	Growth
Areas of narrative review	Theoretical frameworks	Implication(s) which can used to promote resilience
Academic resilience	Motivation boosters and guzzlers model [33]	• Self-belief	✓			✓
		• Value of schooling			✓
		• Learning persistence	✓			✓
		• Planning and monitoring	✓	✓	✓	✓
		• Study management
		• Low anxiety	✓
	The 5-C (confidence, coordination, control, composure and commitment) model of academic resilience. [61]	• Confidence, is synonymous with self-efficacy	✓
		• Coordination, which means the ability of planning study tasks		✓		✓
		• Control, which refers to the ability to exert control over perceived uncertainty.	✓
		• Composure, which indicates a low level of anxiety	✓			✓
		• Commitment, which entails keeping persistence.		✓
	Coping reservoir tank [62]	• Psychosocial support			✓
		• Social/healthy activities			✓
		• Mentorship			✓
		• Intellectual Stimulation	✓			✓
	Academic buoyancy process model [63]	• Increase self-efficacy	✓
		• More engagement in the academic context		✓
		• Low anxiety environment	✓			✓
		• More ability to control surroundings				✓
		• Good teacher-students relationship			✓
	Higher-order model of resilience [64]	• Purpose in life	✓			✓
		• Self-esteem	✓			✓
		• Life satisfaction	✓
		• Cognitive flexibility	✓			✓
		• Proactive coping		✓		✓
		• Social support			✓
Assessment system	Utility of assessment [65]	• The educational effect should be considered as a means of improving students' learning and mental well-being.		✓		✓
	The supporting framework for student learning under assessment [66]	• Assessment expectations should be communicated clearly with students	✓	✓
		• Sufficient feedback should be provided		✓		✓
		• Feedback should be focused on learning rather than on marks or students themselves		✓		✓
		• Feedback should be linked to the purpose of the task and to its criteria		✓		✓
		• Feedback should be understandable to students		✓		✓
	The wheel of competency assessment program [67]	• Assessment should have educational consequences (e.g. feedback)		✓		✓
		• Assessment should be meaningfulness (clear communication of what is expected and what done)	✓	✓		✓
		• A transparent communication of the assessment should be made.		✓		✓
	Good feedback practice model [68]	• The assessment process should assist in clarifying what constitutes good performance (goals, criteria, expected standards).		✓
		• The assessment process should aid in the development of self-assessment (reflection) skills in the learning environment.		✓		✓
		• The assessment process should provide students with high-quality information about their learning progress.		✓		✓
		• The assessment process should promote dialogue between teachers and students about what they are learning.		✓	✓	✓
		• A positive assessment should promote positive motivational beliefs and self-esteem in the participants.	✓			✓
		• When a performance gap exists between current and desired performance, the assessment should identify ways to close it.		✓		✓
		• Teachers should be provided with information from the assessment that they can use to help shape their instruction.
	A new model for designing programs of assessment [69]	• Assessment drives learning		✓		✓
	Criteria of good assessment [70]	• The educational effect should be considered as a means of improving students' learning and mental well-being.	✓	✓		✓
	A model of the pre-assessment learning effects on student assessment. [71]	• Improving learning outcomes through assessment design that takes into account impact appraisal, or the student's appraisal of the assessment	✓	✓		✓
		• Designing assessments to promote learning while taking response appraisal into consideration		✓		✓
		• Assessment should be designed to promote learning and take into account perceived agency.		✓	✓
		• Consideration of interpersonal factors in the design of assessments to promote learning		✓		✓
	Aspire criteria for assessment [72]	• Learning opportunities should be supported, enhanced, and created through the assessment programme.		✓		✓
Test anxiety	The cognitive attentional model [73, 74]	• Students’ cognitive reactions can be better controlled by improving their own perception, which will reduce test anxiety.	✓			✓
		• Students' emotional reactions can be controlled by managing their interactions on exam day.	✓			✓
	The learning deficit model [75]	• Assistance in improving preparation will help students to feel less anxious about tests.		✓	✓
	The dual deficit model [76]	• All the previous implications are applied here as this theory was built on them.	✓	✓	✓	✓
	Self-regulation model [77]	• Help student to set goals through good feedback and continuous monitoring	✓			✓
	Self-Worth Model [78]	• Improving the positive self-image of students by dealing with them in a temperate manner in all situations, particularly during examinations.		✓	✓
	Transactional Process Model [79]	• Interventions to improve the impact of each of the following will reduce test anxiety: Students psychological effects	✓			✓
		• Interventions to improve the impact of each of the following will reduce test anxiety: Environmental situations		✓	✓

The scoping review revealed that factors related to test anxiety can be categorised into two groups: those that increase TA and those that decrease TA (Appendix II). Likewise, the thematic analysis of the medical student focus group discussion yielded three major themes: students, academic resources, and examiners. Each theme was subdivided into subthemes that corresponded to an increase or decrease in TA [51] (Appendix III).

Phase II

Compiling and categorizing the generated guidelines

Based on the scoping review and FGD findings, the authors compiled a list of test anxiety-reducing guidelines (Appendix IV). The list was evaluated, and guidelines that were duplicated were eliminated. Guided by the assessment cycle [52] and the model of ‘assessment as a sociotechnical system’ [53], the authors categorised the generated guidelines into groups and subsequently named and defined each group. The following five phases of evaluation were identified:

• 1. Assessment direction, which focuses on improving the candidate’s understanding of the assessment’s scope and procedure.

• 2. Assessment preparation, which emphasises enhancing the candidate’s cognitive, mental, and psychomotor readiness to optimise assessment performance.

• 3. Assessment experience, which enhances the formative assessment component.

• 4. Examiner focus, which relates to improving examiner behaviour to improve the candidate’s performance and decrease the candidate’s anxiety.

• 5. Student reflection, which encourages reflection by students.

Table 2 shows each phase and its related guidelines.

Table 2.

Categorization of SAR guidelines based on stages and phases of assessment

Stage of assessment	Classified guidelines to enhance resilience
Pre-Assessment “Anticipation and Preparation:	Assessment direction: Improving candidate's knowledge of the assessment scope and process 1. Share of assessment mapping whenever applicable 2. Sharing of assessment rubric in modalities whenever applicable 3. Briefing on the overall assessment coverage 4. Establish a briefing session before exam 5. Familiarize students with assessment methods Assessment preparation: Improving candidates cognitive, mental, and psychomotor preparedness to maximize assessment performance 6. Advice students about study skills 7. Advice students about time management [80–83] 8. Direct students for good material for revision 9. Advice students on exam skills 10. Brief students about the grading system 11. Provide guidelines for students to reduce test anxiety 12. Provide self-care guidelines to students before assessment Increase assessment experience: Enriching the formative component of the assessment 13. Increase frequency of formative assessment 14. Increase continuous assessment 15. Encourage to have targeted mock exams 16. Promote/encourage collaborative assessment 17. Promote open book exam 18. Use peer assessment 19. Introduce progress testing
Intra-assessment	Examiner focus: Improving examiner behavior to maximize candidate performance reduce candidate anxiety 20. Establish non-threatening environment during exam: smiling face, welcoming, professional behavior, rapport, sense of humor [84–88]
Post-Assessment Reflection and feedback	Student reflection: promote student’s reflection 21. Add free space in the written assessment to get students feedback 22. Add free space in the written assessment to write self-reflection 23. Increase feedback by examiner 24. Sharing the key answer if applicable

Content validation of the guidelines

Six of the 10 expert panels responded to the invitation to participate in the content validation process. The expert panels had extensive experience in medical education, student assessment, and psychological well-being (Appendix V). Following the principles for calculating CVIs [55, 57], all items (guidelines) with an I-CVI of 0.83 or higher were deemed relevant and included in the response process study based on the CVI. A global acceptance level of 0.80 or higher for S-CVI/UA and S-CVI(Ave) indicates that all components are relevant to the framework. Before the subsequent step, modifications were made based on feedback, which resulted in the elimination of items scoring less than 0.83. Table 3 shows the list of guidelines based on the content validation.

Table 3.

The CVIs of the guidelines after removing four items

SAR guidelines (n = 19)	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Experts in agreement		I-CVI	UA
1. Sharing of assessment mapping whenever applicable	1	1	1	1	1	1	6		1.0	1
2. Sharing of assessment rubric in modalities whenever applicable	1	1	1	1	1	1	6		1.0	1
3. Briefing on the overall assessment coverage	1	1	1	0	1	1	5		0.8	1
4. Establishing a briefing session before exam	1	1	1	1	0	1	5		0.8	1
5. Familiarizing students with assessment methods	1	1	1	1	1	1	6		1.0	1
6. Increasing frequency of formative assessment	1	1	1	1	1	1	6		1.0	1
7. Encourage to have targeted mock exams	1	1	1	1	1	0	5		0.8	1
8. Promoting/encouraging collaborative assessment	1	1	1	1	1	1	6		1.0	1
9. Promoting open book exam	1	1	1	1	1	1	6		1.0	1
10. Using peer assessment	1	1	1	1	0	1	5		0.8	1
11. Introducing progress testing	1	1	1	1	0	1	5		0.8	1
12. Advising students about time management	1	1	1	1	0	1	5		0.8	1
13. Directing students for good material for revision	1	1	1	1	1	1	6		1.0	1
14. Advising students on exam skills	1	1	1	1	0	1	5		0.8	1
15. Providing guidelines for students to reduce test anxiety	1	1	1	1	1	1	6		1.0	1
16. Establishing non-threatening environment during exam: smiling face, welcoming, professional behavior, rapport, sense of humor	1	1	1	0	1	1	5		0.8	1
17. Adding free space in the written assessment to write self-reflection	1	1	1	1	0	1	5		0.8	1
18. Increasing feedback by examiner	1	1	1	1	0	1	5		0.8	1
19. Sharing the key answer if applicable	1	1	1	1	1	1	6		1.0	1
Total of items	19	19	19	17	12	18		S-CVI/Ave	0.91
Proportional relevance	1	1	1	0.89	0.63	0.95		S-CVI/UA		1.0
S-CVI/Ave	0.91

Figure 2 illustrates the correlation and configuration of the relationship between the findings of phase I of the study. The right side of Fig. 2 presents the recommended guidelines for fostering resilience through five phases of assessment. They are connected to the four resilience constructs with arrows that point in both directions to illustrate the reciprocal relationship between guidelines and resilience constructs. The four resilience constructs were linked to the theoretical foundations of academic resilience and the assessment system, and they were shown to have an inverse relationship with test anxiety.

Fig. 2.

Mapping phase I output with the SAR framework and its guidelines

Phase III

Figures 3, 4, 5, and 6 display the results of the experts’ ranking and sorting. Each figure represents a resilience construct and its corresponding guidelines, on which 50% of the experts agreed.

Fig. 3.

Sorting and ranking of guidelines relating “self-control” construct

Fig. 4.

Sorting and ranking of guidelines relating “management of resources” construct

Fig. 5.

Sorting and ranking of guidelines relating “engagement” construct

Fig. 6.

Sorting and ranking of guidelines relating “growth” construct

Phase IV

The response process aimed to evaluate the use of SAR and its guidelines among medical teachers. Twelve [12] of the 20 invited panels responded to the invitation. The panels consisted of medical teachers from various disciplines and universities (Appendix VI). On the basis of the FVI [60], 17 items (guidelines) with an I-FVI of 0.83 or higher were retained, indicating their relevance, while two guidelines with scores below the threshold were eliminated (two guidelines). Table 4 presents the final list of guidelines. As a result, the overall framework S-FVI/UA and S-FVI (Ave) improved from 0.92 to 0.94, indicating the clarity and understandability of all framework components (Table 4).

Table 4.

The FVI indices of SAR after removing some guidelines

SAR guidelines (n = 17)	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Expert 7	Expert 8	Expert 9	Expert 10	Expert 11	Expert 12	Experts in agreement		I-CVI	UA
1. Sharing of assessment mapping/blueprinting whenever applicable	1	1	1	1	1	1	1	1	1	1	1	1	12		1.00	1
2. Sharing of assessment rubric in modalities whenever applicable	1	1	1	1	1	0	1	1	1	1	1	1	11		0.92	1
3. Briefing on the overall assessment coverage	1	1	1	1	1	1	1	1	1	1	1	1	12		1.00	1
4. Establishing a briefing session before exam	1	1	1	1	1	1	1	1	1	1	1	1	12		1.00	1
5. Familiarizing students with assessment methods	1	1	1	1	1	1	1	1	1	1	1	1	12		1.00	1
6. Advising students about time management and study skills	1	1	1	1	1	1	1	1	1	1	1	1	12		1.00	1
7. Directing students for good material for revision	1	1	1	1	1	0	1	1	1	1	1	1	11		0.92	1
8. Advising students on exam skills	1	1	1	1	1	1	1	1	1	1	1	1	12		1.00	1
9. Providing guidelines for students to reduce test anxiety	0	1	1	1	0	1	1	1	1	1	1	1	10		0.83	1
10. Increasing frequency of formative assessment	1	1	1	1	1	1	1	1	1	1	1	1	12		1.00	1
11. Encourage to have targeted mock exams	1	1	1	1	1	1	1	1	1	1	1	1	12		1.00	1
12. Promoting/encouraging collaborative assessment	1	1	1	1	0	1	0	1	1	1	1	1	10		0.83	1
13. Promoting open book exam	1	1	1	1	0	1	0	1	1	1	1	1	10		0.83	1
14. Using peer assessment	1	1	1	1	0	0	1	1	1	1	1	1	10		0.83	1
15. Establishing non-threatening environment during exam	1	1	1	1	1	1	1	1	1	1	1	1	12		1.00	1
16. Increasing feedback to examinee	1	1	1	1	1	1	1	1	1	1	1	1	12		1.00	1
17. Adding free space/window for self-reflection	1	1	1	1	0	1	0	1	1	1	1	1	10		0.83	1
Total of items	16	17	17	17	12	14	14	17	17	17	17	17		S-CVI/Ave	0.94
Proportional relevance	0.94	1.00	1.00	1.00	0.71	0.82	0.82	1.00	1.00	1.00	1.00	1.00		S-CVI/UA		1.0
S-CVI/Ave	0.94

The medical teachers also gave encouraging written insights about the guidelines and their clarity (Appendix VII), which are summarised as a word cloud in Fig. 7.

Fig. 7.

Word cloud of the most common comments made by medical educators

The SAR framework

The SAR framework (Fig. 8) incorporates the overarching relationship between the four resilience constructs [34], the five phases of the assessment process, and their relevant strategies for promoting resilience. The four constructs of resilience include 1) self-control, meaning that students should be able to govern themselves and face adversity, 2) management, which describes students’ ability to use available resources effectively to overcome obstacles, 3) engagement, which refers to students’ ability to be involved and committed to pursuing the challenge, and 4) growth, which reflects students’ ongoing development to face future challenges. The four constructs are part of an ongoing continuous cycle.

Fig. 8.

The Systematic Assessment for Resilience (SAR) framework

The ‘assessment experience’ phase is represented as the nucleus of the framework, as it is the core of the assessment process. In this phase, students’ resilience may be promoted through various strategies, such as increasing the frequency of formative assessment, encouraging targeted mock exams  [87–91], promoting collaborative assessment and open book exams, peer assessment, and introducing progress testing. In the ‘assessment direction’ phase, students’ knowledge of the assessment scope and process may be improved through various strategies, including the sharing of assessment mapping and the assessment rubric, briefing on the overall assessment coverage, establishing a briefing session before the exam, and familiarising students with the assessment methods. Such strategies will foster resilience, especially the self-control and growth constructs. In the ‘assessment preparation’ phase, students’ cognitive, mental, and psychomotor preparedness are improved to maximise their assessment performance. Various strategies may be used, such as advising students on study skills, time management, and exam skills, directing students to good materials for revision, and providing strategies for students to reduce test anxiety. These strategies will promote resilience, especially the self-control and management constructs. The ‘examiner focus’ assessment phase deals with examiners’ behaviour to maximise students’ performance and reduce their anxiety. This phase can be improved by establishing a non-threatening environment during examinations, for example, by smiling, engaging in welcoming and professional behaviour, building rapport, and showing a sense of humour. All of these will foster resilience, especially the engagement and management constructs. The ‘student reflection’ assessment phase promotes students’ reflection by providing space in the written assessment for students to offer feedback on the examiner. Such strategies will improve students’ resilience, particularly in the engagement and growth constructs.

Discussion

As the prevalence of pathological stress among medical and HPE students continues to rise [3–8], a number of interventional programmes have been designed to improve their mental health [15]. The primary criticism of these programmes is their emphasis on causes other than the primary cause, which is the assessment. This study provides a methodical approach for promoting resilience while practicing the assessment. A variety of strategies promote resilience through the five phases of assessment. Resilience is comprised of four constructs, which are shown in Fig. 8. In the following sections, we will discuss each assessment phase and how it can promote resilience.

Assessment experience

The assessment experience phase is purposively located at the centre of this framework, as it promotes the four resilience constructs. The assessment experience emphasises the frequency of formative assessment or any other assessment experience (e.g. mock exam), offering students an opportunity to engage in a simulated challenge similar to the real assessment. Such an experience creates a space for self-regulation and thus enhances self-efficacy and learning growth [92]. There is a growing body of research highlighting the crucial role of formative assessment in maximising learning [93, 94]. Formative assessment serves as a tool for practicing assessment, reflecting on performance, and identifying weak points and opportunities to improve actual future performance [68]. In this study, the strategies within the assessment experience were designed to maximise self-regulatory learning and evaluative judgment [92, 93]. In a nutshell, the more exposure to the assessment experience, the greater the options for strengthening evaluative judgment and hence self-regulation, which is expressed as ‘self-control’ in our framework. Furthermore, increasing ‘assessment experience’ will lead students to the focus on the prudent use of available resources, ‘management’, vividly experiencing the actual assessment, ‘engagement’, and self-esteem—in other words, on ‘growth’ (Fig. 8).

Assessment direction

The current assessment practice in the medical and HPE fields is competency-based assessment [95–97]. Consequently, it is crucial to communicate with students clearly about these competencies and how they will be evaluated [98]. The ‘assessment direction’ involves directing students towards assessment by providing them with clear instructions on how the assessment will be administered and what is expected of them. Knowing what is expected of them during assessment will enable them to shape their learning and direct their efforts to achieve these objectives [99]. A considerable body of studies has shown that defining the assessment expectations (objectives) will allow students to target their efforts to achieve them [98, 100]. However, there is an issue in terms of which assessment criteria and standards should be communicated to students. There is no research consensus on the suitable methods for communicating with students. In our framework, we provide a variety of strategies examiners could use to direct students toward assessment.

Articulating the assessment direction has benefits for both the examinee and examiner. The examinee can then tailor his or her efforts to the assessment direction. The examiner can select suitable assessment modalities to measure the desired outcomes [96]. Additionally, it reduces the burden on examiners related to answering examinees’ questions about exams. Moreover, examinees’ test anxiety will be reduced, and they will have more agency in meeting the expectations [101]. In summary, directing students toward assessment will enhance their self-control, supporting self-regulated learning [93], and empower them to grow and face future challenges [100].

Assessment preparation

When guiding students in the assessment direction, it is important to ensure that they have the proper tools to succeed. Hence, assessment preparation plays an important role in meeting the challenge of assessment. Studies have shown that helping students control their negative thoughts and advising them on learning skills and time management will maximise their learning [102, 103]. In our framework, we believe that teachers play the central role in maximising learners’ behaviour. In addition to preparing high-quality learning materials, teachers can also provide students with rich resources to improve their mental well-being and reduce the negative effects of assessment. This practice will enhance students’ self-control and shift their mindset so they can use the resources around them to face assessment challenges.

Examiner focus

This framework highlights the examiner’s conduct during the exam. The direct encounter between examiner and examinee has a psychological dimension, creating life-altering memories that can either destroy or reinforce the examinee’s self-esteem and, consequently, resilience. While the presence of an examiner in the examination room automatically causes test anxiety, the situation will become worse if there is a lack of proper communication or if the examiner chooses to fail students based on personal preferences or biases [51]. This negative situation has been dubbed ‘the hawk effect’ [104]. Our framework proposes techniques to mitigate this unintended consequence and foster an atmosphere conducive to reciprocal communication and learning, which automatically enhances students’ ‘engagement’ and prepares them for similar situations in the future by managing learning resources wisely and effectively.

Student reflection

Self-assessment (or self-reflection) has been proven to be an effective approach to support learning engagement. In self-reflection, students evaluate their performance related to both internally set goals and externally set criteria [105]. In this framework, teachers (assessors) provide systematised avenues for self-reflection to achieve the desired resilience construct: growth. While several studies have presented different approaches for promoting self-reflection, we encourage assessors to use the most common reflection method: feedback [100]. Nicol and Macfarlane‐Dick [68] described the most important aspect of high-quality feedback: ‘Good quality external feedback is information that helps students troubleshoot their own performance and self-correct; that is, it helps students take action to reduce the discrepancy between their intentions and the resulting effects’ [68]. Numerous studies on HPE have promoted the use of feedback [106], and it is key component in the programmatic assessment framework [107]. Both feedback and self-reflection support each other to maximise learning and hence ‘growth’ [108]. Through this feedback, students will receive constructive comments regarding their performance based on the teacher’s expectation or established criteria, which they can then use internally to redesign a suitable learning path to achieve their goals [100]. Studies have shown that students’ self-reflection leads to deep learning, self-efficacy, self-regulation, and personal growth [93, 106]. While some researchers argue that students should be trained in self-regulation, others contend that self-regulation is a spontaneous process that can be maximised by providing a suitable platform to practice it [109]. Consequently, the framework provides additional resources for assessors to promote self-reflection.

Limitations and future perspectives

During the literature review phase of developing the SAR framework, efforts were made to broaden the search of the scoping review to include literature in HPE rather than just medicine, and to conduct narrative reviews that considered higher education in general. However, the FGD only included one medical school. Another limitation of this research was that it only included articles written in English, which may introduce bias (also known as language bias [110]) and result in the omission of important cultural contexts and necessary details in data synthesis. However, the triangulation of the findings with those from the scoping review, other narrative reviews, and the FGD mitigated the aforementioned limitations [45].

Conclusions

Resilience is proven to be the desired construct for medical and health professions students. It fosters several characteristics graduates need to meet future challenges and adversities. The current study presents a systematic method for fostering student resilience through assessment practice. Based on rigorous methodological research and a theoretical foundation, the study provides a set of practical strategies for promoting resilience. Through five phases of assessment, namely, assessment direction, assessment preparation, assessment experience, examiner focus, and student reflection, the SAR framework aims to promote four resilience constructs: self-control, management, engagement, and growth.

Abbreviations

CVI

Content Validity Index

FGD

Focus Group Discussion

FVI

Face Validity Index

HPE

Health Professions Education

SAR

Systematic Assessment for Resilience

TA

Test Anxiety

Authors’ contributions

Each author made a substantial contribution to this paper. MW, MSBY, and MHT performed the three narrative reviews, selected the relevant papers, and compiled them in tables. MW, MSBY, AFA, and NZ performed the scoping review and FGD. MW, MSBY, and AFA generated the SAR guidelines and configure their relationship with the findings of the scoping review and FGD. MW, MSBY, MHT, and SS laid out the SAR framework in its final form. MW initiated the writing of the paper. Both MW and SS contributed to describing the SAR framework. MHT, AFA, NZ, SS, and MSBY reviewed the framework carefully. The author(s) read and approved the final manuscript.

Funding

This research is part of a larger project supported by the Fundamental Research Grant Scheme (FRGS/1/2018/SSI09/USM/02/2), Ministry of Higher Education, Malaysia.

Availability of data and materials

Please email the corresponding author for a link to the de-identified datasets. However, due to privacy concerns, the FGD transcripts are unavailable to the general public.

Declarations

Ethical approval and consent to participate

Ethical approval was obtained from the human research ethics committee at the School of Medical Sciences, Universiti Sains Malaysia (JEPeM USM Code: USM/JEPeM/18060286), in accordance with the Belmont Report and Helsinki Declaration. All participants signed a written informed consent form containing clear information about the study’s purpose, methods, and secure data handling procedures. They were also informed that they could withdraw from the study verbally or in writing at any time.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.