The clinical learning environment in anaesthesiology in Kerala---Is it good enough?---A web-based survey

Abstract

Background and Aims:

Evaluation of postgraduate (PG) training is important so that necessary modifications can be made. The quality of anaesthesiology teaching in our country has not been explored. We performed a survey among the anaesthesiology PGs of Kerala to assess their perceptions about their learning environment as well as to look for the deficits in the training program.

Methods:

An online web-based survey was conducted. The questionnaire was developed after a detailed validation process and circulated via Google forms. The questionnaire had three parts. First part asked for demographic details. Second part had 22 statements in four domains to evaluate the quality of training, which were to be answered based on a five-point Likert scale. The third part asked for the suggestions of the trainees on changes to be made for the improvement in quality of their training.

Results:

The response rate was 64%. The social atmosphere was perceived positively in general by our PGs. Majority of them received feedback on their work and had a regular class schedule. Only 34% of the residents got training in non-technical skills. More tutorials and case discussions were the most common change suggested by the residents. Lack of research training and training in simulation labs were the major lacunae identified.

Conclusion:

The clinical learning environment (CLE) was perceived to be of medium to high quality by the anaesthesiology PGs of Kerala. Lack of routine assessments and regular academic sessions were reflected in the responses.

INTRODUCTION

Anaesthesiology postgraduate (PG) training is complex since it happens in a high stakes environment like the operating room. A positive clinical learning environment (CLE) conducive to learning is crucial for proper teaching and training. Measuring the learning environment throws light on the quality of an educational program and gives guidance on the amendments to be made.[1,2,3,4] The anaesthesia CLE in our country has not been adequately explored. So, we conducted a survey among the anaesthesiology PGs of Kerala. The primary objective was to evaluate the quality of the training in Kerala as perceived by the anaesthesiology PGs in the state and the secondary objectives were to assess the deficits and to identify possible areas of improvements.

METHODS

Institutional ethics committee approval was obtained. The standards for reporting qualitative research were followed.[5] An online web-based survey was planned. The questionnaire was developed by two clinical investigators after thorough literature search. The questionnaire was then validated for content by two senior faculties from two medical colleges and a senior anaesthesiologist working in a private institution accredited by the National Board of Examinations. Some revisions were suggested and the revised questionnaire was further validated. Face validation was done by administering the survey to general surgery PGs. Pretesting the questionnaire was done by a senior resident who had recently graduated. Piloting the questionnaire was done in 10 anaesthesiology residents whose results were excluded from the final analysis. The average time taken to answer the survey was 8.5 min.

The final questionnaire [Annexure 1] was composed of three parts. Part A dealt with the demographic details. Part B contained 22 statements which were divided across four domains of social environment in the operating room, supervision and feedback, learning opportunities and training programme and resources. The respondents had to answer on a five-point Likert scale anchored from Strongly agree’ to ‘Strongly disagree’. Part C looked for possible areas of improvement and invited suggestions from the PGs for the betterment of their training programme.

Construct validity was assessed by exploratory factor analysis through principal component method. Intercorrelation of the items in the questionnaire was assessed using the Bartlett test of sphericity, which revealed that the items were intercorrelated (P value <0.001). Kaiser Meyer Olkin (KMO) measure of sampling adequacy was 0.888, which confirmed adequate amount of shared variance among the items. Cronbach's alpha for all the items was 0.8731 which showed good internal consistency. Principal component factor analysis and correlation matrix of the items were done to extract the factors. Two factors with eigenvalue >1.0 were kept and interpreted, which indicated that question numbers 1-3, 5—10, 12, 14-16 and 19 were enough to capture the 76.1% of variance measured by the 22-item questionnaire. As the content validity of the scale was better established using qualitative methods, we retained the 22-item questionnaire for data collection.

The survey was conducted in the months of March and April 2020. The questionnaire was formatted on Google Forms and the link to the form was sent to the participants by messaging on their WhatsApp numbers, with a covering message explaining the purpose of the survey. The contact information of the PGs was collected before beginning the survey. Strict anonymity of the responders was maintained. Three reminders were sent, one every week following the first message. Reminders were sent to all since non-responders could not be identified. The survey was made available online for 6 weeks. All PGs studying for the Doctor of Medicine (MD) and Diploma of National board of examinations (DNB) course in Kerala at the time of the survey were included in the study. The participation was voluntary and responding to the survey was considered as consent to participate in the study.

Statistical analysis was done using Software for statistics and data version 14.2 (StataCorp, College Station, TX, USA). Demographic data was analysed using descriptive statistics. The Likert scale was given scores for each level as follows: Strongly Agree – 5, Agree – 4, Neutral – 3, Disagree – 2, Strongly Disagree – 1. Question numbers 4 and 9 were reverse coded during analysis. Scores were added up to calculate total score of each domain and displayed as mean with standard deviation. Some data is displayed in frequency and percentage. The scores followed normal distribution; hence unpaired t-test was used to compare the score between the residents of government and private institutes and between PGs pursuing MD and DNB courses in anaesthesiology.

RESULTS

The survey link was sent to 374 residents of which 242 responded. Response rate was 64.7%. There are 31 PG institutes in Kerala conducting MD in 17 and DNB in 14 institutes. Seven hospitals are in public sector and the rest in private sector. Of the respondents, 171 (70.6%) residents were doing MD and 71 (29.4%) were doing DNB. 129 respondents (53.3%) were from private institutes. Mean age of the respondents was 30.6 ± 2.82 years. Fifty-nine residents were males and 183 were females. There was no statistically significant difference in the demographics between the residents of government and private institutes and between the students doing MD and DNB.

From the overall responses, the domain of social environment scored the highest and domain of supervision and feedback scored the least. Of the individual statements, only the statement on training in simulation labs was given a score below the middle value (2.5). The response to the domain of social atmosphere in operating room is shown in Figure 1. 50% of the residents felt their teachers were fair in their evaluation and that their trainers helped to build their confidence. 60% residents received positive feedbacks and corrections on their work. Half of the responders shared that they got demoralised when they were blamed in the face of an adverse event. 48% of the residents felt burdened by the workload and stressed out. Majority of them got adequate opportunities to develop technical skills like intubation, but only 34% of the residents got training in non-technical skills. 59% of the students had a regular class schedule. Less than half of them had regular case discussions. 43% of the residents had routine assessments done as theory and practical exams. 62%of the responders had access to learning facilities like internet and journals in their institute. 62% agreed that they get adequate rotational postings in all surgical specialities. Simulation lab training was available only to 20% of the students. 53% of the students felt encouraged to do research.

Figure 1.

Responses of the residents on the social atmosphere in operating room

The mean scores of the responses of the residents from government and private institutes were compared [Table 1]. The total scores of each domain were also compared between the two groups [Figure 2]. Comparison was performed between the mean scores of the responses of PGs doing MD and DNB [Table 2]. The total scores of each domain were also compared [Figure 3]. The response to the last question which looked into the changes suggested by PGs is given in Figure 4. There was an open-ended question asking for recommendations by the residents for modifications in their training. Of the responses, the most recurring themes were autonomy in operating room, reducing the mental stress and creating a more balanced workload.

Table 1.

Comparison of scores between the residents of government and private institutes

Domain and statement	Government Mean (SD)*	Private Mean (SD)*	P
Social environment in the operating room
I feel part of a team in my theatre	4.04 (0.8)	4.05 (1.0)	0.924
I have a friendly relationship with the theatre staff	4.00 (0.8)	4.23 (0.6)	0.002
I can openly discuss my doubts with my teachers	3.27 (1.0)	3.73 (1.1)	0.001
There is gender discrimination in my department||	2.84 (1.3)	3.56 (1.1)	<0.001
Supervision and feedback
I discuss the anaesthetic plan with my teacher posted with me	3.58 (1.1)	3.72 (1.0)	0.271
My teachers are fair in their evaluation	3.21 (1.0)	3.43 (1.1)	0.117
My teachers help to build my confidence	3.27 (1.0)	3.47 (1.2)	0.154
I receive positive feedback and necessary corrections on my work	3.34 (1.1)	3.54 (1.1)	0.145
I get blamed in the face of an adverse event, which demoralises me||	2.28 (0.9)	2.70 (1.2)	0.003
My workload is fine and I don’t feel stressed out	2.92 (1.1)	2.51 (1.3)	0.009
Learning opportunities
I get adequate opportunities to develop my technical skills	3.98 (0.9)	4.04 (1.0)	0.655
My teachers give specific importance to developing my non-technical skills	2.88 (1.1)	2.97 (1.0)	0.545
I feel that most of what I learn is relevant to my career	4.15 (0.7)	4.01 (0.9)	0.162
There is a regular class schedule	3.71 (1.3)	3.22 (1.2)	0.003
Case discussions are given equal importance as theory classes	3.47 (1.2)	3.04 (1.2)	0.005
Regular assessments are done as theory and practical exams	3.10 (1.3)	2.93 (1.2)	0.291
Training programme and resources
I have access to internet and journals in my institution	3.18 (1.3)	3.90 (0.9)	<0.001
I get adequate rotational postings in all surgical specialities	3.42 (1.2)	3.48 (1.1)	0.664
I am relieved from my department to attend CMEs†	3.38 (1.1)	3.43 (1.2)	0.756
We get training in simulation labs	2.43 (1.0)	2.32 (1.1)	0.438
I am given training in BLS‡/ACLS§	3.55 (1.2)	3.70 (1.3)	0.326
I am encouraged to do research	3.19 (1.0)	3.51 (1.0)	0.015

*Standard Deviation, ^†Continuing medical education, ^‡Basic life support, ^§Advanced cardiac life support, ^||Reverse scoring done

Figure 2.

Comparison of scores of domains between government and private institution. *Statistically significant

Table 2.

Comparison of scores between MD and DNB residents

Domain and statement	MD (n=171) Mean (SD)*	DNB (n=71) Mean (SD)*	P
Social environment in the operating room
I feel part of a team in my theatre	3.99 (0.9)	4.15 (0.9)	0.209
I have a friendly relationship with the theatre staff	4.04 (0.8)	4.42 (0.6)	<0.001
I can openly discuss my doubts with my teachers	3.30 (1.1)	4.03 (0.9)	<0.001
There is gender discrimination in my department||	3.00 (1.2)	3.76 (1.1)	<0.001
Supervision and feedback
I discuss the anaesthetic plan with my teacher posted with me	3.55 (1.1)	3.90 (0.9)	0.015
My teachers are fair in their evaluation	3.18 (1.1)	3.70 (1.0)	<0.001
My teachers help to build my confidence	3.22 (1.1)	3.77 (1.0)	<0.001
I receive positive feedback and necessary corrections on my work	3.32 (1.1)	3.75 (0.9)	0.006
I get blamed in the face of an adverse event, which demoralises me||	2.30 (1.0)	3.01 (1.2)	<0.001
My workload is fine and I don’t feel stressed out	2.72 (1.2)	2.65 (1.2)	0.654
Learning opportunities
I get adequate opportunities to develop my technical skills	4.05 (0.9)	3.92 (1.2)	0.322
My teachers give specific importance to developing non-technical skills	2.92 (1.1)	2.96 (1.0)	0.794
I feel that most of what I learn is relevant to my career	4.10 (0.7)	4.01 (0.9)	0.446
There is a regular class schedule	3.62 (1.2)	3.03 (1.2)	<0.001
Case discussions are given equal importance as theory classes	3.50 (1.1)	2.60 (1.1)	<0.001
Regular assessments are done as theory and practical exams	3.13 (1.2)	2.72 (1.2)	0.017
Training programme and resources
I have access to learning resources like internet and journals	3.42 (1.2)	3.92 (1.0)	0.002
I get adequate rotational postings in all surgical specialities	3.44 (1.2)	3.48 (1.1)	0.805
I am relieved from my department to attend CMEs†	3.30 (1.2)	3.66 (1.0)	0.024
We get training in simulation labs	2.54 (1.1)	1.98 (0.9)	<0.001
I am given training in BLS‡/ACLS§	3.66 (1.1)	3.56 (1.4)	0.578
I am encouraged to do research	3.32 (1.0)	3.46 (1.2)	0.320

*Standard Deviation, ^†Continuing medical education, ^‡Basic life support, ^§Advanced cardiac life support, ^||Reverse scoring done

Figure 3.

Comparison of scores of domains between MD and DNB residents * Statistically significant

Figure 4.

Modifications to the training suggested by the postgraduates

DISCUSSION

The CLE was perceived to be of medium to high quality by the anaesthesiology PGs of Kerala, with major lacunae in some areas. Comparison between government and private institutes showed significant differences in total scores with private institutes scoring better in the domains of social atmosphere and training programme and resources. Between residents doing MD and DNB, there were significant differences in several areas. DNB students gave higher scores to social environment and supervision and feedback, while MD students gave higher scores for the domain of learning opportunities.

In general, the social atmosphere in the operating room seemed to be perceived positively by majority of our residents. The climate in the learning environment has a great impact on the potential quality of learning. World federation of medical education states that one of the core targets for evaluation of a teaching program is the learning environment.[6] 48% of our residents felt they were stressed. Gandhi et al.[7] reported similar findings in their survey.

Less than half of the residents agreed to having regular case discussions and assessments in their institution. Having more tutorials and case discussions was the change suggested by most of the residents. Problem-based learning (PBL) is gaining more importance and case-based discussion is the most common form of PBL practised in anaesthesia.[8] The need for improved academics was seen in a similar survey.[9] Regular class schedule and case-discussions were better in government institutions compared to private as per our study findings.

Feedbacks and constructive criticism help residents to identify and rectify gaps in performance.[10] Half of the residents agreed that their teachers gave productive feedback and helped in building their confidence. Planning perianesthestic care and providing feedback were two qualities of a good supervisor identified by the anaesthesiology PGs.[11,12] In medical teaching, formative assessment is now being given more significance.[13] The feedback should be timely and specific so that the resident has opportunity to improvise.[14] Tests should be used not only as a measure of performance but also should be taken as a learning opportunity.[14] 32% of the residents asked for more frequent internal assessments. Though national board of examinations conducts regular formative assessments, Kerala University of Health Sciences has no such provision in its curriculum. Research training was seen to be inadequate in our survey. A study by Freundlich et al.,[15] showed that a dedicated month of training in research improved the performance of the students.

Our survey revealed that training in simulation labs was very low (20%). Our own survey had demonstrated the desire of the attendees of a workshop to make simulation mandatory in the PG curriculum.[16] Learning is a complex process. Spacing learning opportunities and repeating the skills is of paramount importance.[17] This is accomplished by simulation especially for training our residents to perform well in critical situations.[18,19]

The DNB candidates in our survey gave significantly higher scores in multiple areas. This was a surprising finding and maybe an indication of the declining quality of the training program in institutions providing MD course. We speculate one reason for this to be increasing the number of seats without making major changes in the rest of the requirements of the academic programme.

Our survey was conducted online and had a good response rate of 64% response. The non-responders could not be identified and so non-responder bias was not assessed. The respondents may have felt inhibitions in being completely honest fearing repercussions even though strict anonymity was promised. This survey showed the general perceptions of our PGs, and the wide differences that may exist between institutions. Each institution needs to make reforms based on evaluation of opinions of their students. There should also be provision for PGs to evaluate their teachers (feedback).

CONCLUSION

The responses to our survey demonstrated that the quality of training in our state was perceived positively though some major lacunae were seen. This gives us directions for improvising the anaesthesiology PG training program in Kerala. Success of educational reforms ultimately lies with individual institutions and the evaluation should be done at institutional level. Indian Society of Anaesthesiologists (ISA) can conduct such pan- India surveys and take measures to improve the anaesthesiology training in our country.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

ANNEXURE 1

Questionnaire

A. Age : Sex:

Course : MD/Primary DNB/Secondary DNB

Institution: Private/Government

B. Please respond to the following statements as you feel is relevant to your postgraduate training based on the scale given below : (Please tick)

Strongly agree - SA Agree - AG Neutral – N

Disagree - DA Strongly Disagree - SD

Statement
Social environment in the operating room
I feel part of a team in my theatre	SA	AG	N	DA	SD
I have a friendly relationship with the theatre staff	SA	AG	N	DA	SD
I can openly discuss my doubts with my teachers	SA	AG	N	DA	SD
There is gender discrimination in my department	SA	AG	N	DA	SD
Supervision and feedback
I discuss the anaesthetic plan with my teacher posted with me	SA	AG	N	DA	SD
My teachers are fair in their evaluation	SA	AG	N	DA	SD
My teachers help to build my confidence	SA	AG	N	DA	SD
I receive positive feedback and necessary corrections on my work	SA	AG	N	DA	SD
I get blamed in the face of an adverse event, which demoralises me	SA	AG	N	DA	SD
My workload is fine and I don’t feel stressed out	SA	AG	N	DA	SD
Learning opportunities
I get adequate opportunities to develop my technical skills	SA	AG	N	DA	SD
My teachers give specific importance to developing my non- technical skills	SA	AG	N	DA	SD
I feel that most of what I learn is relevant to my career	SA	AG	N	DA	SD
There is a regular class schedule	SA	AG	N	DA	SD
Case discussions are given equal importance as theory classes	SA	AG	N	DA	SD
Regular assessments are done as theory and practical exams	SA	AG	N	DA	SD
Training programme and resources
I have access to learning resources like internet and journals in my institution	SA	AG	N	DA	SD
I get adequate rotational postings in all surgical specialities	SA	AG	N	DA	SD
I am relieved from my department to attend CMEs	SA	AG	N	DA	SD
We get training in simulation labs	SA	AG	N	DA	SD
I am given training in BLS/ACLS	SA	AG	N	DA	SD
I am encouraged to do research	SA	AG	N	DA	SD

C. What changes would you like to be made in the training programme at your institution?

Please tick the OPTIONS you feel are most relevant to you

1) More hands-on with procedures

D. Central Venous Access

E. Central Neuraxial Blockade

F. Invasive monitoring

G. Nerve blocks

H. Difficult airway equipment. Eg: FOB

2) More tutorials and case discussions

3) Frequent internal assessments

4) Simulation based training

5) Better training in research methodology

6) Lectures by visiting faculty

7) Any others, please suggest