Trainee satisfaction before and after the Calman reforms of specialist training: questionnaire survey

Abstract

Objectives

To evaluate the impact of the Calman reforms of higher specialist training on trainee satisfaction.

Design

Questionnaire surveys using portable electronic survey units, two years apart.

Setting

Postgraduate, teaching, district general, and community NHS trusts in North Thames. North Thames deanery includes London north of the Thames, Essex, and Hertfordshire.

Participants

Trainees in all grades and all specialties: 3078 took part in the first survey and 3517 in the second survey.

Main outcome measures

Trainees' satisfaction with training in their current post, including educational objectives, training agreements, induction, consultant feedback, hands on experience acquired, use of log books, consultant supervision, and overall satisfaction with the post.

Results

In the second survey respondents were more likely to have discussed educational objectives with their consultant, used a log book, and had useful feedback from their consultant. They were more likely to give high ratings to induction, consultant supervision, and hands on experience acquired in the post. Each of these elements was associated with increased satisfaction with the post overall. Improvements were most noticeable at the level of specialist registrar, but changes in the same direction were also seen in more junior grades.

Conclusions

After the reforms of specialist training, trainees in all grades reported greater satisfaction with their current posts. The changes required extra training time and effort from consultants.

Introduction

In 1992 Kenneth Calman, then chief medical officer, set up a working group to bring the British system of specialist training into line with the requirements of the European medical directives. The resulting report recommended combining the registrar and senior registrar grades into a unified specialist registrar grade and defining the curriculum and minimum training period for each specialty, the successful completion of which would lead to admission to the specialist register.¹ Features of the new system were set out in A Guide to Specialist Registrar Training and included educational objective setting, training agreements, and induction at the start of each placement; rotational placements designed to offer specified experience; and regular feedback on progress from the supervising consultant.² The reforms were to be cost neutral, and no additional resources were made available to NHS trusts for their implementation. Transition to the new system began in December 1995 and was completed in April 1997.

The new arrangements placed more emphasis on structured teaching and supervised learning and less on experiential apprenticeship.³ Although the reforms received a cautious welcome,^4–6 there were anxieties about the impact of shortening the training time on trainees' experience⁷ especially since junior doctors' hours were being reduced.⁸ It was expected that the reforms would lead to fewer trainees and more consultants.⁹ Consultants feared they would be expected to take on extra responsibility for out of hours emergency work, including resident on-call,¹⁰,¹¹ although this was denied by senior doctors involved in planning the reforms.¹²,¹³ The implementation of the reforms seemed likely to create a major new training workload for consultants.¹⁴ At the same time they were losing control over appointment of their own junior colleagues, who were to be appointed to regional programmes and allocated to posts by a regional committee. It was not clear whether the reforms could be delivered without additional resources, how motivated the consultants would be to implement change, whether higher specialist trainees would consider their training improved, or what the impact would be on the training of more junior grades.

To evaluate the impact of the reforms on specialist registrars (including old style registrars and senior registrars) and any knock-on effects on more junior grades (preregistration house officers and senior house officers), we planned two surveys of trainees in all grades and all specialties in our region: one during transition to the new system and one two years later.

Participants and methods

Surveys

The first survey took place during transition in November and December 1996, with postal questionnaires to non-responders in January 1997. The second survey took place 18 months after transition was completed in November and December 1998, with postal questionnaires to non-responders in January 1999. Although some trainees would have been surveyed twice, it was unlikely any would have been in the same post two years later. The questions were developed from those used for monitoring senior house officer training,¹⁵ modified to be appropriate for all grades. The questionnaire was loaded on to portable electronic survey units. Each question was displayed on a screen, and the participant keyed in the number of the response chosen. Confidentiality was assured. Each trust was provided with from one to five survey units for one to two weeks. A contact person (usually the postgraduate centre manager) organised the survey locally by publicising the survey to trainees, receiving the units and ensuring they were accessible to the trainees, preparing a list of trainees, and making a note of those who responded or those who were unavailable to respond (for example, on leave, moved on). Non-responders were chased up by postal questionnaires.

Participants

Our study sample was defined as all trainees available to respond at the time of the survey in participating NHS trusts in North Thames. We excluded doctors not in training grades and locums who had been in post less than two weeks. We made no attempt to contact trainees not based in participating trusts—for example, those in public health, industry, hospices, private hospitals, or general practice.

Statistical analysis

We used SPSS software (version 8.0) for our analyses. We calculated significance with the Mann-Whitney U test for ordinal data and the χ² test for categorical data.

Results

Respondents

Fifty nine of 61 NHS trusts agreed to take part in the first survey. Of the 4250 trainees available, 3078 responded (response rate 72.4%). All trusts agreed to take part in the second survey. Of the 4765 trainees available, 3517 responded (73.8%). The table details the characteristics of the respondents. The distribution of respondents by grade and specialty did not differ importantly from that of the postgraduate dean's database for the year concerned. The larger numbers in the second survey reflected the participation of two more trusts, a longer duration of the survey in the bigger trusts, and a higher response rate. Not all respondents answered every question.

Training

Educational objectives and training agreements

A Guide to Specialist Registrar Training states that at the beginning of each placement the trainer and trainee should discuss educational objectives. Trainees in the second survey were more likely to report having had such a discussion with their consultant (fig 1a). The specialty most compliant with this requirement was accident and emergency medicine (92% of specialist registrars in the first survey and 97% in the second) whereas ophthalmology was the least compliant (28% v 44%). Compliance increased from 37% to 46% in the surgical specialties and from 42% to 63% in the medical specialties. Compliance increased in all types of NHS trust, the most noticeable change being in postgraduate institutions (38% to 59%) and the least noticeable in the already compliant community trusts (75% to 77%). The training guide also recommends that an individual training agreement be signed by trainee and trainer. Evidence of compliance was disappointing, despite a requirement in the educational contract between trusts and the postgraduate deans for this to be a feature of all training posts. Compliance increased from 18% to 29% among specialist registrars and from 19% to 29% among junior grades.

Induction

The training guide recommends specialist registrars have an induction to each post in a training programme. The postgraduate deans have been encouraging good induction arrangements for junior grades for several years. Since the quality of the induction determines its usefulness, we used a rating scale to measure satisfaction (fig 1b). Trainees at all levels in the second survey gave higher ratings to the induction they received. Obstetrics and gynaecology showed the most improvement, with ratings of “very poor” or “poor” decreasing from 38% to 17% for specialist registrars and 39% to 26% for junior grades. Similar improvements were seen in all types of trusts.

Consultants' feedback

Thetraining guide recommends regular informal discussions between the trainee and the supervising consultant about the trainee's progress. Consultants' feedback of this sort for all grades had been recommended by the postgraduate deans for years, and training in appraisal skills had been provided by them from 1993. Trainees in the second survey were more likely to have had useful feedback or to know that feedback was planned (fig 1c). Improvement was seen across all the specialties. At specialist registrar level trainees in all types of trust were more likely to have had useful feedback in the second survey, with the biggest improvements being in the teaching (38% to 50%) and postgraduate (28% to 40%) hospitals. For junior grades improvement was most noticeable in district general hospital (31% to 40%) or community (47% to 56%) posts.

Hands-on experience

One of the aims of structured training was to ensure that all trainees acquired the experience required by the specialist curriculum. We asked trainees to rate the experience they were acquiring in their current post. There was a slight but significant (Mann-Whitney U test 949 550, P<0.001) improvement in ratings for specialist registrars and no significant change for junior grades. No significant difference was found between specialties or type of trust.

Log books

Although there is no requirement in the training guide to introduce a log book, many specialties have developed these as a means of recording progress in gaining experience required by the specialist curriculum. Some royal colleges also issue log books to senior house officers. We found a noticeable increase between the surveys in the number of trainees in all grades and specialties who had used a log book (fig 1d). Usage among specialist registrars increased in obstetrics and gynaecology from 27% to 69%, in medical specialties from 15% to 55%, and in anaesthetics from 48% to 93%. The surgical specialties showed the smallest increase (80% to 86%). The greatest use among senior house officers was in anaesthetics (79% to 91%), and the lowest use was in accident and emergency medicine (11% to 17%). No significant difference in use of log books was found between the different types of training trust.

Supervision by consultants

Trainees' ratings of the quality of supervision by consultants improved between surveys (fig 1e). The biggest improvements were seen in medical specialties and anaesthetics. For specialist registrars the major improvement was seen in teaching hospitals and postgraduate institutes, whereas for junior grades the improvement was more noticeable in district general hospitals.

Overall satisfaction

Overall satisfaction with the post was measured by asking the respondent, “How would you describe this post to a friend who was thinking of applying for it?” The responses were on a scale of 1 to 5 where 1 was “very poor” and 5 was “excellent” (fig 1f) and showed a modest but significant improvement. The changes were in the same direction in all grades, specialties, and types of NHS trust.

Discussion

After the introduction of the Calman reforms of specialist training, specialist registrars were more likely to report satisfaction with key elements of their training. Similar, if less noticeable, improvements were noted in more junior grades.

Strengths and weaknesses

Strengths—The study included trainees from all grades, specialties, and types of NHS trust, in the largest deanery in the United Kingdom where 25% of the country's specialist registrars are trained. The two surveys were carried out at the same time of year, reducing bias from seasonal variations in workload or casemix. The technology provided assurance of confidentiality, encouraged a good response rate, and reduced risk of errors in the transfer of data.

Weaknesses—The study only sought the views of trainees and not their consultant trainers. We restricted our questioning to the current post and did not explore satisfaction with the whole rotational programme or its management. Other efforts were taking place at the same time to improve the training of more junior grades: the General Medical Council produced recommendations for improving the preregistration year,¹⁶ and several royal colleges introduced new standards for basic specialist training.

Meaning and implications

The satisfaction of specialist registrars with training within their current post increased after the implementation of the Calman reforms. The reforms did not seem to have diverted attention away from the needs of more junior grades and may have had a positive effect on their training. There was no evidence that higher specialist trainees thought they were acquiring less experience in their posts or that job satisfaction had decreased, despite expectations to the contrary.¹⁷,¹⁸ The reforms of specialist training did not take place in isolation but were part of a general trend towards educational improvement and better working conditions.¹⁹ Much of this improvement required additional efforts by consultants both in delivering training and organising training programmes. The impact of this extra work has undoubtedly been to load additional stresses on to an already overburdened consultant workforce.²⁰,²¹

Further work

Further work should address the management of the training programmes including recruitment, rotations, and the annual record of in-service training assessments. As the training period becomes shorter, fitness to take on the consultant role at completion of training should be evaluated. Further evaluation of these educational reforms regarding consultant time, effort, and stress are urgently needed, especially as the scale of consultant expansion on which the reforms were predicated has not so far materialised.

What is already known on this topic

To bring the British system of specialist training into line with European medical directives, the Calman reforms recommended combining the registrar and senior registrar grades into a unified specialist registrar grade and defining the curriculum and minimum training period for each specialty

The reforms were implemented very rapidly, were far reaching in their impact, and were not universally welcomed

What this study adds

Two years after the implementation of the reforms trainee ratings of the educational elements of their current posts had improved; educational objective setting, induction to the post, consultant feedback, and clinical supervision were all more likely to have occurred and been satisfactory

The changes were all in the same direction and affected all training grades, specialties, and types of NHS trust. The changes could not have occurred without considerable extra effort from consultant trainers and may not be sustainable without the consultant expansion on which the reforms were predicated and which has yet to materialise

Table.

Characteristics of respondents in surveys before and after Calman reforms. Values are numbers (percentages)

Characteristic	1996-7 (n=3078)	1998-9 (n=3517)
Grade
Specialist registrar	1381 (45.0)	1582 (45.1)
Junior grades	1691 (55.0)	1928 (54.9)
Sex
Male	1768 (57.4)	2063 (58.7)
Female	1310 (42.6)	1452 (41.3)
Qualified in United Kindom
Yes	2183 (70.9)	2570 (73.2)
No	895 (29.1)	943 (26.8)
Current specialty
Psychiatry	305 (9.9)	369 (10.5)
Pathology	74 (2.4)	83 (2.4)
Obstetrics and gynaecology	235 (7.6)	261 (7.5)
Medicine	874 (28.4)	855 (24.4)
Anaesthetics	266 (8.7)	307 (8.8)
Accident and emergency	118 (3.8)	163 (4.7)
Ophthalmology	87 (2.8)	67 (1.9)
Paediatrics	269 (8.8)	345 (9.9)
Surgery	571 (18.6)	672 (19.2)
Other	274 (8.9)	376 (10.7)
Trust type
Teaching hospital	906 (31.3)	1200 (34.2)
District general	1305 (45.1)	1456 (41.5)
Postgraduate	416 (14.4)	474 (13.5)
Community	266 (9.2)	376 (10.7)

Not all respondents answered every question. 

Figure.

Trainees' ratings of aspects of Calman reforms

Editorial by Catto