Selection for specialist training: what can we learn from other countries?

Abstract

The chaos surrounding the UK's centralised application service led to the system being abandoned. Tony Jefferis examines how similar systems work elsewhere

The UK Medical Training Application Service (MTAS) for specialist training had a bad start. Doctors were angry at its inefficiencies and unfairness.^{1 2} Many reasons were cited for its problems including using an untried system,³ using the same system for new entrants as for those already committed to a specialty, underestimating the numbers of international medical graduates applying, using a flawed computer system, and, finally, buckling to public outcry by revising the timetable and conditions of application. The secretary of state for health apologised in the House of Commons, the website closed for a security review, and the system was eventually abandoned.

Despite all this, a central application portal with local selection has considerable merit. It has been used successfully in the United States, Canada, and, in a modified form, in Australia and New Zealand for at least 30 years (box). It provides an orderly and transparent way for candidates to decide where to train and for programme directors to decide whom to enroll into postgraduate medical training. So how do these countries make it work?

National application portals for specialty medical training

United States

• Electronic Residency Application Service (ERAS) is run by the Association of American Medical Colleges (www.aamc.org/students/eras/). It was computerised in 1996

Canada

• Canadian Resident Matching Service is run by the Association of Medical Colleges (www.carms.ca/). It was computerised in 2002

Australia and New Zealand

• The process is run by individual specialty colleges. The Royal Australasian College of Surgeons, for example, computerised in 2007⁴

Selection process

All four countries have a clearly publicised timetable, outlining each step of the process. The US, Australia, and New Zealand have one round of applications each year. Canada has two, the second being primarily for international graduates and candidates not matched in the first round. Candidates are given two to three months to organise an application before submission.

Candidates can make an unlimited number of applications, to as many programmes as they wish. What restricts the number of applications is the cost, the time involved, and the likelihood of success. All schemes charge their applicants for the process. In the US the charge is $60 (£30; €44) for 10 applications, more pro rata, and Canada charges $205 (£97; €143; $194) for four applications, more pro rata. In Australia and New Zealand the charge for registration as a surgical trainee is AU$3900 or NZ$4350 (£1640; €2420; $3280) for registration, allowing an unlimited number of applications. Other specialties cost less.⁵

Application form

All have application forms with similar content, covering undergraduate and graduate education, medical education, previous training, honours and prizes, research and publications, and extracurricular and community activity.

The process of being admitted to specialty training begins before qualification in the US and Canada. Medical school is a highly competitive graduate programme in both countries. The specialty application form requires details of the candidate's achievements and aptitudes as a medical student. Students take the licensing examination before matching in the US, but after in Canada. Weight is given to applicants who have research experience and peer reviewed publications. Extracurricular activity is also graded, with credit being given to those who have important and consistent leadership roles. In Canada, a major determinant of eligibility for a programme is appropriate electives.

Applicants for specialty training in Australia and New Zealand are qualified doctors who are in basic medical training, similar to the foundation training in the UK. Until this year they were then selected for core training in a specialty, and after two to four years applied for higher specialty training programmes. In 2007 specialty training in surgery is changing to a single programme called Surgical Education and Training. Like the programme in the UK this combines core and specialist training. Doctors wishing to enter specialist training register with the college while in basic medical training, before submitting their application.⁴

International applicants are given a special track in all four countries. They have to show eligibility for each of the programmes and they are considered after local graduates.

Reports and references

All four countries' application procedures use reports and references to help with selection. In the US, candidates are asked for letters of recommendation from the dean of their medical school and other referees. The references cover a series of professional attributes and are sometimes supplemented by telephoning the referees. In the past some candidates had been given misleading references, and programme directors took steps to improve this through direct contact with the referee using questions based on the Accreditation Council of Graduate Medical Education competencies.

Canada has a similar system, requiring a letter from the dean and letters of reference and recommendation, especially from elective supervisors. These letters state the type and duration of the referee's contact with the individual and comment on cognitive skills, knowledge, problem solving, patient management, behaviour, attitudinal skills, communication skills, working relationships, ability to work in a team, motivation, punctuality, sense of responsibility, and aptitude relevant to their chosen discipline. Applicants also need to supply a transcript of their medical school achievements and a personal statement.

As entry to specialist training in Australia and New Zealand follows basic medical training, references come from the candidate's supervisors. Details of what is required vary, but there are some consistent principles. The candidates must supply between five and eight referees. These are their supervisors or managers they have worked for over the past two years. Some or all of these referees are approached to give a structured reference, either over the telephone or by completing a standard form. The domains include clinical skills, surgical competence, potential, and the ability to interact with patients and others.

Interview

In the US and Canada each institution grades the application form, letters of recommendation, personal statement, dean's letter, research experience, extracurricular activity and community involvement and then makes an overall evaluation. Candidates ranked most highly are called for interview. The ratio of candidates to places varies but can be as high as 10:1 since there is no certainty about how the candidates will rank the institution.

At most interviews there are several interviewing stations, usually with two faculty members in each. The candidates are all asked the same questions. Rating is done on the candidate's appearance, communication skills, maturity, self confidence, ability to work effectively, compatibility with the programme, and overall rating. In some programmes current residents rate the candidate's suitability.

In Australia and New Zealand the colleges arrange the interviews. They are held either centrally in the individual colleges for smaller specialties or in the state capitals for larger specialties. The interview is conducted with several stations rather than a panel. The questions are semistructured, with each candidate being interviewed on similar subjects. The interviewers rate the candidate's communication skills, personal presentation and character, decision making ability, clinical knowledge, and professionalism. The competition ratio varies but, for example, this year in surgery it is about four candidates to one place.

Ranking

In all four countries the programme directors rank all the candidates in order of preference using the application form, the references, and the interview. The weight given to each section varies, but the range is 20-35% for the application form and academic profile, 35%-40% for references, and 35-40% for the interview. Individual programmes use different criteria for tie breakers when candidates have identical scores—for example, the medical licensing examination result or the overall consensus of the faculty.

Matching

The candidates' preferences and those of the programmes are matched centrally. There is no controversy about this; programme directors and trainees are happy with the process. The number of candidates selected who fail to complete their programme is small. However, change does bring difficulties, and trainees in New Zealand were concerned that some potential specialists were denied entry to their chosen field in the transition to the surgical education and training system this year. The number of candidates getting their first choice specialty varies: 95% in Canada, not necessarily at their institution of choice, and only 50% in Australia in popular surgical specialties.

UK directions

The UK needs to design a medical specialty application process for the long term that is fair, just, and efficient. Medical graduates are highly talented and have cost the UK taxpayer at least £150 000 to train. The process of embarking on specialty training should be transparent and straightforward as well as competitive. Not everyone wishing to specialise in a glamorous and highly sought after specialty will succeed, but they will know the entry requirements and whether they meet them.

What can we learn from existing matching systems? Firstly, they are all efficient, have clear timetables, and are consistent from year to year. There is time for candidates to make informed choices and to compile their application.

Application forms contain predictable questions about the evolution of a professional career. One of the major concerns about the recent UK experience was that the forms were available for only two weeks, and candidates had to describe their professional achievements in ways which relied heavily on linguistic dexterity. Perhaps the UK should adopt a more conventional scheme for application forms.

Some aspects of medical training are not mentioned in the UK application forms, in an attempt to be fair. For example, the name of the medical school and country where a doctor trained is purposely kept from the selection panel. This is good equal opportunity practice. However, the pooling of international medical graduates with those from the UK and the rest of Europe has been one of the reasons that the system was oversubscribed. Although international graduates are an integral part of the medical workforce in the US, Canada, Australia, and New Zealand, their applications are considered only after domestic graduates. This is not possible in the UK because of employment law. Should we change?

The references and supporting letters of recommendation are important in ranking the candidates. The matching systems all use references based on a series of professional attributes and some supplement the information by personal contact. References in the UK tend to be bland, rarely giving an accurate picture of the candidate. Even the current 360° summaries on each foundation trainee tend to group at the upper end of the scale. The UK needs to develop a more discerning system of references to differentiate between candidates. This could then form part of the ranking process. The current system with references based on the attributes of Good Medical Practice could be adapted by using up to six referees and developing a consistent scoring system.

In many ways the UK interview, with semistructured questions and standardised evaluation, is as fair as elsewhere. Nevertheless, candidates need to know that they will be evaluated on their professionalism, communication skills, decision making, character, and clinical ability at interview.

There is much to put right in the current medical application process. Some specialties have already done so, notably general practice and histopathology. It would be unfortunate if all progress was abandoned and the system reverts to a past that was not as golden as it is sometimes painted.

Summary points

• The US, Canada, Australia, and New Zealand have computerised selection for postgraduate training

• All use a central portal and local selection with a clearly defined timetable

• Ranking is based on an application form, multiple references, and semistructured interviews

• The UK system would benefit from changes to the application process and better references