Factors affecting likelihood of applicants being offered a place in medical schools in the United Kingdom in 1996 and 1997: retrospective study

Abstract

Objective To assess the relation between a range of measures and the likelihood of applicants to medical schools in the United Kingdom being offered a place overall and at each medical school, with particular emphasis on ethnic minority applicants.

Design Data provided by the Universities and Colleges Admissions Service on 92 676 applications to medical schools from 18 943 candidates for admission in 1996 and 1997. Statistical analysis was by multiple logistic regression.

Main outcome measures Receipt of a conditional or unconditional offer of a place at medical school.

Results Eighteen separate measures were independently associated with the overall likelihood of receiving an offer. Applicants from ethnic minority groups were disadvantaged, as were male applicants, applicants applying late in the selection season, applicants making non-medical (so called insurance) choices, applicants requesting deferred entry (so called gap year), and applicants at further or higher education or sixth form colleges. Analysis at individual medical schools showed different patterns of measures that predicted offers. Not all schools disadvantaged applicants from ethnic minority groups and the effect was stable across the two years, suggesting structural differences in the process of selection. The degree of disadvantage did not relate to the proportion of applicants from ethnic minority groups.

Conclusions The data released by the Council of Heads of Medical Schools allow a detailed analysis of the selection process at individual medical schools. The results suggest several areas in which some candidates are disadvantaged, in particular those from ethnic minority groups. Similar data in the future will allow monitoring of changes in selection processes.

Key messages

• The Council of Heads of Medical Schools has made publicly available, on the website of the Universities and Colleges Admissions Service, detailed data on individual applications for medical school in 1996 and 1997

• These data allow analysis of factors influencing selection at individual medical schools in the United Kingdom, although some important measures such as GCSE grades, estimated A level grades, and assessments of personal attributes are not available

• In 1996-7 certain groups, in particular ethnic minority groups and male applicants, were disadvantaged during selection

• The disadvantage of applicants from ethnic minority groups seems stable across years, with some schools consistently showing no evidence of disadvantage

• Provision of similar data in the future will allow continued monitoring of selection and of the proposals for change made by the council

Introduction

The importance of annually monitoring the admissions procedure within each medical school cannot be over-emphasised. It is naturally desirable that factors such as medical parents, social class, private education, race, and sex do not become significant predictors of shortlisting or interview success, whether unconsciously determined or otherwise.¹

The Council of Heads of Medical Schools has recently decided to place anonymised data for individual applications to medical schools in the United Kingdom on the website of the Universities and Colleges Admissions Service (www.ucas.ac.uk). For the first time much of the information that is important in selecting medical students can be examined in detail. The data set is large and complex (4.5 million pieces of information on 93 000 applications from 19 000 candidates over two years). In this paper I summarise measures that are related to receiving an offer at individual medical schools and overall. These measures have been important in previous studies^2–5 or are of intrinsic interest. Not all potentially important measures were available. For example, computer readable data were not available on grades at GCSE (general certificate of secondary education, the examinations generally taken at the age of 16); predicted grades at A level (the examinations generally taken at the age of 18); the textual statements provided by applicants and referees; or the selection process within medical schools, including shortlisting and interviewing, which are often used to assess motivation and personality. A necessarily brief paper cannot do complete justice to such rich data, and a more detailed report is available on the website of the Universities and Colleges Admissions Service (www.ucas.ac.uk); computer programs for carrying out additional statistical analyses are available on request from me.

The two aims of this paper are to provide an overview of measures important in selection and to look in detail at the controversial issue of the selection of applicants from ethnic minority groups.^6–9 Important methodological concerns, to avoid the flaws of some previous studies,¹⁰,¹¹ are adequate control of confounding variables and analysis of offers rather than entrants (since candidates themselves select between schools when holding several offers¹²).

Methods

The Universities and Colleges Admissions Service provided anonymised data for all applications to medical school in the 1996 and 1997 cycles—that is, applications during autumn 1995 and 1996—for applicants with permanent residence in the United Kingdom (home applicants). During 1996 and 1997 applicants could make a maximum of six applications (although they were recommended to make five). Deferred entry (or the so called gap year) was known only for the 1997 data. The service sent non-anonymised data to medical schools for checking. Ethnic origin was self classified by applicants using standard census categories.

Data were supplied as an excel 5.0 workbook, which was analysed using spss for Windows 6.1 (syntax and system files are available on request from me). Separate spss files were produced for analyses of candidates and applications.

Statistical analysis used multiple logistic regression with simultaneous adjustment of each variable for all other factors. The dependent variable for the analysis of candidates was receipt of one or more offers and for the analysis of applications receipt of an offer. Missing values were handled by imputation of population means.¹³

A levels and Scottish highers—

Most applicants to medical schools outside Scotland take A levels after application, whereas about half of applicants to Scottish medical schools take highers, which are taken before application. Selectors typically have no information on achieved grades at A level but only GCSE grades and predicted grades at A level, which were not available in this study. In this study, however, academic achievement is described in terms of achieved A level grades, the current standard of sixth form educational achievement. As in previous studies,²,⁴,⁵ academic achievement is summarised as the number of A levels and highers taken and the mean grade attained in A levels and highers.

Significance testing—

Assessing statistical significance is potentially problematic when about 21 factors are assessed at 27 individual schools in two separate years, giving some 1100 tests. A straightforward solution for avoiding α inflation (type I error) treats as significant only results nominally significant in both years, defined as a nominal significance level of <0.01, or <0.05 on one occasion and a geometric mean less than 0.01—for example, P=0.05 and P=0.002. My unpublished report describes many additional analyses, which cannot be described fully in this short paper. When claims seem not to be fully supported by evidence further information is available in the report (available at www.ucas.ac.uk).

Results

Selection overall—

Table 1 summarises for the 18 943 candidates in the combined 1996-7 data the relation between the 24 variables and receipt of one or more offers. Eighteen predictors were significant with P<0.05 and 17 with P<0.001.

Table 1.

Effects of measures in overall analysis of data on candidates in 1996 and 1997 combined

Measure	Mean (SD) or %	Odds (95% CI) of receiving one offer or more	Significance (P value)
Educational
A level grade (A=10, B=8, C=6, D=4, E=2)	8.03 (1.70)	1.58 (1.54 to 1.62) times per mean grade point	<0.0001
No of A levels	3.19 (0.52)	1.16 (1.08 to 1.25) times per A level taken	0.0001
Highers grade (A=6, B=4, C=2)	5.16 (0.27)	2.57 (2.18 to 3.02) times per mean grade point	<0.0001
No of highers	5.86 (0.23)	1.40 (1.19 to 1.64) times per higher taken	<0.0001
One or more non-science A levels	20	0.93 (0.85 to 1.02) times if non-science A levels taken	0.140
A levels resat	11	2.39 (2.08 to 2.73) times if A levels are not resat	<0.0001
General studies A level taken	25	1.20 (1.10 to 1.31) times if general studies A level taken	< 0.0001
General studies grade at A level (A=10, B=8, etc).	3.84 (0.58)	1.23 (1.15 to 1.31) times per grade point	<0.0001
No of AS levels	0.21 (0.44)	1.07 (0.98 to 1.16) times per AS level taken	0.111
Pertaining to application form
Application date (by 15 Oct; by 15 Nov; by 15 Dec; late)	23; 47; 28; 2	1.64 (1.56 to 1.73) times per month earlier	<0.0001
Previous application	12	1.04 (0.85 to 1.28) times if applied previously for medicine	0.553
Non-medical (insurance) choice	26	1.29 (1.18 to 1.40) times if no insurance choice on UCAS form	<0.0001
No of applications to medical school:
<5	12	2.06 (1.84 to 2.32) times if five or more medical schools on UCAS form	<0.0001
6	5	0.94 (0.80 to 1.11) times if six medical schools on UCAS form	0.470
Deferred entry (gap year)	5	1.38 (1.10 to 1.72) times if no gap year indicated on UCAS form	0.0047
Demographic
Female sex	51	1.57 (1.46 to 1.68) times more likely if female	<0.0001
Mature student (aged >21)	13	2.30 (2.03 to 2.61) times if not a mature applicant	<0.0001
Social class (1=group I, 2=group II, etc)*	1.94 (0.89)	1.11 (1.06 to 1.15) times per point higher on social class scale	<0.0001
Ethnic minority group (non-white)	35	2.76 (2.56 to 3.01) times if not from an ethnic minority group	<0.0001
Type of educational establishment†:
Independent school	30	0.95 (0.85 to 1.05) times if from an independent school	0.310
College of further education or higher education	10	0.50 (0.43 to 0.57) times if from college of further education or higher education	<0.0001
Grammar school	11	1.25 (1.10 to 1.43) times if from grammar school	0.0010
Sixth form college or other school	25	0.67 (0.60 to 0.75) times if from sixth form college or other school	<0.0001
Local applicant	0.51 (0.33)	1.08 (0.97 to 1.21) times if all medical schools local (compared with no schools local)	0.153

UCAS=Universities and Colleges Admissions Service. 

^*Classification group of registrar general; group I is highest socioeconomic group, V is lowest. 

^†Comprehensive school is reference group. 

Candidates resitting A levels and mature applicants—

Mature applicants and those resitting their A levels are often treated differently during selection. Analysis of the 14 773 who were not mature students or resitting their A levels found only one change in significance level (α=0.05): candidates taking an A level in non-science subjects were less likely to receive an offer in the restricted analysis (P=0.040, odds ratio 0.89 (95% confidence interval 0.80 to 0.99)).

Applicants taking Scottish highers—

Applicants taking only highers might not be properly reflected in the overall analysis. Analysis of offers in the 1225 applicants of known ethnic origin (12.7% from ethnic minority groups; 156/1225) applying to Scottish medical schools found number and grade of highers were both significant predictors (P<0.0001), as were early application (P=0.0013), not putting four or less applications for medicine (P=0.0132), being white (P=0.0232), and not applying from a sixth form or other college (P=0.0004).

Ethnic origin—

Figure 1 confirms that non-white applicants are less likely to receive an offer across the whole range of A level achievement, with a qualitatively similar effect in applicants taking highers.

Figure 1.

Percentage of applicants receiving an offer in relation to educational achievement and ethnic group. Bars show 95% confidence intervals; they are not shown for mean grade in Scottish highers because sample sizes were comparatively small in some groups. Overall difference between white and non-white applicants for highers was significant (P=0.0232) after taking all other factors into account (see text)

Differences between ethnic groups—

Figure 2 shows the adjusted odds of disadvantage for each ethnic group. White applicants were more likely to receive an offer than applicants from any ethnic minority group, with only small differences between ethnic groups.

Figure 2.

Disadvantage of non-white applicants in each of the ethnic groups relative to white applicants (odds ratio of 1 indicated by vertical line)

Sex and ethnic origin—

The interaction between sex and ethnic origin was significant (P=0.049), women who were not white being less likely to receive an offer than expected.

Selection at individual medical schools—

Table 2 summarises the significance of each predictor at each medical school. Figure 3 shows the disadvantage of being from an ethnic minority group at each medical school, along with confidence intervals. As with any such league table, results must be interpreted with caution.¹⁴,¹⁵ The weighted correlation of effects in 1996 and 1997 was 0.428. Disadvantage in 1991⁵ also correlated (r=0.419) with a composite of disadvantage in 1996-7. Medical schools differ in their proportion of applicants from ethnic minority groups, although figure 4 shows no relation to the extent of disadvantage (r=0.070).

Table 2.

Summary of selection at individual medical schools in 1996 and 1997 combined^*

CXWMS=Charing Cross and Westminster Medical School, London; QMW=Queen Mary and Westfield College, London; HMS=Hospital Medical School, London; UMDS=United Medical and Dental Schools of Guy’s and St Thomas’s Hospitals, London. 

NA=not applicable as A levels difficult to interpret in Scottish context. 

^*Details of effect sizes, standard errors, and significance levels are available in my unpublished report and on the website of the Universities and Colleges Admissions Service (www.ucas.ac.uk). 

^†P<0.01 in each year, or equivalent (see text). 

^‡For A levels only, larger symbols show larger effect (arbitrarily split at about the median). 

Figure 3.

Disadvantage at medical schools in the United Kingdom of non-white applicants in 1996 and 1997 ordered by size of disadvantage in 1997. Horizontal line indicates odds ratio of 1 (no disadvantage). The estimate of disadvantage is corrected for all other variables in the analysis. Oblique dashed line in 1996 indicates disadvantage at each school in 1997. HMS=Hospital Medical School, London; CXWMS=Charing Cross and Westminster Medical School, London; QMW=Queen Mary and Westfield College, London; UMDS=United Medical and Dental Schools of Guy’s and St Thomas’s Hospitals, London

Figure 4.

Disadvantage of applications from ethnic minority groups expressed as odds ratio (y axis) in relation to proportion of applicants to a medical school who were from an ethnic minority group in 1996 and 1997

Discussion

Limitations of the study

I have summarised a great deal of information in this paper and have inevitably oversimplified some issues because of a tight timetable and constraints of space. However, this is the first time the influence of many interrelated factors has been described properly. Placing the data in the public domain means other researchers can explore the data in far more detail. The limitations of the study must be emphasised, particularly that it provides a historical description of selection in 1996 and 1997, not a prediction of future selection, which will certainly change, as will patterns of application. A concern must be with the factors not measured, although, as the nineteenth century neurologist Hughlings Jackson said: “Absence of evidence is not evidence of absence.” Results at GCSE and estimated A level grades might have clarified the interpretation, but their interrelations in a previous study⁵ suggest that they are unlikely fully to explain differences between white and non-white applicants. Assessments of personal attributes such as motivation, personality, and communicative ability would also have been helpful. Differences in personal attributes between groups might explain the disadvantages found in this study, but such judgments would need to be shown to be reliable, free of bias, and predictive of eventual professional behaviour.

Selection overall

As in the studies of 1981, 1986, and 1991,²,⁴,⁵ many variables predict a candidate receiving offers. Many also raise serious concern. The disadvantage of non-white and male applicants could be construed as evidence under the Race Relations Act 1976 and the Sex Discrimination Act 1975. That applicants taking non-science A levels might be disadvantaged, or that candidates taking AS levels (a sixth form qualification equivalent to half an A level) receive no advantage in selection are unlikely to broaden the intellectual base of medical students outside the modal three sciences (typically mathematics, chemistry, and biology). Reassuringly, general studies at A level does give an advantage to applicants, but selectors sometimes disparage what is seen as “not a real A level” (despite careful standard setting by examining boards). The advantage of applying early suggests a horse race in which some applicants start running before others, and the disadvantage of applicants making a non-medical (insurance) choice¹⁶ sits uncomfortably with medical schools’ policy of “no detriment”¹⁷ and the statement, “without prejudice to [applicants’ perceived] commitment to medicine” in the handbook of the Universities and Colleges Admissions Service. The disadvantage of applicants requesting deferred entry (gap year) seems at odds with the importance of “encourag[ing] 18-year olds to take a year off before coming to medical school, so they can reflect on what they really want.”¹⁸ Although the much claimed advantage of applicants from independent schools is (again) not confirmed, the clear disadvantage of applicants from further and higher education and sixth form colleges seems unlikely to broaden access to medical education.

Selection at individual schools

Inevitably, table 2 will be controversial. However, it reports only results that are significant on a strict criterion and statistical power differs in relation to the number of applicants and offers and the mix of applicants. Also, as figure 3 emphasises, effects are distributed on a continuum, which should not be naively divided into the good, the bad, and the ugly. However, the fact that schools show different effects on each measure is extremely important as it implies that intrinsic, unmeasured differences between groups of applicants are unlikely to explain the overall effects in table 1 and that the differences in table 2 therefore instead reflect structural differences in the processes of selection itself. The long term stability of the disadvantage of ethnic minority groups, with schools such as Bristol, Newcastle, and the Royal Free showing no evidence of disadvantage in 1991, and 1996-7, emphasises that good practice can occur. Correlation of the detailed mechanics of selection with the extent of disadvantage is therefore urgently needed. Space precludes detailed discussion of all differences in table 2, but questions about the ethics of selection can be raised through the touchstone of the advantage of local applicants. Such an advantage may be seen as justifiable if it reinforces a strong link between medical school and local community in the provision of medical services. It is less desirable, however, if it inadvertently disadvantages a non-local applicant who might not be adequately informed of such a policy—for example, in a clearly stated code of practice.¹⁹

Racism, discrimination, and disadvantage

These data may provoke claims that medical schools are racist (and perhaps also sexist). The Commission for Racial Equality defines racism in terms of attitudes, and other definitions consider it as ideology²⁰; in so far as this study does not assess the attitudes or ideologies of selectors, little further can be said. In terms of behavioural outcome, there is no doubt that applicants from ethnic minority groups are disadvantaged. Whether that disadvantage is due to discrimination cannot be shown from these data alone. However, in conjunction with previous studies and other literature, it seems reasonable to conclude that in some cases at some schools discrimination is occurring, wittingly or unwittingly. Direct discrimination seems unlikely (but not impossible²¹), although an aberrant minority of selectors may be subverting an otherwise fair system. Indirect discrimination is more possible, perhaps either through assessment of A level achievement indirectly from estimated grades and GCSE results or from assessing motivational and personality factors indirectly through achievements and experiences with different meanings in different cultural groups.

The important and correct decision by the Council of Heads of Medical Schools to release detailed information on the process of medical student selection will inevitably result in a debate about how selection should take place. This study may be the start of that process, but it is certainly not the end; other researchers will dig deeper into these data and will analyse other evidence of their own. My findings may be uncomfortable reading for some medical schools and some medical school applicants. Nevertheless, putting the information in the public domain was undoubtedly the correct decision since open scrutiny is the only way that democratic societies can reassure themselves that those acting on their behalf are doing so appropriately. Proper information is therefore the first step towards improving selection. This analysis also provides a precedent for selection into other courses, professions, and occupations, none of which has yet provided data of equivalent richness and detail. The Council of Heads of Medical Schools is committed to change. The release of future data on selection will allow monitoring of those changes.

Editorial by Abbasi and Education and debate p 1149