The impact of disability on performance in a high-stakes postgraduate surgical examination: a retrospective cohort study

Abstract

Objective

Despite rising numbers of doctors in the workforce with disabilities, little is known about the impact of disabilities on postgraduate performance. To ensure all groups are treated fairly in surgical training, it is essential to know whether any attainment differences exist in markers of surgical performance. To address this gap, we assessed the impact of disabilities on performance on the Intercollegiate Membership of the Royal College of Surgeons examination (MRCS).

Design

Retrospective cohort study.

Setting

Secondary care.

Participants

All UK MRCS candidates attempting Part A (n = 9600) and Part B (n = 4560) between 2007 and 2017 with linked disability data in the UK Medical Education Database (https://www.ukmed.ac.uk) were included.

Main outcome measures

Chi-square tests and correlation coefficients established univariate associations with MRCS performance, while multiple logistic regressions identified independent predictors of success.

Results

Though MRCS Part B pass rates were similar (p = 0.339), candidates with registered disabilities had significantly lower first-attempt Part A pass rates (46.3% vs. 59.8%, p < 0.001). Candidates with disabilities also performed less well in examinations taken throughout school and medical school, and after adjusting for prior academic performance and sociodemographic predictors of success, logistic regression found that candidates with disabilities were no less likely to pass MRCS than their peers (odds ratio 1.04, 95% confidence interval 0.66 to 1.62). No significant variation was found in MRCS performance between type of disability or degree of limitations caused by disability (p > 0.05).

Conclusion

Although candidates with registered disabilities performed less well in formal, written examinations, our data indicate that they are as likely to pass MRCS at first attempt as their peers who achieved similar grades at high school and medical school. In order to enable equity in career progression, further work is needed to investigate the causes of attainment differences in early career assessments.

Introduction

Diversity and inclusivity are high on social and political agendas worldwide. Various definitions of diversity and inclusivity exist but generally, while diversity acknowledges individual differences, such as disability, gender and ethnicity, inclusivity considers how we engage with these differences to establish equity or fairness. Equality and diversity have been prioritised on medical workforce agendas to ensure fairness within the profession, and for the medical workforce to reflect and represent the population for which they provide care. ¹

Some protected characteristics have received more attention in research than others. In the United Kingdom UK, ethnicity has been an area of particular focus since the landmark case of the British Association of Physicians of Indian Origin vs. the Royal College of General Practitioners and General Medical Council in 2014, which highlighted the potential legal ramifications of attainment differences in postgraduate examinations. ² Despite this, there remains a paucity of data on the performance of doctors with other protected characteristics, such as registered disabilities, in postgraduate assessments.

This is of interest given the number of doctors with disabilities (defined by the UK’s Equality Act, 2010 as a ‘physical or mental impairment that has a substantial and long-term adverse effect on his or her ability to carry out normal day-to-day activities’) ¹ is increasing. ³ Individuals with disabilities often have access to support and reasonable adjustments within school and medical school, but evidence indicates a loss of pastoral and learning support after graduation.^4–6 Additionally, many doctors describe facing discrimination within the workplace and worry that declaring their disability has an impact on career progression.^3,7,8 However, whether disability has an impact on postgraduate performance remains largely unknown. One study found no significant difference in pass rates for dyslexic candidates in the written Applied Knowledge Test of the Membership of the Royal College of General Practitioners examination after socio-demographic factors were taken into account. ⁹ Conversely, doctors with dyslexia had lower pass rates and required more attempts to pass the Membership of the Royal College of General Practitioners clinical examination. ¹⁰ Candidates with reasonable adjustments (referring to removing barriers or providing support, such as extra time in examinations) were also found to perform less well on the United States Medical Licencing Examination. ¹¹ Unfortunately, to the best of our knowledge, no studies have investigated whether specific learning difficulties (SpLD) and other disabilities have different impacts on performance at postgraduate examinations.

More generally, within medicine, there has been concern that multiple-choice format examinations may disadvantage candidates with specific learning disabilities, especially dyslexia. ¹² However, evidence from a small number of studies looking at multiple-choice performance in medical school does not support this hypothesis.^13,14 Similarly, there have been concerns regarding the impact of dyslexia on student’s verbal memory and processing speed during Objective Structured Clinical Examinations with one study demonstrating poorer performance in stations assessing examination skills and data interpretation. ¹⁵ Conversely, another study found no difference in the performance of medical students with specific learning disabilities in Objective Structured Clinical Examinations. ¹³ In short, the evidence from medical school assessments is conflicting, again justifying the need to examine the impact of disability on medical examination performance in more detail.

In the UK, examining bodies are required under the Equality Act of 2010 to eliminate discrimination against people with protected characteristics. ¹⁶ If discrimination within surgical training is to be eliminated, it is necessary to identify whether attainment differences exist in markers of surgical performance. Thus, we examined the relationship between registered disability and success at first attempt of the Membership of the Royal College of Surgeons (MRCS) examination while also accounting for other sociodemographic factors that are known to influence performance at the examination, including gender, ethnicity and graduate status.^9,17

In the absence of a gold-standard with which to compare medical examinations, one way of validating an examination is to determine the strength of associations between similar assessments. Performance in one test should predict the performance in a future similar test and scores between each assessment are therefore expected to show significant linear correlation (predictive validity). If group-level attainment differences are identified in a single examination only, the fairness of this examination would be questionable. Conversely, if identical group-level attainment differences are identified throughout all undergraduate and postgraduate assessments, then the equitability of training and education would be brought into question. Given that prior academic attainment is known to be the strongest predictor of later success in medical education,^18–20 we also assessed the performance of candidates with and without disabilities in other written examinations taken earlier in their careers to identify attainment gaps and to assess the fairness of the MRCS examination.

Methods

A retrospective cohort study was undertaken using data from the UK Medical Education Database (UKMED: https://www.ukmed.ac.uk/) and four Royal Colleges of Surgeons of the UK and Ireland (Edinburgh, Glasgow, England and Ireland). Anonymised data were extracted for all UK graduates that attempted the MRCS between 2007 and 2017. Candidate first attempt results were used as they have been shown to be the best predictor of future performance in post graduate examinations.^18,21

Membership of the Royal College of Surgeons

The Intercollegiate MRCS is an internationally renowned, high-stakes postgraduate assessment, attempted by 6000 trainees from across the world every year. The MRCS Part A includes two multiple-choice papers taking 5 h to complete in total if no further time is added for reasonable adjustments (also known as testing accommodations). The MRCS Part B is a 3.5-h Objective Structured Clinical Examination during which candidates rotate through 18 stations examining knowledge and skills outlined in the MRCS syllabus.

Registered disability status

Disability data were extracted from the UKMED and MRCS databases including whether a candidate with a registered disability declared themselves limited in their day-to-day activities as a result of their disability. Disability was coded as either; ‘SpLD’, which includes dyslexia, dyspraxia and attention deficit hyperactivity disorder (of which dyslexia is the most common), or ‘Other disability not including SpLD’. ‘Other disability not including SpLD’ includes any physical or mental disability, impairment or medical condition including serious visual or hearing impairment.

Markers of prior academic achievement

Individual-level linked performance data were extracted for A-Levels (high school exit examinations) as a marker of prior academic achievement and UK medical school admissions test results for the University Clinical Aptitude Test (UCAT), Biomedical Admissions Test (BMAT) or Graduate Medical School Admissions Test (GAMSAT). Total A-Level scores used in data analyses are the sum of all A-Level scores achieved, i.e. A = 10 (being the highest score achievable for each A-Level), B = 8, C = 6, D = 4, E = 2, U = 0 (being the lowest score for each A-Level). A small minority of candidates in the dataset (n = 30) undertook A-Levels after A* grades were implemented in 2010. These were subsequently excluded for cohort homogeneity. Note that Situational Judgement Test (SJT) performance was excluded from UCAT scores as was section 3 (written task) performance from BMAT scores as the introduction and marking of these sections changed over the course of the study period.

In the absence of a medical licensing examination in the UK (such as the Medical College Admission Test [MCAT]), the performance of medical school graduates is quantified by the award of an Educational Performance Measure (EPM) decile score based on their performance decile; ranging from 34 points for the 10th (lowest) decile to 43 points for students in the 1st (highest) decile. Scores from the Foundation Programme Situational Judgement Test (FP-SJT) (a test of the behaviours and attitudes expected of doctors) are also used for the selection of graduating doctors into the UK Foundation Training Programme.

Statistical analysis

All analyses were conducted using SPSS® v22.0 (IBM, Armonk, New York, USA). The following standardised and anonymised data were extracted from UKMED prior to analysis by the research team: total A-Level scores, UCAT, BMAT, GAMSAT, EPM decile scores and FP-SJT scores, MRCS Part A and B first attempt results, self-declared gender, ethnicity and graduate status. Only candidates with MRCS and disability data were included in the analysis, negating the need for imputation of missing data. All test scores extracted from the UKMED dataset and used in the analyses were standardised according to the applicant to medical school pool and converted to Z-scores to negate changes to pass marks between each examination diet within the study period. Part A and B MRCS performance was categorised as ‘pass’’ or ‘fail’ at first attempt for logistic regression modelling. Sociodemographic variables were dichotomised to avoid subcategories with small numbers. Self-declared ethnicity was coded as ‘white’ or ‘non-white’ as used in similar studies to maximise statistical power.^18,20 Gender was coded as ‘male’ or ‘female’ and graduate status was coded as ‘graduate’ if a candidate completed a degree level qualification prior to studying medicine or ‘non-graduate’.

Univariate analysis identified associations with first attempt MRCS pass/fail outcomes and the relationship between disability status and performance on measures of prior academic attainment. Logistic regression models were used to identify independent predictors of first attempt success at MRCS. Initial regression models included disability status and were adjusted for known socio demographic predictors of MRCS success (gender, ethnicity and graduate status).^9,17 A subsequent model included measures of prior academic performance from high school (A-Level scores) and medical school (EPM decile scores). Variables with p < 0.10 on univariate analysis were entered into the logistic regression model and potential predictors with p > 0.05 in the full model were then removed until only statistically significant predictors remained. Effect sizes are given as odds ratio (95% confidence interval). Correlation coefficients were calculated to examine the linear association between test scores and score relative to MRCS Part A (written paper) pass mark. Pearson correlation coefficients are presented, except for EPM decile and MRCS Part A score where Spearman’s correlation coefficient was used.

When handling, storing and analysing data, the highest standards of security, governance and confidentiality were maintained. In line with the Higher Education Statistics Agency data standards (www.hesa.ac.uk), all counts presented have been rounded to the nearest 5 to ensure person-level anonymity.

Results

Between 2007 and 2017 a total of 9730 UK graduates attempted MRCS Part A: 9600 had disability data and the other 130 candidates were excluded from further analysis. Of the candidates, 59% (n = 5655) passed at the first attempt. Of the candidates who attempted MRCS Part A, 7% (n = 635) had a registered disability, 65% (n = 415) of these candidates reported a specific learning disability and 9% (n = 60) declared that their disability limited their day-to-day activities. Of all candidates attempting Part A, 63% were male (n = 6090), 58% were white (n = 5560) and 85% non-graduates (n = 8200). Within the study period, a total of 4560 UK graduates attempted MRCS Part B, 71% (n = 3230) passed at first attempt. Similarly of all Part B candidates, 6% (n = 275) had a registered disability, 69% (n = 190) of these candidates had a specific learning disability and 7% (n = 20) declared that their disability limited their day-to-day activities. A total of 65% of Part B candidates were male (n = 2975), 60% were white (n = 2740) and 86% non-graduates (n = 3940).

Table 1 shows MRCS pass rates displayed by socio demographic group. Of all candidates without a registered disability, men, white candidates and non-graduates were each more likely to pass MRCS Part A and Part B at first attempt (p < 0.05). Table 2 shows the univariate analysis of MRCS success by socio demographic group for all candidates with a registered disability. Of all candidates with registered disabilities, men, white candidates and non-graduates were each significantly more successful at MRCS Part A first attempt (p < 0.05). White candidates with disabilities performed significantly better in MRCS Part B (p = 0.005), but there was no significant association between MRCS Part B success and gender or graduate status (p > 0.05). Table 3 shows the univariate analysis of MRCS success and candidate disability status. The MRCS Part A first attempt pass rate for the cohort with registered disabilities was found to be significantly lower than the cohort without disabilities (46.3% [95% confidence interval (CI) 42.4 to 50.2] vs. 59.8% [95% CI 58.8 to 60.8], respectively). However, there was no statistically significant difference found in Part A pass rates between ‘Specific learning disability’ and ‘Other disabilities’ (44.7% [95% CI 39.9 to 49.5] vs. 49.3% [95% CI 42.7 to 56.0] respectively) or between candidates with and without limitations in their day-to-day activities caused by disability (37.1% [95% CI 24.7 to 49.5] vs. 46.8% [95% CI 42.4 to 51.1], respectively). No statistically significant difference was found in Part B pass rates based on disability status.

Table 1.

Descriptive statistics of the study cohort and associated MRCS pass rates by sociodemographic group.

		MRCS Part A			MRCS Part B
	Registered disability	Total candidates	Pass rate (%)	95% confidence interval	Total candidates	Pass rate (%)	95% confidence interval
Male	No disability	5700	63.4	62.1–64.6	2790	72.3	70.7–74.0
	SpLD	280	47.8	41.9–53.8	135	66.7	58.6–74.7
	Other disability	115	53.0	43.8–62.3	50	70.0	56.8–83.2
Female	No disability	3265	53.6	51.9–55.3	1500	68.3	65.9–70.6
	SpLD	135	38.2	30.0–46.5	55	75.5	63.5–87.4
	Other disability	105	45.3	35.7–54.9	35	61.1	44.4–77.8
White	No disability	5110	64.6	63.3–65.9	2530	76.8	75.2–78.5
	SpLD	295	52.7	47.0–58.4	145	74.8	67.6–82.0
	Other disability	150	53.3	45.3–61.3	60	68.3	56.4–80.1
Non-white	No disability	3710	53.2	51.6–54.8	1700	62.8	60.5–65.1
	SpLD	115	23.7	15.8–31.6	45	51.2	35.6–66.7
	Other disability	70	39.7	27.8–51.6	25	60.9	39.3–82.5
Non-graduate	No disability	7700	60.9	59.8–62.0	3715	71.5	70.0–72.9
	SpLD	320	47.5	42.0–53.0	155	71.0	63.7–78.2
	Other disability	180	50.6	43.2–57.9	70	66.7	55.3–78.1
Graduate	No disability	1265	53.3	50.6–56.1	575	67.3	63.5–71.2
	SpLD	95	35.1	25.3–44.9	35	60.6	43.0–78.2
	Other disability	40	43.6	27.3–59.9	15	64.7	39.4–90.0

MRCS: Membership of the Royal College of Surgeons Examination; SpLD: specific learning difficulties.

Note: Graduate/non-graduate describes whether or not candidates had completed a degree level qualification prior to studying medicine.

Table 2.

Univariate analysis of MRCS success by socio-demographic group for all candidates with a registered disability.

Variable	MRCS Part A (n = 635)	MRCS Part B (n = 275)
Male	49.4% (195/395)	67.6% (125/185)
Female	41.3% (100/240)	69.7% (60/90)
p-value	0.048	0.727
White	52.9% (235/450)	72.8% (150/205)
Non-white	29.7% (55/180)	54.5% (35/65)
Missing	n = 5	n = 5
p-value	< 0.001	0.005
Non-graduate	48.6% (245/500)	69.6% (155/225)
Graduate	37.6% (50/135)	62.0% (30/50)
p-value	0.024	0.294

MRCS: Membership of the Royal College of Surgeons.

Note. All p-values are from Chi-square analysis.

Table 3.

Univariate analysis of MRCS success and registered disability status.

Variable	MRCS Part A (n = 9600)	MRCS Part B (n = 4560)
No registered disability	59.8% (5360/8965)	70.9% (3040/4285)
Any registered disability, impairment or medical  condition including specific learning difficulties	46.3% (295/635)	68.2% (190/275)
p-value	<0.001	0.339
Registered disability, excluding specific learning difficulties	49.3% (110/220)	66.3% (60/85)
Specific learning difficulties	44.7% (185/415)	69.1% (130/190)
p-value	0.278	0.676
Disability not limiting day-to-day activities	47.2% (265/560)	69.4% (170/240)
Disability limiting day-to-day activities	37.1% (25/60)	70.0% (15/20)
Missing	n = 15	n = 15
p-value	0.129	0.957

MRCS: Membership of the Royal College of Surgeons.

Note: All p-values are from chi-squared analysis.

Table 4 shows the univariate analysis of performance on prior academic assessments for those MRCS candidates with and without disabilities. Candidates with disabilities performed worse than their peers at A-Levels, BMAT and the EPM (p < 0.05). According to Cohen’s guidelines, there was a weak, but statistically significant correlation between MRCS Part A performance and A-Level (r = 0.15), UCAT (r = 0.25) and FP-SJT (r = 0.22) scores (p < 0.001). Moderate correlation was found between MRCS Part A performance and BMAT (r = 0.32) and GAMSAT (r = 0.35) scores and a large correlation was identified between MRCS Part A and EPM decile (r = 0.59) scores (p < 0.001).

Table 4.

Univariate analysis of performance on prior assessments and registered disability status.

		No registered disability		Registered disability
Assessment		Number of candidates	Mean score (standard deviation)	Number of candidates	Mean score (standard deviation)	p-value
A-Levels		1960	34.7 (9.4)	110	30.1 (9.2)	<0.001
UCAT		2615	2534.4 (226.0)	165	2494.0 (242.9)	0.255
BMAT		2125	10.3 (1.6)	115	9.9 (1.8)	0.004
GAMSAT		260	61.5 (6.6)	30	60.5 (6.1)	0.305
EPM decile		2460	38.7 (2.8)	175	38.0 (2.9)	0.006
FP-SJT		2420	39.5 (3.6)	230	39.2 (3.4)	0.480

Table 5 shows that men, white candidates and non-graduates were each significantly more likely to pass Part A at first attempt (p < 0.05) compared to their counterparts. When accounting for socio demographic factors, candidates with a registered disability were almost twice as likely to fail MRCS Part A compared to their peers (OR 0.55 [95% CI 0.46–0.64]). Subsequent models adjusting for both socio demographic factors and markers of prior academic attainment from high school (A-Levels) and medical school (EPM decile scores) first separately and then simultaneously revealed no statistically significant difference in the likelihood of success at MRCS Part A between candidates with and without disabilities (OR 1.04 [95% CI 0.66–1.62]).

Table 5.

Logistic regression model showing predictors of pass at first attempt at MRCS Part A for UK medical graduates before and after accounting for prior academic performance.

	Full cohort adjusted for socio demographic factors n = 9450	Adjusted for socio demographic factors and A-Levels n = 2070	Adjusted for socio demographic factors and EPM decile scores n = 2635	Adjusted for socio demographic factors, A-Levels and EPM decile scores n = 1715
Variable	Odds ratio (95% CI)	Odds ratio (95% CI)	Odds ratio (95% CI)	Odds ratio (95% CI)
Disability status Disability vs. no disability	0.55 (0.46–0.64)	0.70 (0.50–1.00)	0.87 (0.62–1.21)	1.04 (0.66–1.62)
Gender Males vs. females	1.54 (1.41–1.68)	1.68 (1.39–2.02)	2.40 (1.98–2.91)	2.32 (1.82–3.00)
Ethnicity White vs. non-white	1.73 (1.59–1.89)	1.45 (1.22–1.75)	1.10 (0.91–1.33)	1.08 (0.86–1.35)
Graduate on entry into medicine Non-graduates vs. graduates	1.48 (1.31–1.66)	1.10 (0.70–1.74)	1.82 (1.44–2.31)	2.16 (1.17–4.00)
A-Levels Total score	–	1.32 (1.19–1.46)	–	1.42 (1.25–1.61)
EPM Decile	–	–	1.58 (1.52–1.64)	1.60 (1.52–1.68)

MRCS: Membership of the Royal College of Surgeons; CI: confidence interval.

As a subgroup analysis, the 1715 individuals with complete data were examined. This subgroup was comparable to the full cohort in terms of distribution of sociodemographic factors (Table 1) and prevalence of declared disability. There was also no statistically significant difference found in Part A pass rates between ‘Specific learning disability’ and ‘Other disabilities’ (p = 0.106) or between candidates with and without limitations in their day-to-day activities caused by disability (p = 0.095). Mean A-Level score for doctors with disabilities in this cohort was 31.0 (SD 9.3) and the mean A-Level score for doctors without disabilities was 34.9 (SD 8.9). Mean EPM decile score for doctors with disabilities was 38.2 (SD 3.0) and the mean EPM decile score for doctors without disabilities was 38.7 (SD 2.9). Following adjustment for socio-demographic factors, candidates with a registered disability within this subgroup analysis were more likely to fail MRCS Part A compared to their peers (OR 0.66 [95% CI 0.45–0.97]), data not shown).

Discussion

MRCS candidates with disabilities had significantly lower MRCS Part A pass rates than their peers. They also performed less well in formal, written examinations on exit from high school (A-Levels) and medical school (EPM). After adjusting for prior academic attainment, no statistically significant difference was found in the likelihood of success at MRCS between candidates with and without disabilities. In short, while candidates with disabilities had lower Part A pass rates as a cohort, they were no less likely to pass Part A at first-attempt than candidates without disabilities who had similar grades at high school and medical school. Additionally, no statistically significant relationship was found between disability and success at MRCS Part B.

Significant correlations were found between both school and medical school assessment and MRCS Part A scores. Candidates with disabilities performed worse than their peers in all of the assessments included in this study. According to government statistics, this attainment gap exists in all levels of education from early school written examinations to postgraduate assessments. ²² This suggests that the MRCS is no less fair than other tests and that systemic inequalities underpin this engrained attainment gap. However, further investigation is imperative to exclude discrimination in education and training as a cause of poorer MRCS examination performance by students with registered disabilities.

There was no statistically significant difference in Part A performance between candidates with specific learning disabilities and those with other registered disabilities, with both cohorts performing worse than their peers without disabilities. This is important given that candidates with non-specific learning disabilities have been underrepresented in previous studies that have tended to focus on the majority; that is, candidates who report specific learning disabilities. We hope that our data prompt others to include this breadth of disabilities when looking for associations between attainment and disability in myriad markers of success. A better understanding of the attainment differences between groups of people reporting different disabilities could drive the appropriate redistribution of support and resources throughout school, university and postgraduate training.

Interestingly, no statistically significant difference was found in Part B pass rates between candidates with and without disabilities, suggesting that while candidates with disabilities may find written examinations more difficult, they subsequently perform as well as their peers in clinical examinations. This pattern merits further investigation such as comparing the impact of disability status on performance in other clinical undergraduate and postgraduate examinations.

Reasonable adjustments

Candidates with registered disabilities are often granted reasonable adjustments for examinations, typically including extra time in which to sit the examination. These adjustments have been found to equalise performance in undergraduate medical examinations.^3,13–15 The Intercollegiate Committee for Basic Surgical Examinations (ICBSE) assesses MRCS reasonable adjustment requirements on a case-by-case basis. However, reasonable adjustments for examinations are a highly contested topic in education and there is a need to balance fairness and equity with exam validity in order to maintain testing accuracy. ²³ Our data suggest that the current methods used for MRCS candidates are sufficient to enable performance unrestricted by disability to at least the same degree as methods used for the other examinations.

The need for inclusivity

These data further highlight some important issues within the workplace warranting further discussion by the medical community. Similar proportions of MRCS Part A and Part B cohorts (7% and 6%, respectively) had registered disabilities, but is this a true representation of all doctors? One study has estimated the prevalence of medical students with specific learning disabilities within a medical school as high as 10%. ²⁴ Is this difference in prevalence between medical school and surgical training due to fewer doctors with disabilities choosing to pursue a career in surgery or are those in surgery not disclosing their disabilities? We know that many do fail to disclose their disability and access the support that they are entitled to including reasonable adjustments for examinations.^1,25 This is unsurprising given ongoing reports of bullying, undermining and harassment of clinicians with disabilities within the workplace,^8,25 causing many to worry that declaring a disability may impact their careers. ⁷ The British Medical Association recently created ‘A charter for medical schools to prevent and address racial harassment’. ²⁶ This charter highlights the systemic discrimination of underrepresented groups and provides guidance on enacting culture change within the workplace. The creation of similar charters for other protected characteristics such as disability and the addition of equality, diversity and inclusivity training at undergraduate and postgraduate levels could increase awareness of differential attainment, discrimination and bullying and would help to drive positive culture change. Similarly, greater appreciation for the unique skills, insight and perspectives that doctors with disabilities can bring to the surgical team^1,12,27 and the elimination of the current stigmatising and bullying culture could enable greater disability disclosure and access to increased support in the postgraduate environment.

Differential attainment

Group-level attainment differences seen in this study between socio demographic groups are similar to those seen in previous studies ¹⁷ and are not unique to the MRCS.^18,28–30 White healthcare professionals with disabilities are known to feel more supported than their Black, Asian or Minority Ethnic counterparts ⁸ and lower pass rates were observed in non-white candidates with disabilities in this study in comparison to their peers. It is likely that the causes of differences in performance are likely to be complex and multifaceted, social and systemic. Further work is required to ensure that bias at the question level (item level bias or unfairness) is ruled out as a source of these group-level attainment gaps, using differential item functioning.

Many candidates with disabilities use computer technology as part of their adaptive techniques to optimise learning and practice.^3,6 It is possible that recent computerisation of the MRCS Part A examination will further benefit candidates with disabilities by aiding reading fluency and enabling the optimisation of font sizes, screen colour and exam accessibility. Future studies are planned to monitor the impact of these changes in order to ensure the provision of a fair and equitable examination for all candidates.

Strengths and weaknesses

Despite the strengths of large-scale retrospective cohort studies such as this for identifying group-level attainment differences, further work is needed to provide a more detailed look at individual-level performance. Disability and specific learning disabilities are broad terms and reported disabilities will range in type, severity and impact on activities of daily living and perhaps educational opportunities and experiences which cannot be represented in cohort level analyses. One of the strengths of the study was that no bias was apparent when the subgroup (with complete data) were examined. Self-reported ethnicity data were dichotomised into two discrete groups as has been used in other studies^18,20 to maximise statistical power for group comparisons, rather than this being an ethical or social decision. Larger cohort sizes would have enabled a more granular analysis. The study included only UK graduates to create a homogenous study cohort within one context with an overarching definition of disability and certain approaches supporting people with disability. While thousands of non-UK graduates attempt MRCS every year and further analysis of this population may ascertain the impact of disability status on the performance of the whole group, doing so is challenging. This is because countries have different legislation regarding disability which will impact the disclosure of disabilities and the support received. The small numbers of GAMSAT candidates with disabilities (n = 30) make it difficult to draw substantive conclusions in respect of this admissions test and later performance. In terms of future research, due to comparatively low numbers of candidates with registered disabilities and low associated power, large group differences are required before statistical significance is reached.

Conclusion

Candidates with registered disabilities perform less well in formal, written examinations throughout education and postgraduate training. Our data indicate that doctors with disabilities are as likely to pass MRCS A at first attempt as their peers who achieved similar grades at high school and medical school. No statistically significant relationship was found between disability and success at MRCS Part B. Further work is needed to investigate the causes of attainment differences in early career assessments, and to drive the appropriate distribution of support and resources to enable equity in career progression.

Declarations

Competing Interests

None.

Funding

This work was supported by Royal College of Surgeons of England, Royal College of Surgeons of Edinburgh, Royal College of Surgeons of Ireland and Royal College of Physicians and Surgeons of Glasgow.

Ethics approval

No formal ethical approval was required for this study of existing UKMED data. UKMED has received ethics exemption for projects using exclusively UKMED data from Queen Marys University of London Ethics of Research Committee on behalf of all UK medical schools (https://www.ukmed.ac.uk/documents/UKMED_research_projects_ethics_exemption.pdf). The Intercollegiate Committee for Basic Surgical Examinations (ICBSE) and its Internal Quality Assurance Subcommittee, which monitors MRCS standards, research and quality, approved this study.

Guarantor

RE is the study guarantor.

Contributorship

RE wrote the first draft of the manuscript and performed statistical analyses with AL’s supervision. All authors contributed to, edited and reviewed the manuscript. JC led the study proposal for access to UKMED data. All authors approved the final draft of the manuscript.

Provenance

Not commissioned; reviewed by Kieran Fernando and Julie Morris.

Data availability

The dataset used in this study was acquired from the UK Medical Education Database and is held in Safe Haven. Data access requests must be made to UKMED. Full information for applications can be found at https://www.ukmed.ac.uk.

ORCID iD

R Ellis https://orcid.org/0000-0001-7310-6904