Opening up the black box of a Gateway to Medicine programme: a realist evaluation

Abstract

Objectives

A Gateway to Medicine programme, developed in partnership between a further and higher education setting and implemented to increase the socioeconomic diversity of medicine, was examined to identify precisely what works within the programme and why.

Design

This study employed realist evaluation principles and was undertaken in three phases: document analysis and qualitative focus groups with widening access (WA) programme architects; focus groups and interviews with staff and students; generation of an idea of what works.

Setting

Participants were recruited from a further/higher education setting and were either enrolled or involved in the delivery of a Gateway to Medicine programme.

Participants

Twelve staff were interviewed either individually (n=3) or in one of three group interviews. Nine focus groups (ranging from 5 to 18 participants in each focus group) were carried out with Gateway students from three consecutive cohorts at 2–3 points in their Gateway programme year.

Results

Data were generated to determine what ‘works’ in the Gateway programme. Turning a realist lens on the data identified six inter-relating mechanisms which helped students see medicine as attainable and achievable and prepared them for the transition to medical school. These were academic confidence (M1); developing professional identity (M2); financial support/security (M3); supportive relationships with staff (M4) and peers (M5); and establishing a sense of belonging as a university student (M6).

Conclusions

By unpacking the ‘black box’ of a Gateway programme through realist evaluation, we have shown that such programmes are not solely about providing knowledge and skills but are rather much more complex in respect to how they work. Further work is needed to further test the mechanisms identified in our study in other contexts for theory development and to identify predictors of effectiveness in terms of students’ preparedness to transition.

Strengths and limitations of this study.

• Our findings elucidate a working programme theory of a Gateway to Medicine programme and may be useful in the generation and development of other access to medicine programme aims and curricula.

• Data collection included a longitudinal aspect, which ensured changes occurring over the course of the Gateway programme’s evolution were captured in the interviews.

• The use of realist evaluation and identification of six key contexts, mechanisms and outcomes provides an evidence base as to how Gateway programmes work in respect of providing the necessary context and mechanisms, related to outcomes, for students to overcome barriers to entry to medicine.

• Our study was carried out in one context, which may limit its conceptual generalisability, or transferability, however, Gateway programmes are increasingly commonplace, and we do not think the aims and content of the programme we studied are unique.

• As with any voluntary study, there would have been an element of participant self-selection.

Introduction

Increasing the diversity of medical students is a global challenge. Certain groups are under-represented in the medical workforce and medical student body.^{1 2} This under-representation is associated with social and demographic factors such as ethnicity, minority group membership or low-income, which are in turn related to a multitude of historical and social systemic factors which vary by country.^3–6

Governments approach widening participation in education and medicine through macro-level policies which are then enacted by universities and medical schools in the form of widening access (WA) initiatives.^7–9 WA to medicine encompasses a range of interventions from outreach schemes (eg, activities undertaken by universities or professional medical bodies to engage students from WA/under-represented backgrounds), to the particular use of selection tools or contextual data, to ‘Gateway’ programmes.^{10 11} Gateway programmes are typically 1 year transitional courses, either stand alone or linked to a standard medical programme.¹² They are designed to attract students with educational and social disadvantage and to support these students to succeed.

Despite considerable investment by governments and associated agencies in Gateway programmes (eg,^13–15), to date evidence of their effectiveness in terms of educational outcomes has been limited to small, mostly descriptive, single site studies.^16–18 However, a recent multi-site study identified that Gateway to Medicine courses do work, with many students thriving academically and with the majority graduating as doctors.^{19 20}

This is encouraging, however, we now need to understand how these programmes work and what contributes to their effectiveness. Research to date has looked only at specific aspects of supporting WA students into medicine, such as the role of mentoring and the personal qualities needed to be a mentor (eg,^21–23). This is useful but insufficient: WA Gateway programmes typically contain many potential, inter-relating components or mechanisms of action, of which mentorship (for example) is only one. They tend to be an academic year in length, include many different activities (eg, tutorials, laboratory classes, skills practice, feedback, examination practice, admission preparation), diverse curricula (formal, informal and hidden),²⁴ different target groups, different cocurricular activities and subtly different aims and objectives. In other words, WA Gateway programmes can be considered complex educational interventions with multiple, interacting components or potential mechanisms of action.²⁵ Detangling what these mechanisms of action are and how they contribute to effectiveness is critical in ensuring that efforts are evidence-based, and resources are well spent.²⁶ Further, and although the quality of WA research has improved in recent years,^27–29 there is still a need for more robust and theory-driven approaches to identify conceptual generalisability.^{8 30 31}

In summary, it is essential that WA programmes are evaluated to understand how and why they work, and for whom. To address this gap in the literature, our aim was to examine how the various interconnecting parts of a WA intervention can be delineated to identify precisely what works and how. To do this, we drew on realist evaluation,³² a theory-driven approach to evaluation of social programmes, developed to understand how interventions work in the social context within which they are implemented.^{32 33} Realist evaluation is useful in terms of understanding why the same intervention may produce different outcomes when implemented in different settings: what works, for whom, how and under what circumstances.^{32 34} Moore³⁵ and Wong et al³⁶ proposed that realist evaluation lends itself particularly well to evaluating educational interventions in the complex field of medical education, developing new knowledge and insights by moving beyond simplistic evaluations of ‘Did it work?’ to ‘Why did it work (or not)?’. We anticipate that this study will add knowledge in respect to making recommendations on tailoring, implementing and designing strategies to widen access to medicine. Our research will address the questions: what are the various interconnecting parts of a WA intervention which work?; how do the various interconnecting parts of a WA intervention work together?

Methods

Overview of context

The Gateway programme under study commenced in 2017 in response to a Scottish Government call for initiatives to support those from less traditional/privileged backgrounds to pursue a career in medicine. The programme is delivered in partnership with a further education (FE) college and was developed by programme architects at both institutions. Gateway students spend the first session/semester attending classes at the partner FE institution and the second at the host university. Through both semesters, students are provided with university campus accommodation and undertake a programme of cocurricular activities with a dedicated tutor. These cocurricular activities are designed to support students in pursuing a career in medicine, from preparing them for pre-entry assessments, to developing a professional identity, to adapting to living away from home. Bursaries are available and students are also offered the opportunity to undertake bank staff work as a healthcare support worker (HSW) within the local National Health Service board to gain healthcare experience and income. Gateway students are guaranteed entry into the first year of medicine at the host university if they fulfil predetermined academic criteria and perform satisfactorily on the other medical school admissions processes (UCAT (https://www.ucat.ac.uk/ucat/) and a Multiple Mini Interview (MMI)).

Methodology

This study employed realist evaluation principles³² positioned within a critical realist ontological perspective.³⁷ Realist evaluation describes what mechanisms cause which outcome (intended or unintended consequences) and in which context (social and cultural conditions external to the interventions).³² This is typically presented as context (C), mechanism (M) and outcome (O) configurations. These configurations foster an understanding of what works for whom in what circumstances.

The aim of undertaking realist evaluation is to generate an initial programme theory whereby key CMOs configurations are identified, which are then tested and refined.^{32 38} Programme theory may be derived deductively, inductively and/or formulated from stakeholders’ mental models.³⁹

Data collection

Driven by a realist evaluation approach, this study proceeded in three key phases (table 1).

Table 1.

The realist evaluation process and data sources

Phase	Sources of data and activity
Phase 1 Identifying a programme theory	Document analysis of programme specific material Focus groups with key stakeholders and architects involved in the programme design and implementation
Phase 2 Testing a programme theory	Focus groups with three consecutive student cohorts enrolled on the Gateway programme
Phase 3 Refining the programme theory	Analysis of material and refinement of CMOs

CMOs, contexts, mechanisms and outcomes.

Phase 1 was an analysis of the Gateway programme promotional materials from the host university website and the Scottish Government website. The rationale behind the selection of these specific documents was that they had either been used to inform the Gateway programme development and to conceptualise. The programme theory was further refined via group and/or individual interviews with key programme architects (those involved in developing and implementing the programme) where programme aims and context were explored. Twelve staff were interviewed either individually (n=3) or in one of three group interviews. Staff were interviewed in the early stages of programme implementation. Staff interviews included questions on how the vision for the programme, how this had been operationalised, and challenges anticipated or experienced. This process aimed to initially identify the key CMOs involved in the Gateway course, to develop an initial programme theory (ie, how the Gateway programme was envisaged to work particularly in relation to understanding the context).

In phase 2, to test the initial programme theory, we gathered qualitative data from those who had experienced the programme using a purposive sampling approach. This included students on the programme and staff delivering teaching. At the time of data collection, three cohorts, totalling 65 students, had entered the programme. Nine focus groups (ranging from 5 to 18 participants in each focus group) were carried out with Gateway students from three consecutive cohorts at 2–3 points in their Gateway programme year. The longitudinal nature of data collection meant some students were interviewed at multiple time points whereas others may have only contributed to one focus group. Students were asked how they were settling into the Gateway programme, what worked well, what did not work so well, any challenges experienced, particular highlights and recommendations for improvement. Two members of staff were also interviewed again (they had previously been interviewed as stakeholders and were also involved in programme delivery), in the third year of the programme, and asked the same questions

After obtaining ethical permission, staff and students were invited by JC, KA or KGS to participate in the study via email. Staff focus groups and interviews were conducted by JC, KA or KGS, student interviews and focus groups by KA or KGS, either in person or virtually (during the 2020 COVID-19 pandemic). Each interview was recorded with written consent. Verbatim transcription was undertaken by a third party. Individual identifiers were removed at the analysis stage.

In phase 3, we used the interview data to test the CMOs developed in phase 1, by comparing what happened vs what was envisaged. We then refined the programme theory by considering if the data supported the initial programme theory.

Data management and analysis

Document data were coded by KGS to identify an initial programme theory and findings reviewed by JC. Preliminary analysis of interview and focus group data, both in relation to the initial programme theory and programme theory, was iterative and carried out by KA and KGS. These data were then analysed formally by KGS using NVivo V.12 Pro. In accordance with the preferred retroductive approach to realist research⁴⁰ both inductive (codes developed from the data) and deductive approaches (codes developed from the research question) were used.⁴¹ The inductive and deductive approaches were therein advanced to consider a more retroductive approach whereby data were interrogated to identify causal pathways. Inductive codes were generated by coding all statements initially in terms of content and were further refined by grouping into overarching themes by KA and KGS. The deductive code template was developed by JC and KGS as per the three core concepts of realist evaluation (CMO) and related only to content which was pertinent to what ‘worked’ within the Gateway programme. Initially, themes relating to outcomes were developed and then mapped to mechanisms with the context unto which they ‘worked’ identified. CMO configurations were then worked up into causal pathways using the aforementioned retroductive approach and compared with the initial programme theory. CMO configurations were reviewed and any disagreements were resolved via discussion.

Patient and public involvement

Patients and the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Results

The initial programme theory (table 2) was generated via document analysis of Gateway material along with interview data derived from the programme architects (phase 1). It was drafted in relation to predicted CMOs. The initial programme theory was subsequently refined into a programme theory (phase 2) on analysis of interview data. The interview data analysis generated a programme theory whereby six inter-relating mechanisms, which helped students see medicine as attainable and achievable and prepared them for the transition to medical school, were identified. These were academic confidence (M1); developing professional identity (M2); financial support/security (M3); supportive relationships with staff (M4) and peers (M5); and establishing a sense of belonging as a university student (M6).

Table 2.

Initial working programme theory

Initial working programme theory
The success of the Gateway programme will be dependent on students being adequately supported across academic, professional, financial and personal domains. Implementation of relevant support systems will enable students to successfully transition across educational environments.
Predicted context	Predicted mechanisms	Predicted outcomes
Students undertake a curriculum which is relevant to medicine (C1)	Develop confidence in their abilities (M1)	Successfully transition through educational environments (O1) View an application to medicine is attainable and achievable (O2)
Students are provided with opportunity to undertake relevant healthcare work experience (C2)	Develop confidence in their abilities (M1) Develop professional identity (M2) Develop financial security (M3)	View an application to medicine is attainable and achievable (O2)
Students are supported over the course of the programme (C3)	Develop confidence in their abilities (M1)	Successfully transition through educational environments (O1)

Phase 1: identifying a programme theory

Web data detailed the envisaged outcomes of the Gateway Programme—promoting equality in access to medicine and ensuring students are prepared to study medicine: ‘… to provide a novel, accessible and supportive route into medicine’ for students from a WA background which … will allow them to reach their full potential and become doctors’ (University of Aberdeen webpage text 2020). These aims were principally based on university, college and government drivers for WA.

Programme architects (n=5) saw broad entry criteria as key, as these would overcome academic hurdles, which are the primary reason for rejection to medical school, but which may not accurately identify potential for those disadvantaged by systemic, social or demographic factors. They wanted the entry criteria to challenge preconceived ideas of deprivation (eg, that deprivation may not merely be experienced by those living in the most deprived postcode areas). Their vision of the programme centred on the appropriateness and relevance of the curriculum, promoting equal opportunities for students, ensuring the right students were admitted to the programme and that students’ transitions to, and through, the programme were adequately managed. This last point included providing the opportunity for students to train and work with the local healthcare provider, so students could gain relevant work experience in healthcare and financially support themselves. Table 2 outlines the initial programme theory in terms of predicted context, mechanisms and outcomes.

Phase 2: testing the programme theory

The programme theory was tested by collecting and analysing data from staff and student focus group and interviews. Data from the three cohorts of students were merged to protect anonymity.

Admissions and progressions

In addition to fulfilling the requisite academic entry criteria, applicants were required to be resident in a postcode within one of the most deprived areas in Scotland (as determined by the Scottish Index of Multiple Deprivation: n=30 students),⁴² be currently in care or a care leaver (n=3) or meet three of the following: attended a school identified as under-performing by the national Scottish Funding Council (n=47)) (University of Aberdeen 2019); first in family to enter higher education (n=34); being a registered carer (n=4); eligible for free school meals (n=28); resident in a remote or rural location (as determined by the Urban Rural Classification of the Scottish Government (n=14); estranged from family (n=3); eligible to receive an Education Maintenance Award (n=35); evidence of hardship from their High School Head Teacher (n=15) and/or English as a second language (n=11). While there has been variation in the numbers fulfilling specific eligibility criteria across each of the cohorts, the diversity within each has been maintained.

Student and staff perspectives

Student and staff findings were grouped into six distinct themes (tables 3 and 4) prior to grouping into CMO configurations. These related to what ‘works’ in the Gateway programme: establishing staff support, developing peer support networks, identity formation, experiencing a tailored and relevant curriculum, establishing geographical familiarity and provision of financial support. Findings are presented below in relation to each category and relevant quotes presented in table 4. Some of these relate directly to the CMOs in the Initial Working Theory (table 2) and these are referenced (eg, C1). Other findings were developed in this testing phase and are referenced as such (not in initial working theory (NIIWT)).

Table 3.

Student and staff perspectives in relation to themes

Group	Example
Addressing practical issues Impact on experiencing smooth educational transitions and establishing financial security
Student	Familiarity with the city. Provision of bursaries during Gateway.
Student and staff	Familiarity with the university environment. Opportunity to secure relevant paid work experience in Gateway and beyond.
Establishing staff support Impact on positive transitions through educational environments and supporting progression
Student	Staff supporting development of peer networks.
Student and staff	Staff supporting transitions.
Developing peer support networks Impact on positive transitions through educational environments, feeling and developing confidence
Student	Establishing early peer support networks prior to commencing Gateway. Opportunities to integrate with wider university student body.
Student and staff	Developing peer support networks within the Gateway programme.
Development of professionalism and identity Establishing self-belief and sense of belonging
Student and staff	Developing identity as a professional.
	Developing identity as a university student.
Staff	Developing identity as a young adult.
Experience of relevant curriculum Impact on enhancing confidence and preparedness, and experience of smooth educational transitions
Student and staff	Scaffolded learning throughout Gateway to support transitions.
	Provision of practical sessions which enable students to develop knowledge and skills to help with an application to medicine.
Staff	Using Gateway as an opportunity to test suitability for a career in medicine.
	Refinement of Gateway curriculum based on student feedback.

Table 4.

Detailed staff and student findings with exemplar quotes

Establishing staff support in the Gateway programme: experiencing positive transitions through educational environments and supporting progression
Staff supporting development of peer networks	‘Yeah, so we all got, like, talking and we got coordinated who’d bring what. And it, it was nice. It alleviated a lot of, like, stress and nerves having kind of, like, semi met you all before moving in. But that was good’ (Female student, cohort 1)	Student
Staff supporting academic transitions	‘They, like, arrange sessions for you to go in and just get, like … You can have, you know, little small groups or one to one. And clear up anything you’re unsure of, like, straight away. And they’re always really, always really wanting to do that. They’re always encouraging of that. But yeah, they’re always wanting to help’ (Female student, cohort 1) ‘Like I say, the tutor’s been doing different things to what I expected, but the tutor is seeing them regularly and doing the pastoral side really thoroughly; having one to one meetings with them all rather than seeing them as a group, because people tend not to tell you stuff—well, some people tend not to tell you stuff unless they’re on their own. I think that’s been very useful’ (Staff interviewee 1)	Student and staff
Developing peer support networks: experiencing positive transitions through educational environments, feeling and developing confidence
Developing peer support networks out with the Gateway programme	‘Yeah, if you were part of a society that [sic] you would’ve met people, so then you’d meet them in the lecture as well’ (Male student, cohort 2)	Student
Developing peer support networks within the Gateway programme	‘I felt comfortable because we had that college experience, so you’ve got someone to be within [sic] the university, so it didn’t really bother me that much’ (Female student, cohort 2) ‘They have established a good cohesion among them, so they help each other with extra-curricular activity but also with the teaching, you know the study commitment they have. I think this could work in term … You know when they will be in a big class of 120 or 200 students, they’re able to recognise each other and if something came up as a difficulty or whatever they have already a well-established network of people because as you know coming from this kind of different background’ (Staff interviewee 2)	Student and staff
Development of professionalism and identity (young adult, university and medical student): establishing self-belief and sense of belonging
Developing identity as a university student	‘Yeah, we never get told that we’re uni students as well, I don’t know if you guys … I pick up on it quite a lot, it’s one of the things that really annoys me, they don’t … they never treat … well yeah, they don’t treat us like uni students but they don’t call us it either, so they’ll be like, they’ll say, “Oh, you’re not at a secondary school, you’re at NESCOL”, they’re never say, “You guys are uni students”’ (Female student, cohort 3) ‘So I think that’s a good thing as well because again it was people from the university that they were introduced to that they would maybe not have come across otherwise at that stage so it was those links again because of the way we’ve timetabled the college one semester, university the second semester, kind of trying to make sure that they didn’t feel to divorced from the others. From that first few months’ (Staff interviewee 3)	Student and staff
Developing identity as a professional	‘It’s been amazing, for me, like I’ve came from my Highers, and I’m still 17, and I’ve got more medical experience than most medical students will have, going into that, and as part of the university and maturity’ (Male student, cohort 1) ‘That (Healthcare Support Worker role) is such a useful thing and it worked pretty well the first year. Lots of the students were attracted by that to start off with and it gives them financial support and it is really good at giving them an insight into what they’re going to be doing, I think it give them … they feel they’re better off than the other students when they start medicine because they’ve already been in that sort of clinical environment and talking to patients. I think most of them are very comfortable at that’ (Staff interviewee 1)	Student and staff
Developing identity as a medical student	‘I want to go into the exam with the mentality to prove to myself that I am meant to do medicine, that I’m good enough to do medicine, that I should be allowed into the course’ (Female student, cohort 3)	Student
Developing identity as a young adult	‘It was a success in term to see them behave as an adult, you know? Completely an adult’ (Staff interviewee 2)	Staff
Experience of a tailored and relevant Gateway curriculum: enhancing confidence and preparedness, and experience of smooth educational transitions
Scaffolded learning throughout Gateway to support transitions	‘Yes, more so than coming out of school because we’ve experienced lectures, taking notes in lectures, the pace of lectures you know will be …Well, it’s a lot more faster, it’s a lot faster than school, so that’s provided a lot more insight into being a medical student really’ (Male student, cohort 1) ‘It is useful in being a small step from school and they are taught of manner that they’re taught at school, but then they’ve had to cope with, most of them, with moving because most of them don’t live at home so they’ve had to do the bit about eating and washing and learning all of that stuff as well. I think it is good that that’s a small step, but I think that they feel fairly quickly that they haven’t come to university in that first half session. Well, they haven’t really. Fairly soon they don’t feel that was a big enough step, but I think I prefer that than the other way round; if they were thrown into university classes of 200 or something in the first year and they don’t know who their little group are and they haven’t got someone who’s monitoring whether they’re there at each class session. I still think that’s a useful steppingstone in reducing the size of those transitions, I think is still useful and yeah, at the moment I’m quite happy that they’re ready to leave’ (Staff interviewee 1)	Student and staff
Provision of cocurricular activities and practical sessions which enable students to develop knowledge and skills to help them in making an application to medicine	‘We, we’d done that much like presentation-wise, speaking, you know, out-wise, um and also we were doing all the mock stuff. We’ve done like two mock MMIs … I’ve done that much going up to MMI that I knew exactly what was going to happen, and what it was going to be like’ (Male student, cohort 1) ‘I think there is that bit about expectations and being in a position to perform well in the interview and in the UCAT and there’s lots of stuff that they don’t know and there’s no reason why they should be expected to know it, they probably haven’t had the background support at home and at school to help them with that’ (Staff interviewee 1)	Student and staff
Using Gateway as an opportunity to test suitability for a career in medicine	‘I was surprised when we asked students what they thought of the course at the end of the first year, a few of them said that they liked it as an opportunity to find out if they wanted to do medicine, so they weren’t taking it as starting the G2M course as being the commitment that I’m going to do medicine’ (Staff interviewee 1)	Staff
Refinement of Gateway curriculum based on student feedback	‘So for this reason, after receiving a lot of feedback from the students, after discussing a lot with [FE College] people which are the best options to offer them, we have identified a couple of different courses that can be introduced into the actual curriculum to replace the previous one. Now we have a comprehensive curriculum where NESCOL is able to offer us the kind of courses that we think are useful to fill any gaps’ (Staff interviewee 2)	Staff
Geographical familiarity: experiencing smooth educational transitions
Familiarity with the university environment	‘Especially, even at [the halls], even walking around, getting around, where to go, things like that. The lecturers, who they are, the kind of teaching style, actually learning at uni, I feel like I’ve done pre-med’ (Male student, cohort 1) ‘Yes, we had the opportunity to create tailored activities with them just because there is this transition times from medical school to the university. So it’s good to then be actually in a way familiar with the university and the kind of information they need because probably they won’t be just in a building like here but they will be you know in the whole campus in Aberdeen, many different locations. They have to know how to know where to go, how to ask’ (Staff interviewee 2)	Student and staff
Familiarity with the city	‘… the location of [College], because it’s closer to town, we got to know that town really well’ (Male student, cohort 3)	Student
Provision of financial support: establishing financial security during studies
Opportunity to secure relevant paid work experience in Gateway and beyond	‘… the paid work experience and the small-group learning as well. It just ticks a lot more boxes than your general undergraduate course’ (Female student, cohort 1) ‘That (Healthcare Support Worker) is such a useful thing and it worked pretty well the first year. Lots of the students were attracted by that to start off with and it gives them financial support’ (Staff interviewee 1)	Student and staff
Provision of bursaries during Gateway	‘… the bursary and stuff like that makes it more manageable to afford to, like, study. And so that kind of pulls you towards applying’ (Male student, cohort 1)	Student

Addressing practical issues

Students and staff highlighted the importance of addressing practical issues, such as ensuring familiarity with the city, campus, university processes and staff (NIIWT). This seemed to enhance students’ confidence in relation to transitioning through the course (M1) and contributed to their sense of preparedness to study medicine (NIIWT). Another practicality was work experience (C2). Having the opportunity to undertake work as an HSW was perceived, by both students and staff, to be a significant pull factor of the Gateway programme in terms of insight into healthcare and income generation (M3). The provision of bursaries was also a key pull factor (M3).

Establishing staff support

Staff support was critical to students’ development (C3). Students highlighted the importance of staff facilitating, via social media, development of peer support networks prior to arrival (NIIWT). Students and staff reported the benefits of staff supporting transitions away from home and from one educational environment to another (M4). This was enhanced via the provision of one-to-one support which gave students a safe space to report any relevant personal circumstances (NIIWT).

Developing peer support networks

Students and staff reported that the opportunity to develop a relatively small peer support network (M5) with students from a similar background was beneficial for students in terms of developing confidence and providing support (C3). Students also highlighted that being able to integrate within the wider university student body (M5), for example, via membership of university societies, was helpful when transitioning through educational environments.

Development of professionalism and identity

Students reported that it was important that they were referred to as university students throughout the Gateway year (C4), and to get to know senior university staff (to assist in assimilation into university (C3)). Staff and students identified the programme, and its interrelated components (eg, opportunity to work as an HSW (C2)) as important in developing the professional identity of a medical student (M2) and increasing their self-confidence (M1). Progression on the course proved that they had the capability to study medicine (C5). Staff also highlighted that the programme content (C1) helped students to develop their identity as a young adult (M1).

Experience of relevant curriculum

Both staff and students saw the Gateway curriculum as a steppingstone to university (C1)—increasing autonomy for learning to develop preparedness for transitions (M1). In addition, both staff and students believed that the practical sessions (eg, MMI and UCAT preparations) helped students develop relevant knowledge and skills which would assist them in making an application in medicine and to overcome barriers to access to medicine in terms of their background (C1). There was a sense of co-construction of the programme as it moved forward, with student feedback shaping programme refinement (eg, removal of some modules) (NIIWT).

Phase 3: testing the programme theory and refined CMOs

Six mechanisms were identified in phase 2, three of which were reflective of our initial working programme theory (table 1). The six mechanisms which contributed to the success of the Gateway programme and students experiencing successful transitions included: development of confidence in abilities (M1), development of professional identity (M2), establishment of financial security (M3), development of positive relationships with staff (M4) and peers (M5), and establishing a sense of belonging (M6). In phase 3, we refined the initial working programme theory following testing (figure 1, table 5).

Figure 1.

Refined programme theory.

Table 5.

Refined programme theory

Refined programme theory
The success of the Gateway programme was dependent on students being adequately supported across academic, professional, financial and personal domains. Implementation of relevant academic, personal and financial support systems enabled students to successfully transition across educational environments and away from home.
Predicted context	Predicted mechanisms	Predicted outcomes
Students undertake a curriculum which is relevant to medicine (C1)	Develop confidence in their abilities (M1)	Successfully transition through educational environments (O1) View an application to medicine is attainable and achievable (O2) Develop preparedness to transition to medical school (O3)
Students are provided with opportunity to undertake relevant healthcare work experience (C2)	Develop confidence in their abilities (M1) Develop professional identity (M2) Develop financial security (M3)	View an application to medicine is attainable and achievable (O2) Develop preparedness to transition to medical school (O3)
Students are supported over the course of the programme (C3)	Establishing supportive relationships with staff (M4) Establishing supportive relationships with peers (M5) Establishing sense of belonging (M6)	Successfully transition through educational environments (O1) Develop preparedness to transition to medical school (O3) Successful transition away from home (O4)
Students are identified as university students (C4)	Develop professional identity (M2)	View an application to medicine is attainable and achievable (O2)
Students feel that they are challenged by the curriculum (C5)	Develop professional identity (M2)	View an application to medicine is attainable and achievable (O2)
Students receive financial support over the course of the programme (C6)	Develop financial security (M3)	View an application to medicine is attainable and achievable (O2) Develop preparedness to transition to medical school (O3)
Students develop familiarity with the university environment (C7)	Establishing sense of belonging (M6)	Successfully transition through educational environments (O1) View an application to medicine is attainable and achievable (O2) Develop preparedness to transition to medical school (O3)
Students develop familiarity with the city environment (C8)	Establishing sense of belonging (M6)	Successfully transition through educational environments (O1) View an application to medicine is attainable and achievable (O2) Develop preparedness to transition to medical school (O3)

Students’ development of confidence in their abilities related to both interpersonal skills and academic ability (M1). These were facilitated by undertaking a curriculum which they perceived as relevant to medicine (C1), being provided with an opportunity to undertake paid healthcare experience (C2) and feeling supported over the course of the programme (C3). The various contexts were critical in enabling students to develop new ways of learning, knowledge specific to medicine and an understanding of the non-technical skills required to be a doctor. These key contexts and mechanisms enabled students to successfully transition through educational environments (O1), view an application to medicine as both attainable and achievable (O2) and develop a sense of preparedness to transition to medical school should they be accepted (O3).

The development of professional identity (M2) was enabled by providing students with the opportunity to undertake a placement, and paid work if they wished (C2). Being identified as a university student from the outset (C4), while attending the partnership college, was important in developing a professional identity as a doctor, as was feeling adequately challenged (C5). Having the opportunity to develop a professional identity contributed to students viewing an application to medicine being within their reach (O2).

Establishing financial security (M3) was facilitated by being provided with the opportunity to undertake paid work as an HSW (C2) and via provision of bursaries to support living costs over the course of the Gateway year (C6). Feelings of financial security were important to students when considering their suitability to make an application to medicine (O2) since this alleviated some of the stress associated with moving away from home and ensured they felt prepared to transition to medical school should they be successful (O3).

The development of positive relationships with academic staff (M4) and peers (M5) were critical factors in ensuring that students were supported through their studies (C3) and able to successfully transition through educational environments (O1) and in ensuring they felt prepared to make a transition both away from home (O4) and to medical school should they be successful in their application (O3). Feelings of support within the programme were enhanced by staff providing one-to-one support to students and facilitating the formation of early peer support networks from the outset. Students felt that developing strong peer support both within and out with the programme, via having the opportunity to join university societies, were important factors in developing a support network.

Establishing a sense of belonging (M6) was important to students in the context of medicine itself, the city and university and in facilitating students to successfully transition through educational environments (O1), view an application to medicine as both attainable and achievable (O2) and develop a sense of preparedness to transition to medical school should they be accepted (O3). This sense of belonging was facilitated by allowing students to develop familiarity with the university (C7) and city (C8) environments over the course of their Gateway year, in addition to ensuring students were well supported over the course of the programme (C3).

Discussion

To the best of our knowledge, this is the first study to use a realist approach³² to evaluate a Gateway to Medicine programme in relation to what works, for whom and in what circumstances. We identified six inter-relating mechanisms inherent in the Gateway programme under study, each of which facilitated students towards their goal of studying medicine: developing confidence and professional identity; financial security; positive relationships with staff and peers; and establishing a sense of belonging. Identification of these key CMOs starts to unpack the ‘black box’ of Gateway to Medicine programmes. Our findings elucidate a working programme theory of a Gateway to Medicine programme and may be useful in the generation and development of other access to medicine programme aims and curricula. 

Previous studies have indicated that students from WA background lack both the academic qualifications^{42 43} and social support and links which can facilitate entry into medicine.^{11 44–46} The Gateway programme addressed these ‘deficits’ by helping students develop confidence in their academic ability, and increasing their understanding of medicine and the medical school application process. The second mechanism was linked to professional identity development and a sense of belonging as a university student. The literature on strategies to promote a sense of belonging in under-represented groups highlights the importance of ensuring that academic environments are inclusive.⁴⁷ These were enabled by the work placements, being a university student and feeling adequately challenged by the Gateway curriculum. Establishing financial security was identified as mechanism three. This was facilitated by the provision of (paid) healthcare work experience and bursaries to support living costs, which addressed the significant practical barrier of being able to afford to come to university and commit to a long programme of study, long reported as an issue for students from socioeconomically disadvantaged backgrounds (eg,^{48 49}).

The fourth and fifth mechanisms were the development of positive and supportive relationships with academic staff and peers, important for social cohesion and assimilation within the university.⁵⁰ Indeed, social influences are a key factor in helping students from disadvantaged backgrounds develop agency and establish a sense of belonging in the university environment.⁵¹ Finally, mechanism six was a sense of belonging, which helped students see medicine as attainable and achievable and prepared them for the transition to medical school. This suggested that the Gateway programme helps students manage the social and cultural shift from home to university⁵² which may be more challenging for WA students compared with their counterparts for whom university/medicine has always been part of their ambition or assumed as achievable (eg,⁵³).

Reflecting on our findings, we propose that the six key mechanisms identified in this study could be viewed as fostering the development of self-determination in the Gateway students. Self-determination is defined as ‘the capacity to choose and to have those choices’ (p38)⁵⁴ and pivots on three tenets: autonomy (feeling that we are able to exercise control over the execution or direction of a behaviour), competence (feeling that we can exercise self-efficacy in interactions with our environment) and relatedness (establishing a sense of belonging, and developing secure and supportive relationships with others). We tentatively suggest that the Gateway programme provided the environment and instructional experiences which helped students develop the cognitive, social and behavioural qualities needed to achieve the goal of studying medicine.^{55 56} Equally, we posit that more privileged students—‘traditional’ medical students—are likely to have had the familial, social and educational contexts which facilitate self-determination earlier in life.⁵⁷ The applicability of this conceptual framework in relation to Gateway to Medicine programmes requires further exploration.

Realist evaluation proposes that the identification of programme theories should precede testing and refining those theories.^{32 34} This process is not linear but iterative. New concepts were identified which were not initially formulated as programme theories during the testing process, so we revisited the initial programme theories to accommodate these new concepts. This process was useful in terms of enriching the analysis, as was collecting data from different perspectives (staff and students: data triangulation⁵⁸). The framework of a realist evaluation is also useful in respect of establishing dependability and confirmability.⁵⁹ Our study was carried out in one context, which may limit its conceptual generalisability, or transferability,^{59 60} however, as stated in our introduction, Gateway to Medicine programmes are increasingly commonplace, and we do not think the aims and content of the programme we studied are unique.

As with any voluntary study, there would have been an element of participant self-selection. However, our student participant group was large and diverse in terms of student background, allowing inclusion of a wide range of views and experiences. The number of staff involved in the delivery of the Gateway programme was relatively small, and we were able to gather the views of those who had the most student contact. Student data were collected by KA and KGS, who were external from the design and teaching of the Gateway programme, to minimise any power differentials and elicit more honest answers about the challenges of the programme. Combining data from interviews and focus groups was a pragmatic decision and we acknowledge that both methods have their strengths and weaknesses.⁶¹ Data collection included a longitudinal aspect, which ensured changes occurring over the course of the Gateway programme’s evolution were captured in the interviews. Even longer-term follow-up could usefully consider students’ progression through medical school and into postgraduate training in respect of how the Gateway programme prepared them for later transitions. Given emerging evidence that medical students from WA backgrounds are more likely to select particular medical careers,^{20 62 63} longitudinal studies of professional identity formation in this group would be a useful addition to the literature.

Another area for future study may be to focus specifically on liminality, to explore the transition and associated emotions between the identity of a school pupil and a medical/university student.^{64 65} Linked to this, our focus is a Gateway programme in a context where 90% of students enter medicine directly after high school. The mechanisms which contribute to the success of a Gateway programme and students experiencing successful transitions are likely to be very different in contexts with graduate entry to medicine. More generally, further work is needed to examine and further test the mechanisms identified in our realist evaluation in other contexts.

Recognising that no research is free of the biases, assumptions and personality of the researcher and we cannot separate self from those activities in which we are intimately involved,⁶⁶ we constantly considered our own positions in relation to the study and the data. We were continuously reflective about how our differing life courses (eg, life stage), education and training, and experience of research and practice in WA to medicine may have shaped data collection and interpretation. Perspectives differed but all authors shared a strong belief in the importance of addressing inequality and increasing medical student diversity through extending knowledge and changing practices.

While a realist approach allowed us to identify exactly what works within the programme, we do not know which mechanism(s) was most important. To examine this requires different methodological approaches, such as a discrete choice experiment (DCE).⁶⁷ A quantitative DCE study would require WA students and relevant staff to rank mechanisms in terms of importance to what works within a Gateway programme. Undertaking such work would promote the application of findings and enable educators to precisely tailor a programme to maximise effectiveness.

Conclusion

The use of realist evaluation and identification of six key CMOs provides an evidence base as to how Gateway programmes work in respect of providing the necessary context and mechanisms for students to overcome barriers to entry to medicine. By unpacking the ‘black box’ in this way, we have shown that Gateway programmes are not solely about providing knowledge and skills but are rather much more complex in respect to how they work.

Data availability statement

No data are available. NA.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

The study was approved by the College of Life Sciences and Medicine Ethics Review Board, University of Aberdeen. Participants provided written consent to take part in the study and were made aware of confidentiality, anonymity and data handling procedure in advance of taking part.