Anatomical Society core regional anatomy syllabus for undergraduate medicine: the Delphi process

Abstract

A modified Delphi method was employed to seek consensus when revising the UK and Ireland's core syllabus for regional anatomy in undergraduate medicine. A Delphi panel was constructed involving ‘expert’ (individuals with at least 5 years’ experience in teaching medical students anatomy at the level required for graduation). The panel (n = 39) was selected and nominated by members of Council and/or the Education Committee of the Anatomical Society and included a range of specialists including surgeons, radiologists and anatomists. The experts were asked in two stages to ‘accept’, ‘reject’ or ‘modify’ (first stage only) each learning outcome. A third stage, which was not part of the Delphi method, then allowed the original authors of the syllabus to make changes either to correct any anatomical errors or to make minor syntax changes. From the original syllabus of 182 learning outcomes, removing the neuroanatomy component (163), 23 learning outcomes (15%) remained unchanged, seven learning outcomes were removed and two new learning outcomes added. The remaining 133 learning outcomes were modified. All learning outcomes on the new core syllabus achieved over 90% acceptance by the panel.

Introduction

The place of basic sciences in medical programmes has been the subject of significant scrutiny and change in recent years (Fincher et al. 2009; Norman, 2012; Bergman et al. 2014). The model of medical education that has been in place since the publication of the Flexner Report (Flexner, 1910) prescribes a period of basic science training lasting 2 years followed by a period of clinical training lasting a further 3 years. This model has come under increasing challenge from three directions (Sugand et al. 2010; Drake, 2014). Time in medical programmes has always been at a premium because of the traditionally content‐rich nature of medical education. Changes in policies by the professional and statutory regulatory bodies, such as the General Medical Council (GMC) in the United Kingdom, have seen a shift in emphasis towards more skill‐based teaching (McHanwell et al. 2007; GMC, 2009; Regan de Bere & Mattick, 2010). Finally, there have been changes in the methods of curriculum delivery with greater emphasis on integrated approaches that stress clinical relevance (Louw et al. 2009). The combined effect of these changes has been to reduce the time available for basic science in the curriculum (Holla et al. 1999; Heylings, 2002).

The amount of time available for anatomy teaching and the contribution it makes to a medical programme have been important aspects of the debates surrounding anatomy education (Regan de Bere & Mattick, 2010). While there has been little disagreement about the importance of anatomical knowledge for safe and effective medical practice, there are different perceptions of what constitutes the body of anatomy knowledge adequate for that purpose (Regan de Bere & Mattick, 2010; Bergman et al. 2014). One stresses the need for students to master an extensive body of factual knowledge more or less in its entirety (Older, 2004). Another would emphasise the importance of teaching anatomy within clinically relevant contexts, implying a move towards a more skill‐based curriculum with reduction in content. There have been reports that students’ knowledge has declined to levels that are unsatisfactory (Monkhouse, 1992; Monkhouse & Farrell, 1999) and this is considered by some to be one of the contributory factors to an increase in litigation for surgical errors (Ellis, 2002; Older, 2004). There have been a number of responses by anatomy educators to this central debate. These have included the use of teaching innovation to present anatomical knowledge in ways that make more effective use of the time available (Sugand et al. 2010), understanding better how students learn anatomy in order to target teaching more effectively (Smith & Mathias, 2010; Smith et al. 2014), discussions on who teaches anatomy (Cahill & Leonard, 1999; Dyer & Thorndike, 2000; Lockwood & Roberts, 2007; Bergman et al. 2014) and the importance of dissection (Winkleman, 2007).

One of the difficulties that surround the central debate over what constitutes an adequate knowledge of anatomy is what might be thought of as the unique characteristic of anatomy (at least within the context of medicine) in that it consists of a large factual base with its own highly specialised language; a feature identified by both teachers and students of anatomy (Regan de Bere & Mattick, 2010). Young (2008) has emphasised the importance of discipline knowledge as a means of deriving general principles of predictive value (‘powerful knowledge’). Thus, helping students to acquire knowledge that is appropriate and achievable requires difficult decisions to be taken about how much should be known and taught. This paper will focus on that central challenge of deciding what anatomical knowledge is necessary for the newly qualified graduate and the consensus view of experts within the field.

As described by Moxham et al. (2014) there have been several attempts to define a core syllabus in anatomy in both Europe and in the USA (Educational Affairs Committee, American Association of Anatomists, 1996; Leonard et al. 1996, 2000; Griffioen et al. 1999; Kilroy & Driscoll, 2006; Swamy et al. 2014; Tubbs et al. 2014). These syllabi have been criticised as being either too detailed or too specialised for adoption into an undergraduate medical curriculum (McHanwell et al. 2007) and have failed to gain widespread acceptance or adoption. As a consequence, in 2003 the Anatomical Society of Great Britain and Ireland (now the Anatomical Society) decided to define the minimum knowledge of topographical anatomy that a new medical graduate is required to demonstrate upon graduation, also recognising the need to define a syllabus that was fit for purpose Initially the syllabus was published on the web pages of the Anatomical Society under the title ‘Setting a benchmark for anatomical knowledge and its assessment, a core curriculum for the teaching of anatomy to medical students’ by Davies et al. (2003). It was subsequently refined to become ‘A core syllabus in anatomy for medical students – adding common sense to need to know’ (McHanwell et al. 2007). This core syllabus was formulated by a process of consensus within the Anatomical Society's Education Committee followed by a consultation largely, but not exclusively, within the membership of the Anatomical Society.

The original Anatomical Society syllabus has been in existence for more than 10 years, during which time there has been a significant advance in the means by which anatomy is taught and assessed and in our understanding of how students approach the learning of anatomy (Smith & Mathias, 2010). Consequently, it seemed timely to undertake a review of the syllabus and at the same time take the opportunity to employ a more rigorous means to arrive at agreement on its content. This core anatomy syllabus was formulated by a process of consensus within the Education Committee followed by a consultation largely, but not exclusively, drawn from within the membership of the Anatomical Society. While this constituted a significant body of anatomical opinion it is open to challenge on the grounds of the breadth of its representation. In deciding to review the syllabus, the challenge within this project was to identify a research approach that would ensure the previously generated learning outcomes in the core syllabus could be systematically and comprehensively reviewed, challenged, developed and adapted within a transparent and structured research framework designed to generate consensus.

The Delphi method, also referred to as a process, technique or study, is a form of consensus survey (Keeney et al. 2011), it is a mechanism of communication between experts and is used to establish agreement between a panel of ‘experts’ focusing on a single, specified issue. It is used by many disciplines to support planning, decision making and policy research (Landeta, 2006) and is utilised extensively to advance and inform changes to curricula and education within healthcare settings (for example, Feigenbaum et al. 2014; Lisk et al. 2014; Moxham et al. 2014; Ross et al. 2014; Tubbs et al. 2014). The Delphi method exposes a panel of experts to multiple iterations of datasets, often learning outcomes, for the purpose of refining the content and reaching some level of agreement among its members. As members of the Delphi panel work independently from one another, each member has equal status within the group and strong personalities or opinions cannot dominate the process. The researchers, working to specified criteria (Yeates et al. 2008), organise rounds of structured communication between the panel members, systematically collating their responses before returning these analyses to the panel for further input.

There are many variants of the method, hence the proliferation of the ‘Modified Delphi’ within the literature, but the procedure usually starts with the generation of items by the panel for inclusion, sharing the items among all participants and further iterations used to refine and ultimately seek agreement on what should and should not be included. The value of this approach lies in the transposition of collegial knowledge held by experts within a field or discipline (professional understanding that is not necessarily discussed but still known) (Eraut, 1994) from its natural implicit to an explicit state. The nature of the modified Delphi process is that they are all different, hence the use of the term modified; what is important is therefore that the process followed in each case is clearly stated. The ‘experts’ that make up the panel are chosen because of their knowledge and insight into the topic under investigation. The nature of what constitutes an ‘expert’ is the subject of much debate, as is the optimum number of panel members (Keeney et al. 2001). Reviews of panel sizes have shown significant differences between studies (Powell, 2003). Findings suggest that it the knowledge the experts hold with regard to the subject under investigation is more important than the size of the panel (Akins et al. 2005). Also contested within the literature is what level of agreement constitutes consensus. Keeney argues that setting the goal before the commencement of the study is good practice, with 75% agreement a minimum requirement for ‘consensus’ (Keeney et al. 2006). Probably the most often cited criticism of the Delphi Technique is the researcher's role in the decision process as they are responsible for defining what constitutes an ‘expert’, choosing the panel, making modifications from respondent comments and selecting what is presented to the panel. The researcher also decides at what level of agreement ‘consensus’ can be considered reached. For these reasons, the researcher must demonstrate transparency in decision‐making in order to demonstrate rigour in the procedures he or she has selected. Delphi methodology has been applied to a range of disciplines, including: Emergency Medicine, Social Sciences, Nursing, Orthopaedics and Mental Health (Kilroy & Driscoll, 2006; Landeta, 2006; Keeney et al. 2011; Ross et al. 2014; Swamy et al. 2014).

The present enquiry arose from a wish to address the criticisms made of the first review and, where necessary, to revise and update the current Anatomical Society Core Syllabus for Medical Students. The core syllabus is a cited and referenced document comprising learning outcomes on the anatomical knowledge expected of medical students at graduation and therefore needs to be current and to stand up to academic scrutiny.

The overall aim of this study was therefore to produce an agreed set of learning outcomes (consensus) for the core anatomical syllabus that were derived from ‘experts’ within the field of anatomy. The specific objectives of this study were:

1. To refine the wording of the learning outcomes to be included within the syllabus.

2. To develop, if needed, new learning outcomes for inclusion within the syllabus based on the collective and collegial knowledge of participants.

This paper will describe the methodology adopted and the challenges that needed to be overcome in arriving at decisions as to what constituted consensus. Its purpose is both to explain how the revised syllabus was achieved and to identify the challenges to be overcome for others wishing to undertake similar surveys to support curriculum change. It will also discuss what the results of such methodologies allow us to do in terms of curriculum design, development and assessment. The accompanying paper (Smith et al. 2015) will set out the results of the survey in the form of the revised Anatomical Society Core Syllabus.

Materials and methods

Ethical approval

The study sought and received ethical approval from the University of Southampton (ERGO 4645).

Study design

The Delphi traditionally begins from a blank script but in order to reduce the number of rounds needed to achieve ‘the items’ for consideration and consequently consensus, a tentative content may be drawn‐up from the literature or textbooks and panel members asked to designate terms as ‘Essential’, ‘Important’, ‘Acceptable’ or ‘Not Required’ in rounds of decision‐making. The approach taken here, since the content had already been formulated as part of the original consensus survey, was to conduct a two‐stage modified Delphi method with alterations of the original learning outcomes in the first stage and with the second stage decisions being confined to a simple ‘accept’ or ‘reject’ of the learning outcomes. The Delphi method used here is considered ‘modified’ because the original Delphi technique begins with blank statements, whereas the modified method begins with a set of already collected items, i.e. the existing syllabus.

Selection of the Delphi panel

The selection of participants to be invited to join the Delphi panel was undertaken by asking members of the Anatomical Society Council and Education Committee to nominate individuals whom they deemed to fulfil the study inclusion criteria of ‘expert’ within this field. The inclusion criteria were that panel members should have at least 5 years’ experience in teaching and assessing medical students during their undergraduate studies and be either an anatomist or a clinician. To avoid bias, it was decided that all members of the original group that devised the syllabus would be excluded from the Delphi panels. Sixty‐one individuals were initially proposed. These included five members of the original core syllabus group, who were immediately excluded from the survey. The rest were accepted and invited to participate. An initial email was then sent to the nominees inviting them to join the study. Five nominees were found to be untraceable by email, making the final invited sample 54. A reminder email was sent 30 days later. Thirty‐four individuals agreed at the time to participate in the study (n = 34). No members of the research team were involved in the Delphi panel.

Preparation of the learning outcomes statements for the survey

The original Anatomical Society Core Syllabus consists of 163 learning outcomes statements (to be referred to simply as learning outcomes hereinafter) divided into eight sections, comprising the regions of the body, vertebral column, upper limb, thorax, abdomen, pelvis, lower limb, head and neck, and neuroanatomy plus a section relating to anatomical terms. There were 19 learning outcomes related to neuroanatomy and these were excluded from the Delphi study, as it was decided that neuroanatomy would be better considered as a separate syllabus. Also excluded from the survey were the summary statements that accompanied the original statements, as they merely reiterated the content of those statements. The original learning outcomes of the core syllabus document were entered into SURVEY MONKEY using the University of Southampton Survey Monkey Account (Survey Monkey, Palo Alta, CA, USA). (The authors are pleased to acknowledge the support of the University of Southampton in allowing us to use this account). In addition to the statements to be reviewed, there were 11 demographic items, making a total of 174 items. The purpose of the demographic items was to provide information on the range of expertise within the panel, which would enable a judgement to be made on how representative it was of anatomical expertise. For each of the learning outcomes, check boxes were provided for the panel members to record their decisions at each of the two stages. Panel members could not return to review earlier statements once they had recorded their decisions. An open comments text box was also provided for panel members so that they could, if they wished, record the reasons for their decisions or any other comment relating to the statements being reviewed. Prior to being made live, the data‐collection form was checked and piloted by the research team.

Research team

The research team comprised all the present authors. One author (JS) had a different role to the other researchers, involving leading and guiding the Delphi methodology, the decision process and its consistency. JS was not involved in any anatomical decisions. The remaining members of the research team (CS, GF and SMcH) were always all involved in the decision‐making process of the methodology. Each statement and decision was collectively made by three researchers assisting with an unbiased and triangulated decision‐making process. The original authors were involved only in Stage 3 as detailed below and the final new learning outcomes are presented in Smith et al. (2015) with a combination of the research team and the original authors of the 2007 syllabus.

Processes

Stage 1 preparation

In the first stage the panel members were asked to consider each learning outcome to decide whether it should be included in the revised Anatomical Core syllabus and, if so, in what form. Accordingly, panel members were asked either to accept it as it stood without modification, reject it completely or accept it with modifications. If a modification was being proposed, panel members were asked to write the modification in the open comment text box. A sample screen shot of the survey is illustrated in Fig. 1. The survey was left open for 6 weeks. After this time, the survey results were reviewed by the research team.

Figure 1.

Sample screen shot of survey from Stage 1.

Stage 1 analysis

Learning outcomes achieving a consensus level of 100% were accepted. Learning outcomes rejected by the panel members because they were a lower level than the pre‐agreed consensus level of 90% (decided by the researchers) were identified but no further editing action was taken with these unless modifications or comments were made in the open text box indicating how these could be refined. One other reason for rejecting statements was when panel members identified the content of a statement as repeating content found elsewhere.

A total of 699 comments were made by the panel. For the purposes of analysis, the panel comments were reviewed by the research team and following discussion and agreement were assigned to one of the following categories: Supportive (S), Contextual (C), Modify (M) and Not Relevant (NR) Supportive comments were identified as being those which supported the inclusion of the statement without further revision. Examples of comments identified as supportive were: ‘yes important’, ‘key to know’. These comments were noted for recording purposes but no further action was taken. Contextual comments were those from which it could be inferred that support for inclusion or rejection of a learning outcome was being given but where the comment itself did not require any action on the part of the research team to modify that learning outcome. An example of a comment in this category is on learning outcome 43: ‘Describe the main anatomical features of typical and atypical vertebra. Identify the atlas, axis, other cervical, thoracic, lumbar, sacral and coccygeal vertebrae and recognise their characteristic features’. One of the comments made was: ‘Important as bones susceptible to stress/fractures, back injuries/problems are common’. Comments classified as not relevant were diverse in nature: for example, some panel members suggested the inclusion of embryology or histology in the syllabus, content that was specifically excluded. Other examples would be where panel members suggested the addition of material to a statement where it was already present. Finally, possibly to save themselves time, some panel members would write a comment ‘as above’ or ‘see above’ but in nearly all cases it proved impossible to relate the comment to a particular learning outcome appearing earlier in the survey. This category of comments was noted but not acted upon.

The most numerous group of comments (436 of a total of 699, see Results) were those that were classified as Modify, in which panel members were suggesting changes to the learning outcomes. It was these comments that required the most active consideration from the point of view of making changes that would result in higher levels of consensus being achieved in Stage 2. For each learning outcome, all Modify comments were carefully reviewed and considered by members of the research team and relevant alterations made to the learning outcome. This was the most challenging group of comments to deal with, especially in cases where comments from individual panel members were contradictory. To ensure consistency in how the comments made by panel members were utilised, a set of rules was drawn up and agreed in advance by the research team.

These rules were applied to all learning outcomes where modifications were proposed (even where there was 100% acceptance). They can be summarised as follows:

1. If all, or the majority of, comments suggest a particular change, then the learning outcome will be modified accordingly.

2. If contradictory comments are being made, then discussion between the research team members will be used to decide which changes should be adopted and which rejected. The basis of these decisions should be ensure clarity and reduce repetition.

3. In situations where one comment is felt by the research team to be especially apt, even if no other panel members’ comments match, then this single comment could be used to modify a learning outcome.

4. Where a panel member makes a comment regarding inconsistency in terminology relating to a small number of learning outcomes, then the research team will discuss whether this inconsistency should be addressed across the whole syllabus and changes made. For example, some panel members observed that vascular supply and blood supply had been used interchangeably. A decision was taken to use vascular supply throughout the syllabus.

5. Anatomical terminology follows the guidelines laid out in Terminologica Anatomica (1998).

6. All decisions are recorded.

7. These rules are applied, recognising that all changes will receive further scrutiny in Stage 2. Where any change results in lower levels of consensus being achieved, then the research team will restore the original learning outcome.

At the end of Stage 1 a revised syllabus was produced in which some learning outcomes were retained unchanged, other learning outcomes were rejected including those which duplicated material and in which many learning outcomes were modified. This revised syllabus was then subject to a further round of scrutiny (Stage 2).

Stage 2 preparation

In Stage 2 the same panel of 51 experts were invited to review the modified or new learning outcomes. In Stage 2 they were simply asked to either accept or reject outcomes; there was no option to modify learning outcomes. However, an open free text comment box was retained at the end of each of the eight sections of the syllabus, rather than for each learning outcome, to allow panel members to make further comments if they wished about any of the learning outcomes. Learning outcomes which had been accepted by the panel at 100% agreement were included in the Stage 2 survey, so that panel members could identify them as being part of the syllabus, but no further input was sought.

Stage 2 analysis

In Stage 2 the consensus level for acceptance of a comment was also 90%. As panel members could only accept or reject statements and comments were not permitted for individual learning outcomes, different rules were required in deciding how to utilise comments made at the end of each section in the interests of refining and clarifying further the learning outcomes. The rules followed for Stage 2 analysis were:

1. Where grammatical or typographical errors are identified, these are simply corrected.

2. If duplication of content between learning outcomes is identified, these duplications are removed.

Stage 3 analysis

At the conclusion of Stage 2 the revised Anatomical Society syllabus was then referred back to the original authors for comment. This ‘third stage’ is not considered a part of the Delphi but it was important here for two reasons: first, it gave the original authors, who had been exempt from the panel, an opportunity to make any minor editorial changes they had wished to see and secondly, it offered an opportunity to correct any anatomical errors or oversights. These comments and amendments were recorded.

Results

The results presented in this paper relate solely to the way in which the Delphi method was applied in order to produce the revised Anatomical Society Core Syllabus (Smith et al. 2015).

Composition of the Delphi panel and response rates

For Stage 1, of the 51 experts that formed the panel, 34 completed Stage 1 of the survey. Four of the participants completed one of the eight sections only and their responses were excluded from the final analysis to ensure that syllabi had been viewed holistically. Of the remaining 30 participants who completed the survey, not all (three) provided responses for every learning outcome, so that the number of respondents for a particular learning outcome varied from 27 to 30. The demographic composition of the panel for Stage 1 was 19 anatomists and 11 clinicians. The composition of the panel involved a range of seniority levels including: lecturer, senior lecturer, reader, professor, dean, specialist registrar and consultant. The specialties of the clinicians included gynaecology, pathology, ENT, pain, radiology, orthopaedics, trauma and paediatrics. These figures represent an overall response rate for those included in the analysis of 59%.

For Stage 2, all 51 experts were once again invited to participate. On this occasion, 29 participants responded to the invitation, although in the event, only 26 completed the survey. However, only 17 of those participants (59%) had responded during Stage 1 of the survey, meaning that nine new participants had elected to join the study. Again, as in Stage 1, not all participants made responses to all learning outcomes, the number varying between 23 and 26. The demographic composition of the panel for Stage 2 was 13 anatomists and 14 clinicians. The range of seniority levels and specialties of the clinicians was the same as Stage 1.

Overall

The final overall panel (n = 39, response rate 76%), involved 30 panel members at Stage 1 and 26 panel members at Stage 2. Seventeen (59%) of Stage 2 panel members also completed Stage 1, and nine new members completed Stage 2. The demographic composition of the panel included 77% responses from England, 8% from Ireland, 2% from Scotland and 13% from Wales.

Levels of consensus

The key challenge in any application of the Delphi method is deciding when ‘consensus’ has been reached. To avoid potential sources of bias, consensus levels need to be agreed and established before data is collected. For the purposes of this study the percentage level of agreement from the Delphi panel members was set at 90% for both Stages 1 and 2. This was the minimum level and, in the event, many learning outcomes achieved higher levels of consensus.

The overall effect of this upon the composition of the syllabus is summarised in Table 1. At the outset of the study the original syllabus as published in 2007 consisted of 163 learning outcomes distributed across the eight sections of the syllabus as shown in Table 1. At the end of the process the overall number of learning outcomes had fallen to 156, with learning outcomes being removed both at Stage 1 and 2. A reduction of seven learning outcomes does not seem large in itself given the level of input but that simple figure conceals the fact that although the number of learning outcomes did not change dramatically, the content of many of the outcomes remaining had been refined as the result of comments made during Stage 1 that were then accepted at the agreed consensus level in Stage 2.

Table 1.

Learning outcome numbers by body region over the Delphi process

Section of syllabus	Initial number of learning outcomes	Number of learning outcomes after Stage 1	Number of learning outcomes after Stage 2	Number of learning outcomes after Stage 3	Number of original (from 2007) learning outcomes remaining (%)
Anatomical terms	4	4	5	5	0
Head and neck	39	38	38	37	6 (16)
Vertebral column	7	7	7	7	0
Thorax	24	23	24	24	6 (25)
Upper limb	22	20	20	21	1
Abdomen	21	20	21	21	2 (1)
Pelvis and perineum	20	20	20	19	2 (1)
Lower limb	26	23	22	22	6 (3)
Total	163	155	157	156	23 (15)

Outcomes of Stage 1

Overall nine learning outcomes (6%) were accepted at a consensus level of 100% without further refinement (Table 2). Thirty‐two learning outcomes were rejected by one or more individuals (20%). Of these, 24 were rejected by one participant, five by two participants, one by three participants, one by four participants and one by five participants.

Table 2.

Learning outcomes accepted with 100% consensus at Stage 1

13. Describe the anatomical relationships of the meninges to the spinal cord and dorsal and ventral nerve roots, particularly in relation to root compression and the placement of epidural and spinal injections. Describe the anatomy of lumbar puncture

48. Describe the arrangement and contents of the superior, anterior, middle and posterior parts of the mediastinum

49. Identify the major anatomical features of each chamber of the heart and explain their functional significance

72. Explain the nerve supply of the parietal and visceral peritoneum and the role of the visceral peritoneum in referred pain

117. Describe the structures at risk from a fracture of the femoral neck or dislocation of the hip and explain the functional consequences of these injuries

141. Demonstrate the origin, course and major branches of the common, internal and external carotid arteries and locate the carotid pulse

142. Describe the courses of the accessory, vagus and phrenic nerves in the neck

159. Describe the stages of swallowing and the functions of the muscles of the jaw, cheek, lips, tongue, soft palate, pharynx, larynx and oesophagus during swallowing

164. Name the paranasal sinuses, describe their relationship to the nasal cavities and sites of drainage on its lateral wall and explain their innervation in relation to referred pain

At the end of Stage 1, 47 learning outcomes remained unchanged (29%). Eight learning outcomes were removed (5%) (Table 3) and six new learning outcomes were added (Table 4).

Table 3.

Learning outcomes removed after Stage 1

33. Describe where the axillary, musculocutaneous, radial, median and ulnar nerves are and why these are commonly injured. Describe the functional consequences of these injuries

34. Explain the loss of function resulting from injuries to the different parts of the brachial plexus

61. Demonstrate the surface projections of the margins of the pleura and the lobes and fissures of the lungs

88. Demonstrate the points of attachment of the muscles of the abdominal wall and those of levator ani

127. Describe the movements of inversion and eversion at the subtalar joint, the muscles responsible, their innervation and main attachments

129. Describe the structures at risk to a fracture of the femoral neck or dislocation of the hip and describe the functional consequences of these conditions

131. Discuss the structures of the lower limb that may be used for autografts

150. Describe the relationship of the termination of the facial vein (draining into the internal jugular vein) and the mandibular branch of the retromandibular vein (supplying facial muscles controlling the angle of the mouth) to the submandibular gland and related upper jugular lymph nodes in relation to exploration of this area

Table 4.

New learning outcomes added after Stage 1

1. Discuss the arrangements of the pleural membrane. Describe their clinical significance in conditions including pneumothorax

2. Describe the major veins of the face and neck and their important tributaries

3. Describe the fascial planes of the neck and explain their importance in the spread of infection

4. Describe the anatomy of the breast including its neurovascular supply. Explain the lymphatic drainage of the breast and its clinical relevance to metastatic spread

5. Explain the clinical significance of normal and anatomical variation

6. Describe the key anatomical differences between a neonate, child and adult

A total of 699 comments on the learning outcomes were made by panel participants, which represents (for the 163 learning outcomes) an average response rate of two to three comments for each learning outcome. The number of comments for any one learning outcome ranged between 0 and 10. A significant number of comments made by the Delphi panel participants required no further action on the part of the research team, as Supportive (n = 89) or Contextual comments (n = 35) were simply recorded but were not used to modify a learning outcome. The comments that were deemed ‘not relevant’ also required no further action by the research team (n = 139).

The focus was thus upon those comments suggesting modifications (n = 436) to the learning outcomes, especially where the decision of the panel to accept the comment fell below the pre‐agreed consensus level. The research team were then faced with decisions as to how to modify learning outcomes to achieve panel consensus that would then make them acceptable in the second round. This could be particularly challenging in situations where several comments had been made by the panel participants and those comments had to be reconciled to produce a single learning outcome. Table 5 illustrates the categorisation of comments. The key to consistent decision‐making by the researchers was the formulation of rules for dealing with comments prior to the analysis being undertaken and then the recording of any decisions made should they need to be revisited after Stage 2. This permitted the research team to be transparent in their decision‐making and also to create an audit trail of decision‐making. An example is provided in Table 6 illustrating how those rules were applied in relation to a single learning outcome and shows how the comments received at Stage 1 were incorporated into a revised learning outcome for Stage 2. The rules allowed for the fact that any changes would be subject to the scrutiny of Delphi panel participants and would need to reach the pre‐agreed consensus level if the modifications were to be carried forward into the final syllabus.

Table 5.

Categorisation of comments from Stage 1 and 2 summary free text boxes

Learning outcome	Stage 1				Stage 2
	Modify	Not relevant	Supportive	Contextual	Modify	Not relevant
Anatomical terms	28	0	3	8	2	0
Vertebral column	25	2	11	8	2	1
Upper limb	92	12	20	0	2	4
Thorax	77	19	27	2	7	4
Abdomen	60	29	1	0	2	2
Pelvis	44	19	2	0	3	1
Lower limb	50	21	9	9	3	2
Head and neck	60	37	16	8	5	1
New learning outcome	NA	NA	NA	NA	4	1
Total	436	139	89	35	30	16

Table 6.

Example of Delphi results on one learning outcome

Delphi Stage 1	Comments M = Modify NR = Not relevant S = Supportive C = Contextual	Delphi Stage 2 (changes shown in italics)	Accept/reject	Final learning outcome
No. 16. Describe and demonstrate the main anatomical landmarks of the clavicle, scapula, humerus, radius and ulna. Recognise the bones of the wrist and hand and their relative positions, identify those bones that are commonly damaged (scaphoid and lunate) and predict functional impairment following such damage.	Demonstrate would be better (M) Rather: ‘…Identify the bones of the wrist and hand and their relative positions…’ (‘recognise’ is an ambiguous learning outcome word) (M) Not necessary to be able to identify each and every bone in wrist, but important to know about scaphoid and lunate (M) Commonly injured. (NR) Lunate dislocation is rare!! I would therefore exclude it. Scaphoid fracture is common. Blood supply to scaphoid is very important. (C) Identify clinically relevant anatomical landmarks on clinical or radiographic examination. Describe the osteology of the pectoral girdle and upper limb. Recognise common musculoskeletal injuries in the pectoral girdle and upper limb and discuss their causes and complications. Identify peripheral nerve injuries in the pectoral girdle and upper limb and discuss their causes and complications. (M) The first part can be interpreted as all classic bony landmarks for each bone, which are not required for the junior doctor. Perhaps state the major palpable or functional. features (M) Clinically relevant (S).	No. 16. Describe and demonstrate the main palpable anatomical landmarks of the clavicle, scapula, humerus, radius and ulna. Identify the bones of the wrist and hand and their relative positions, identify those bones that are commonly injured, e.g. scaphoid and predict functional impairment.	Accept = 26 Reject = 1	No. 74. Describe and demonstrate the main anatomical landmarks of the clavicle, scapula, humerus, radius and ulna. Identify the bones of the wrist and hand and their relative positions, identify those bones that are commonly injured, e.g. scaphoid.

Stage 2

At the conclusion of Stage 2, 108 learning outcomes achieved 100% consensus, and a further 48 learning outcomes achieved the previously agreed 90% or greater consensus level for acceptance. Thirty of these learning outcomes were rejected by only one participant. Two individuals rejected a higher rate than others, rejecting 20 and 21 learning outcomes. Not considering these individuals, the average number of rejections was 2. This low level of rejections justifies the decision to set the level for consensus at 90%. Only six learning outcomes were rejected, of which four were the new learning outcomes added to the syllabus after Stage 1 in response to comments made by panel members, while the remaining two learning outcomes simply duplicated syllabus content that appeared in other learning outcomes within the syllabus. The relatively low level of rejection of learning outcomes from Stage 2 (confined to four of the new outcomes and two further outcomes that simply duplicated content elsewhere) was taken as evidence that learning outcomes had been modified to reflect the consensus views of panel participants as exemplified through their comments. A total of 30 further free text comments from the end of section summary boxes were reviewed by the research team and categorised using the rules outlined in Methods and appropriate modifications were made.

New learning outcomes in Stage 2

As the result of the analysis of comments from Stage 1 concerning the coverage of the original syllabus, areas were identified as not being covered in that syllabus. As a result, the research team created six new learning outcome statements based upon these comments for inclusion in Stage 2. The areas identified by the Delphi panel members as needing to be covered are detailed in Table 4. In the event these were found to have a higher rate of rejection, being rejected on average by four individuals. Four of these learning outcomes were rejected because they failed to meet the necessary level of consensus. Two of the new learning outcomes were accepted by Delphi panel members and subject to minor modifications, highlighted by comments while the others were removed. The new learning outcome statements are numbers 4 and 6 in Table 4.

Summary

At the end of Stage 2, from the original syllabus, 47 learning outcomes (29%) remained unmodified. Eight learning outcomes were removed and two new added. The remaining 108 learning outcomes were modified. An overall acceptance rate of over 90% was achieved. Review at Stage 3 resulted in one learning outcome being removed as it was felt to duplicate another outcome, this left 156 final learning outcomes in the published syllabus (Smith et al. 2015). By the end of Stage 3 of those 156 final learning outcomes 23 were completely unmodified from the original syllabus meaning 133 had been subject to modification at some point during the process (see Table 1).

Discussion

Determining the necessary knowledge base for professional degrees in which many disciplines are represented has been a subject of significant controversy. While discipline experts will wish to see their subject represented fully within a programme, issues of balance and depth will always arise. The place of knowledge in curricula is a debate found across education policy at all levels going back at least 50 years. Dewey (1966) emphasised the important role that subject‐matter must play within education as one means to inform our experience. Young (2008) has argued for the necessity of transmitting to students, objective knowledge from which they derive general principles, which he terms ‘powerful knowledge’. As anatomists we would surely recognise the force of this argument in relation to human structure. If we accept Dewey's argument (Dewey, 1966) that our action in the world should be intelligent and informed and not ‘stupid’, then we must argue that anatomical knowledge matters in the context of safe and effective practice.

This paper presents the results of a Delphi analysis of the Anatomical Society Core syllabus (McHanwell et al. 2007) revisiting the results that were presented in the original 2007 syllabus. As the result of this research process the original syllabus has remained broadly unchanged in terms of its scope but with a considerable degree of refinement and restructuring of the original learning outcomes. Learning outcomes have been reworded in many instances to clarify their intent, a small number of learning outcomes have been removed and other learning outcomes added to cover some omissions. At the conclusion of the second stage of the process it has been possible to achieve consensus levels of at least 90%, with many learning outcomes being accepted at 100% consensus.

There have been a number of previous attempts to arrive at a core syllabus in anatomy for medical students (Educational Affairs Committee, American Association of Anatomists, 1996; Leonard et al. 1996, 2000; Griffioen et al. 1999; Kilroy & Driscoll, 2006; Tubbs et al. 2014). This reflects the extensive and continuing debate about the breadth and level of anatomical knowledge needed by medical students, which has been a prominent feature of discussions over anatomical pedagogy for many years (Sugand et al. 2010). The lack of detailed guidance from professional, statutory and regulatory bodies was one impetus that led the Anatomical Society to publish its core syllabus in 2007. The other impetus was dissatisfaction with the core syllabi already published. These syllabi were often highly detailed with the expectation of the level of content to be mastered set at far too high a level, unrealistic in the context of a crowded medical programme. The guiding principle adopted by the Anatomical Society, at the time, was one of common sense, recognising that anatomy was only one of many subjects needed to be covered in a medical programme and that balance had to be sought given the multiple demands being made upon medical students. The process by which that syllabus was arrived at was through consensus discussions between members of the Education Committee of the Anatomical Society. This was followed by a period of open consultation among anatomists and other professionals prior to publication. One difficulty that approach gave rise to, was that because relatively few people took time to respond in a detailed fashion, it was not possible to say with certainty the response had been balanced. It was for this reason, and because the syllabus had been in place for several years, that it was felt timely to undertake a review of the content of the syllabus using a more recognised research method and employing a Delphi method for this purpose. Given the scope of the original consultation it was interesting to note that the broad structure of the syllabus remained, albeit with extensive refinement.

The reason for the choice of a Delphi method was that it offers significant advantages over simpler forms of consultation. It allows people who are members of the panel to work independently and prevents strong opinions (or individuals), influencing the outputs. It provides structure to the consultation and a means for systematically recording consultation. It provides a means to uncover professional knowledge that may be semi‐tacit and often not expressed or discussed explicitly. The usual starting point for a Delphi method to be undertaken is to start with the generation of a list of items for inclusion (Moxham et al. 2014). This is then followed by one or more rounds of consultation until the pre‐agreed level of consensus is achieved. This study began from a different starting point, with a set of learning outcome statements rather than a list of terms. Our study has shown that a Delphi method can prove an effective method to establish consensus on a syllabus, albeit one that is time‐consuming and complex to analyse. Although in this study we gained a lot of consensus, it is not complete consensus, as is frequently the case for Delphi method.

There are two criticisms frequently made of the Delphi process. The first relates to the initial choice of items, which, it is often said, allows for the introduction of bias at an early stage (Keeney et al. 2011). The second relates to the composition of the panel. In this study, the choice of items had already been made by utilising the existing syllabus. The items were not in the form of topics, as might normally be the case, but rather a series of learning outcomes (something that will be taken up later in this discussion). These were derived from a process also born from consensus, although not as systematically approached as here. Other Delphi processes are in progress to arrive at topic lists (Tubbs et al. 2014) and it will be instructive to see how much these two different processes converge (Moxham et al. 2014); the points of strong similarity and areas of difference will raise issues for further discussion with regard to the production of a syllabus in anatomy for medical students.

The issues of panel size and selection are both contested in the literature and these issues interact such that panel size can be seen as a lesser issue if the panel itself is representative of the knowledge necessary and relevant to the subject(s) being studied. The consensus level for agreement between panel participants was set at 90%, a significantly higher level than the 75% frequently regarded as good practice in studies of this type (Keeney et al. 2011; Eskes et al. 2014; Hewitt et al. 2014). We consider that the results of this study justified that decision. On the other hand, it could be argued that the consistency of the findings was simply a product of a small, homogeneous panel rather than a reflection of the robustness of the approach. However, the composition of the panel did comprise a broad range of anatomical and clinical opinion and as such is regarded as suitably representative (Akins et al. 2005). Another issue relates to the composition of the research team in which there were no clinically qualified members. This could lead to a concern about the possibility of bias in the judgements made about how to handle comments in Stage 1, especially where clinical issues arose. However, it is important to be clear that it was not the research team that were responsible for final decisions on learning outcomes; these were always made by the panel.

Another criticism that might be levelled at the approach adopted here was the use of learning outcomes as the basis of the syllabus rather than a broader topic‐based approach. The use of learning outcomes in curriculum design has become a widespread feature of learning at all levels of education and that was one motivation for their use in the original syllabus. Yet their use has been criticised on the grounds both of their inadequacy and of their unwanted effects on assessment and learning (Hussey & Smith, 2002; Yorke, 2008). The almost universal use of learning outcomes in medical education has led us to retain them in the core syllabus. Learning outcomes also help curriculum planners in their decisions as to the level at which to pitch knowledge. We would agree with Hussey & Smith (2002) that their full interpretation is critically dependent upon context, which in the case of anatomy will emerge through its clinical relevance and its application to the solving of clinical problems.

Of course a syllabus such as has been produced by this Delphi process is not simply content and involves knowledge, understanding and skills. What this syllabus does, we hope, is provide a framework for decisions about course design in relation to anatomical content. How it will be used is at the discretion of every medical school that decides to adopt it and, in particular, when in the curriculum these outcomes might be achieved will inevitably vary between medical schools. It is our hope and intention that this syllabus will provide a checklist that will help schools to ensure that a coherent anatomy syllabus can be delivered across a medical programme. This coherence could provide some of the benefits of a discipline‐based education within a problem‐ or scenario‐based curriculum. It could also support the learning of students for, as Entwistle (2009) has argued, deep learning is favoured in curricula that are coherently constructed. In arriving at this consensus syllabus we have deliberately not sought to prescribe a curriculum in terms of the means by which it would be delivered, for we do not propose here to enter into the debate about the relative virtues of traditional discipline‐based curricula, problem‐based learning or learning based upon scenarios. These approaches have complementary strengths which are entirely predictable, with knowledge outcomes being better in didactic courses whereas problem‐solving skills being better in problem‐based curricula (Hattie, 2009). We do agree with the proposition (Hattie, 2009) which identifies the importance of surface learning, by saying that in order for deep and constructed learning to occur, surface learning is also necessary. From this idea we would argue that necessary knowledge‐based syllabi do have a part to play through their emphasising the importance of knowledge upon which deep learning must be founded. What is sometimes said to occur in problem‐based curricula is the fragmentation of discipline‐based learning, resulting in gaps in student knowledge that can compromise deep learning. We consider this syllabus could help in addressing that issue. However, we also recognise that in anatomy, despite concerns expressed by students in some studies, the evidence about the effect of curriculum structure is inconclusive and that the structure of a curriculum is not critical for anatomy learning (Prince et al. 2000, 2003; Bergman et al. 2011, 2014). What does seem to be important is the teaching of anatomy in context, and this has been emphasised in numerous studies (Bergman et al. 2011, 2014). It is the importance of knowledge both for clinical application and safe practice that has been the prime motivation for carrying out this study to arrive at this series of consensus learning outcomes.

In summary, a Delphi method has been used to review and revise the original Anatomical Society syllabus and establish a consensus on the content of that revised syllabus. The details of the revised syllabus are given in the accompanying paper (Smith et al. 2015). Within a hectic curriculum it is important to ensure that the teaching of specific subjects is targeted, specific and coherent, and under constant review in order to be certain that it remains fit for purpose. Medical curriculum planners have to balance the competing demands of different disciplines, ensure that new disciplines find their place in the curriculum as it is developed and ensure also that students are equipped with the necessary skills to prepare them for practice. In this difficult balancing act, it is important to ensure that students have a foundation of knowledge to equip them with the appropriate skills (Young, 2008). Discipline‐based syllabuses such as the one in anatomy presented here can provide a type of checklist to ensure that within a programme discipline, coherence – so important for deep learning (Entwistle, 2009) – can be maintained.

One element that is important is that this syllabus is implemented in the best way given the structure of the curriculum at a given institution. Curricula are like road maps and have the flexibility to accommodate local requirements (Pawlina & Drake, 2014), with the syllabus presenting key junctions and turns that must be taken on a journey successfully to get to the end destination.