Determination of required anatomical knowledge for clinical practice in emergency medicine: national curriculum planning using a modified Delphi technique

Abstract

Objectives

To identify the content of the postgraduate anatomy curriculum for trainees in emergency medicine (EM) in the UK.

Methods

Modified Delphi technique involving 160 randomly sampled consultants in EM. The outcome measure was percentage agreement, based upon a three round iterated Delphi process, of participants to items derived from a standard anatomy text being core knowledge for postgraduate clinical practice in the specialty. A national curriculum document was derived as a result of the data obtained.

Results

Response rates ranged from 61% to 70% after three Delphi rounds. From an initial overall questionnaire content of 808 discrete items, 64% was retained as core required knowledge following iteration. This formed the basis of a national consensus anatomy curriculum both to inform question development in postgraduate EM examinations and to benchmark anatomical knowledge requirements for safe clinical practice.

Conclusions

Application of a national consensus methodology allows for determination of curricular content in anatomy. The principles can be applied to all aspects of training and curricular policy at national collegiate level to guide the development of robust documentation.

The relative importance of anatomy as a taught discipline and the methods used in its teaching have significantly altered in recent years.¹ For graduates entering training in emergency medicine (EM), however, the challenge of acquiring sufficient anatomical knowledge to achieve success in postgraduate examinations remains. The self directed and systems based approaches to learning now used in medical schools are not reflected in the postgraduate educational environment,² where service commitments and rationalisation of clinical supervision may compromise the time available for formal study.

The literature suggests that there are significant concerns about the effects of reformed undergraduate approaches to anatomy in terms of demonstrated knowledge by postgraduate trainees.³,⁴ The scope of knowledge truly required for safe, autonomous clinical practice has, however, remained undefined, and the content of postgraduate medical curricula has only recently been subject to meaningful scrutiny. The introduction of the Postgraduate Medical Education and Training Board (PMETB) in late 2005 in the UK brought with it novel requirements for parent colleges to develop clear and robust curricular documents with which to inform their training and examination structures.⁵ Within this context, the content of training curricula should arguably reflect key or “core” knowledge so that time is not wasted in the study of material that, however potentially interesting to know, is more peripheral in its practical daily application for postgraduate clinicians in training.

EM is a composite discipline encompassing the safe initial assessment of a range of undifferentiated conditions. Anatomy is a key element of safe clinical practice; a significant portion of the daily workload is concerned with musculoskeletal injury, and there are anatomical implications to procedural skills and the correct interpretation of symptoms and signs, as with any other branch of medicine.

Knowledge requirements in any given domain of medical practice are not easy to determine, and the methods historically used to guide curricular content have ranged from small expert group opinion to individual preferences. The use of a Delphi consensus methodology to develop relevant content has been described in relation to aspects of skills and knowledge, both in industry and in medicine, since the 1960s.⁶,⁷ There are, however, no published reports of its use in a national project to comprehensively determine curricular training requirements for a specialty.

In the context of national curriculum planning, use of a group opinion tool must aim to maximise validity by the use of relatively large and suitably representative expert panels. Many reported applications of the Delphi technique involve small numbers of participants,⁸,⁹ which reflects the practical feasibility of the handling of large numbers of questionnaires and their interpretation. Although this is acceptable for the determination of small aspects of practice, it is not sufficiently inclusive for specialty based curriculum development.

We used the Delphi technique to determine consensus in relation to required knowledge of anatomy for postgraduate trainees in EM, enabling the development of a national curriculum document. The principles of the technique extend to any domain of medical practice.

METHODS AND PARTICIPANTS

Development of questionnaire content

Initial questionnaires were developed based upon topographical anatomical areas. The source document was Last's Anatomy.¹⁰ From the chapters of the book and its index, anatomical items were listed for each region. Using this technique generated four separate initial questionnaires. The numbers of discrete items listed on each ranged from 233 (head and neck) to 398 (upper and lower limb). Pilot studies were undertaken to test questionnaire format and ratings scales.

Within the body of each document, items were listed alongside a four part Likert scale. Participants were asked to express their opinion on the relevance of knowledge of each anatomical item from “1” (an essential tem of knowledge for practice) to “4” (no knowledge required for practice). A sample initial questionnaire section is shown in fig 1.

Figure 1 Sample section from round 1 questionnaire: upper limb

Selection of participants

The database of the Fellows of the College of Emergency Medicine was entered into an anonymised spreadsheet format. For each of the four questionnaire streams, 40 participants were randomly selected. Potential participants found to be working in substantive overseas posts or those engaged primarily in allied clinical practice were excluded, and a new anonymous selection made for that participant. Letters of invitation to participate were then sent to the 160 selected Fellows. These were followed up with an initial questionnaire mailing within 1 month. Participants were asked to then review and complete the initial questionnaire. Participants from one stream were then removed from the pool so that selection for more than one stream was avoided.

Questionnaire rounds

The returned questionnaires for each stream of study were collated and assessed. For each anatomical item, cutoff for progress of that item into round 2 was deemed to be <75% consensus of that item scoring 2 or better in the initial round. Free text comments were noted, and suggestions regarding format and clarification incorporated into the subsequent round.

Analysis of the final questionnaire rounds generated consensus documents for each of the four assessed anatomical regions. These were incorporated into a single curricular content document which went forwards for ratification by the College education and examination committee prior to its adoption as the national anatomy curricular document for trainees in the specialty.

RESULTS

In total, 160 initial questionnaires were sent out to invited participants. The response rates for each stream of the project, by anatomical region and by round, are shown in table 1.

Table 1 Project response rates according to anatomical region.

Anatomical region		Response rate
		Round 1		Round 2		Round 3
Upper and lower limb		77		70		70
Head and neck		67		64		61
Chest and abdomen		74		70		66
CNS and cranium		76		73		69

Rate is shown as a percentage of the total number of participants who returned completed questionnaires for the region indicated. For each of the four anatomical regions, 40 participants were sent the initial round 1 questionnaire. CNS, central nervous system.

Analysis of the Likert scoring for items in each round enabled the questionnaires to be sequentially refined. The proportions of content eliminated in each round, by anatomical region, are shown in Table 2.

Table 2 Content refinement by anatomical region and round.

Anatomical region		Questionnaire stems, number (%) retained
		Quest 1		Quest 2		Quest 3
Upper and lower limb		172 (100)		122 (71)		109 (63)
Head and neck		259 (100)		212 (82)		167 (64)
Chest and abdomen		202 (100)		160 (79)		147 (73)
CNS and cranium		175 (100)		127 (73)		100 (57)

Quest, questionnaire; CNS, central nervous system.

Free text comments from the initial round related primarily to formatting. There were some comments in relation to the terms or wording used in subsequent rounds, but in no case was it mentioned that this had materially altered a respondent's ability to complete the exercise.

The final amalgamated curricular document comprised 523 discrete anatomical facts deemed by consensus to constitute required knowledge for postgraduate trainees in EM. This list was then reformatted into a more readable style, which created a 44 page applied anatomy document suitable for use by trainees and available on the Faculty's website as a downloadable item (www.emergencymed.org.uk/cem/curriculum).

The anatomy component of the MCEM examination was subsequently reviewed: multiple choice questions are now based upon the consensus document. Knowledge of the content of the document will be reflected in success in the anatomy component of the examination.

DISCUSSION

Anatomical knowledge amongst medical graduates has become an area of concern both in the UK and overseas²,¹¹ yet despite the significance of this issue, we did not find any previous studies that have attempted to rationalise the relevance and quantity of its learning after graduation. Its clinical usefulness extends beyond the “traditional” surgical specialties into anaesthesia, radiology, and other branches. EM serves as an excellent example of a specialty where knowledge requirements are broad, and the need to focus required knowledge important.

This study aimed to determine content. It did not address learning styles or suitable teaching tools. However, educational interventions of any type are useless without a clear basis of content, so this was a necessary first step in the much wider process of developing a comprehensive educational package for postgraduates.

Questionnaire methodology

The use of the Delphi technique in achieving consensus has been well described, and its applications range from industry to the military. Critics of the technique argue that, as initial content is determined by a lead investigator or small study panel, the entire process is biased toward that material selected by the investigator.¹² This is a credible argument when the technique is employed to determine future trends (“forecasting”).¹³ In this study, however, the technique was modified in that questionnaire content was based upon a well established textbook and was free from investigator bias. Furthermore, by incorporating free text areas on each round, we were able to respond to criticism and refine the content if needed.

The decision to utilise a four part Likert response scale was deliberate, in order to prevent items achieving a “neutral” score at the midpoint of an odd numbered rating scale. Employing an even numbered scale in this way is accepted practice in consensus techniques. Using a consensus cutoff of <75% agreement for items scoring ⩾2 was a decision based upon the need to restrict the required knowledge to those items considered core to safe clinical practice. Our aim was to develop a focused document containing key knowledge rather than one with additional items that, though not essential, might be “nice to know”.

Performing the questionnaire over three rounds was considered a suitable balance of allowing opinions to be refined while not overburdening respondents and therefore reducing response rates.

Choosing 40 participants per stream reflects the upper limit of participant numbers for previous Delphi studies.¹⁴ We deliberately sought to use large numbers to maximise the validity of the findings for such an important area of practice. Based upon an anticipated overall response rate of 65%, we calculated that using an initial cohort of 160 would allow collective opinion from 100 Fellows.

Participant selection

The use of relatively high numbers to maximise validity has been mentioned. We decided to use Fellows of the College as they could reasonably be assumed, having passed through the training structures of the specialty and its examinations, to possess suitable knowledge and experience to undertake the study. Some curriculum design studies have incorporated trainees into the decision making process.¹⁵ We elected not to do so on the basis that it is difficult to determine required knowledge when not having undertaken the specified length of training nor passed the specialty examinations.

Response rates

The overall response rates, from an initial cohort of 40 participants per region, ranged from 61% to 70% (table 1). We thought this a fair reflection of the effort required to sift and review up to 259 separate anatomical items and determine the utility of each. The range of response rates is small (9%), which suggests that levels of clinical experience across the four participant cohorts were broadly similar, and that the content and format of the questionnaire documents were equally acceptable within all arms of the overall project.

Content refinement

An average of 64% (range 57–73) of initial questionnaire content was retained in the final curricular document. The degree to which content was removed between each round, for any of the four anatomical regions, did not significantly vary. The inference from this is that up to one third of the contents of a “standard” anatomy textbook are of limited clinical relevance to clinicians in EM. This is not unreasonable; different readers require different types and amounts of knowledge. What we have demonstrated, however, is a method of determining what this knowledge actually is. The principles by which this was done could be reapplied to any other given domain of medical practice.

Implications of the study

The learning restrictions of working time legislation, coupled with increasing utilisation of shif based working across teams, and the effects of this upon supervision, place a significant educational burden upon postgraduates in all disciplines.¹⁶ There is simply less time in which to learn, and yet in many cases the required knowledge base remains undefined. By employing a large cohort Delphi approach we have been able to rigorously determine required knowledge in an important domain of practice. The principles involved are equally applicable to many other key aspects of clinical medicine.

The need to provide clear and robust curricula is a pressing one in the UK. The PMETB has mandated the development of such curricula as one of its key objectives. Using a consensus approach to national curriculum development is arguably the “gold standard” methodology.

Caution must be exercised, however, in the use of the methodology if the wider aims of the approach are not considered. Within EM, we are now extending the project to achieve national consensus documents in relation to required knowledge in pharmacology, microbiology, and statistics over a 3 year period. Involving separate cohorts for each aspect of this study in a relatively small specialty could realistically exhaust the pool of respondents over several years. For larger disciplines this is less of an issue.

There remains a crucial need to “future proof” curricula of this type, particularly in relation to more dynamic basic science content such as pharmacology and microbiology; arguably the only way to achieve this will be to subject established documents to further consensus review after a set period of time. This is a resource intensive yet necessary component of the process of dynamic curricular management and one the specialty will have to face.

Future work

Having established the utility of the consensus approach to curriculum development in anatomy, required pharmacology knowledge is being determined in 2006. Beyond this phase, our attention will turn to microbiology and physiology over the subsequent 2 years.

It is sensible to explore ways in which curricula can be internationally adopted to provide a common learning document. We are actively engaged in discussion with colleagues in Australasia, the Far East, and the West Indies to try to establish the consensus anatomy curriculum as an international training tool in relation to examinations requirements for parent colleges. We hope that future aspects of basic science curriculum development can be undertaken as Delphi projects between these and other interested countries.

Abbreviations

EM - emergency medicine

PMETB - Postgraduate Medical Education and Training Board