Determination of required pharmacological knowledge for clinical practice in emergency medicine using a modified Delphi technique

Abstract

Aims

To identify the content of the national postgraduate pharmacology curriculum for trainees in UK emergency medicine.

Methods

Modified three‐round iterated Delphi technique employing a participant panel of 160 randomly selected consultants in emergency medicine. Initial Delphi content was derived from objective analysis of pharmacy stock lists and patient record cards. The outcome measure was percentage agreement, among participants, to listed aspects of therapeutics as being core knowledge for autonomous clinical practice in the emergency department. A national curriculum document was derived from the study data.

Results

Response rates ranged from 66–76% after three Delphi rounds. From an initial overall questionnaire content of 526 discrete items, 59% was retained as core knowledge following iteration. The subsequent national curriculum document acts as a resource tool both for question setting in postgraduate examinations and for self‐directed trainee learning before the sitting of these examinations.

Conclusions

Application of a national consensus methodology allows determination of curricular content in pharmacology as part of the development of a robust College educational portfolio.

Clinical practice in emergency medicine involves pharmacology and therapeutics on a daily basis. Although the General Medical Council has de‐emphasised the role of the basic sciences as discrete entities,¹ there remains a need for trainees at even a relatively junior level to demonstrate that they know the principles of safe prescribing and understand the basis of common pharmacologically mediated clinical presentations. In the emergency department these clinical scenarios can unfold extremely rapidly.

Concerns have been expressed about the competencies which junior postgraduates possess in relation to the broader issue of applied basic science,²,³ and studies have suggested that medical students themselves feel least confident when it comes to their knowledge of pharmacology.⁴ It is difficult to determine whether such findings relate to a background of reformed undergraduate approaches to learning, although it is indisputably the fact that current educational trends lean away from factual acquisition and firmly toward the vertical integration of relatively ill‐defined knowledge with clinical experience.

The rapid pace of change at undergraduate level has not been reflected in the postgraduate educational environment,⁵ where service commitments and rationalised opportunities for senior supervision and teaching serve to limit opportunities for study. It can be difficult for trainees to prioritise their learning within the time available.

The Postgraduate Medical Education & Training Board (PMETB) introduced requirements in 2005 for parent colleges to develop clear and robust curricular documentation with which to inform their training and examination structures.⁶ Within this context, the content of training curricula should arguably reflect key or “core” knowledge so that time is not wasted in the study of material that is arguably more peripheral to everyday clinical practice.

Against this background, the College of Emergency Medicine has sought to develop learning tools for its postgraduates, based upon both the need both to determine a level of knowledge which should be expected of trainees and to provide clarity and rigour in the assessment and examinations process.

The methods historically used to guide curricular content have ranged from small expert group opinion to individual preferences. The use of the Delphi consensus technique has its origins in military strategic planning in the 1960s⁷ but has been described in a range of additional situations ever since that time. There are no published reports of its application in a national project to comprehensively determine the pharmacology curricular requirements for a medical speciality.

This paper describes the application of a structured national consensus methodology in order to determine the content of a core curriculum in pharmacology and therapeutics for postgraduate trainees in UK emergency medicine.

Methods and participants

Development of questionnaire content

Initial questionnaires were developed based upon collation of the content of emergency department pharmacy stock lists at 10 randomly selected UK hospitals, together with review of 50 randomly selected and anonymised emergency department attendance cards at one base hospital. Each attendance card was scrutinised to identify the drugs implicated either as part of the primary presentation, the management within the emergency department, or as merely coincidental aspects of the presentation.

Using these techniques generated an initial list of 526 therapeutic agents. To produce manageable questionnaire documents the list was broken down based upon relevant chapter headings in the British National Formulary, chosen for its widespread familiarity to respondents.

Four separate initial questionnaires were then each administered with an average of 130 agents for consideration on each.

Within each initial document, the agents were listed alongside a four‐part Likert preference scale. Respondents were asked to express their opinion on the relevance of knowledge of each pharmacological item from “1” (an essential item of knowledge for practice) to “4” (no knowledge required for practice).

It was made clear to respondents that “required knowledge” of a given item was taken to be an appreciation of its indications, actions at the tissue level, contraindications, side effects and common interactions.

Spaces for free text response were provided within the body of each questionnaire document to allow respondents to comment upon aspects of the content if they wished to do so.

Selection of participants

The database of Fellows of the College of Emergency Medicine (2006 edition) was entered into an anonymised spreadsheet format. Possession of the College Fellowship is a prerequisite of consultant status. For each of the four questionnaire streams, 40 participants were randomly selected. Those then found to be working overseas, retired, or engaged primarily in non‐clinical roles were excluded and a new selection made to replace them. Letters of invitation to the study were sent to the selected participants, followed up by the initial questionnaire mailing within 1 month.

Questionnaire rounds

As completed questionnaires were returned, they were collated and assessed. For each pharmacological item, cutoff for progression into the second round was set at <75% consensus of that item scoring 2 or better in the initial round. Free text comments were noted and acted upon where feasible. Following a second round of questionnaires with modified content, further analysis was undertaken using similar principles to again eliminate those items scoring relatively poorly. The final questionnaire displayed each remaining item alongside its modal (most commonly occurring) rank position, and allowed the respondents one final chance to alter their opinion of each item's perceived importance.

Completion of the three‐round Delphi process generated four discrete consensus documents. These were incorporated into a single composite. The composite curricular document went forwards for ratification by the College Education & Examinations Committee before its adoption as the national curriculum for applied pharmacology in emergency medicine.

Results

In total, 160 initial questionnaires were sent out to participants.

Analysis of the Likert preference scores for items in each round enabled the questionnaires to be sequentially refined.

The composite curricular document comprised 310 discrete pharmacological facts reflecting consensus opinion on the importance of key agents and their safe prescribing.

To facilitate learning, the curricular document was reformatted into a 23 page web training tool based upon BNF chapter headings and can be found at http://www.emergencymed.org.uk.

The pharmacology aspects of College examinations were subsequently reviewed. Multiple choice questions are now based upon the consensus document, such that knowledge of the detail of the document will be reflected in success in the pharmacology component of the speciality examination.

Discussion

Pharmacological knowledge among medical graduates continues to attract comment and concern.⁸,⁹ Despite several studies scrutinising the impact of reformed undergraduate teaching styles, there is relatively little literature addressing the issues of effective postgraduate learning in this key basic science. The emergency department places heavy emphasis on pharmacological knowledge—patient presentations are commonly primarily or secondarily drug related—and a small but clinically significant set of drugs are routinely prescribed as part of management, often by relatively junior staff. A separate community of therapeutics is often implicated in presentations of cardiac, respiratory, endocrine and toxic derivation.

In the context of a significantly altered undergraduate and, latterly, postgraduate medical educational culture in the UK, the focus of effective learning towards a firm grounding in the most basic principles of therapeutics, the indications and side‐effects of common agents, and competence in safe prescribing can easily be lost. We wanted to produce a simple, usable yet very rigorously derived curricular document to facilitate key knowledge acquisition.

Questionnaire methodology

The use of Delphi techniques to determine pharmacological curricular content has been described previously. Critics of the methodology argue that the entire process is biased in that collective opinion is only invited upon material selected by the investigator as constituting the initial dataset. Our study based its content upon objective data derived from the stock lists of large hospitals' emergency departments and actual patient records in order to eliminate any such investigator bias. We had, in any case, no inherent interest in seeking to introduce such bias. By incorporating free text areas within the questionnaires, we were able to respond to criticism and refine the content if needed. No such criticisms were articulated.

This was an unusually large study in terms of the participant pool. Many reported applications of Delphi involve only a few dozen respondents at most. We utilised such a large panel very specifically to address both the breadth of the initial dataset (it would be impractical to employ a few panel members to handle such a large body of data) and the importance of the project in shaping national curricular policy. In approaching 160 panellists we aimed to maximise the validity of our findings. Based upon an anticipated overall response rate of 65%, we calculated that using an initial cohort of 160 respondents would allow collective opinion from 100 Fellows.

The use of a specifically four‐part Likert scale was a deliberate decision, taken to prevent items achieving a “neutral” score at the midpoint of an odd numbered rating scale. Employing an even numbered scale in this way is accepted practice in consensus techniques. Using a consensus cutoff of <75% agreement reflected a need to restrict required knowledge very rigorously to those items considered core to safe clinical practice. Our aim was to develop a very focused document containing key facts rather than one with additional items that, though not so important, might be “nice to know”.

Performing the Delphi study over three rounds was considered a suitable balance of allowing opinions to be refined while not overburdening respondents and potentially reducing response rates.

Participant selection

We decided to use Fellows of the College of Emergency Medicine as our respondent pool based upon the fact that, having passed through the training structures of the college and its examinations, they could be reasonably assumed to possess suitable knowledge and experience to give meaningful opinions in the study. Some curriculum design studies have incorporated trainees into the process. We elected not to do so on the basis that it is difficult to determine required knowledge without having completed the required training or passed the speciality examinations.

Response rates

The overall response rates (66–76% depending on questionnaire stream) seem a fair reflection of the effort required to sift and review over 100 aspects of pharmacology and pass a judgement on the relative worth of each. The relatively narrow range of response rates (10%) suggests that levels of clinical experience across the four participant cohorts were broadly similar, and that the content and format of the questionnaire documents were equally acceptable within all arms of the study.

What this paper adds

What is already known about this subject

Pharmacology is a fundamental aspect of clinical practice in emergency medicine and is assessed in the MCEM examination. Reforms at medical undergraduate level have placed less emphasis on factual learning. Developing models of postgraduate working allow restricted time for study. There is a need to establish the scope of required pharmacological knowledge as part of a national core curriculum strategy for emergency medicine in order to facilitate effective learning.

What this study adds

By using a structured consensus methodology applied to national expert panels, the breadth and depth of a core postgraduate pharmacology curriculum can be determined. This national learning tool for trainees in the speciality informs the composition of the examination and promotes the concept of learning tailored to key elements of safe and effective prescribing.

Implications of the study

The need to provide clear and robust curricula has been mandated by PMETB as an essential element of College educational portfolios. In real terms, however, the impact of working time legislation, shift‐based working and reduced opportunities for dedicated study at postgraduate level all contribute to an unsettling situation where, for adult learners, there is simply less time in which to learn. Despite these challenges, the required pharmacological knowledge base with which to practise safe clinical emergency medicine has hitherto remained undefined.

Our desire to bring clarity to the problem of required pharmacological knowledge stemmed from the concept of the “student formulary”⁹—mitigating the very real problem of learning about an ever increasing number of prescription drugs by developing learning priorities based upon core, common and/or important agents. Experience gained in the earlier development of an anatomy curriculum¹⁰ led us to appreciate that, despite their graduate status, many adult learners in our speciality demonstrated a poor grasp of relatively elementary facts in their junior posts. The fact that the our national curriculum now includes such items as the indications, dosage and side‐effects of paracetamol may seem surprisingly rudimentary to some, but reflects the changing landscape of demonstrated levels of competence which are to be found in the training years of postgraduate clinical medicine.

This study developed curricular content. It did not address the challenge of suitable learning styles by which to acquire core knowledge. Speciality training programmes, which have attracted so much recent controversy in their initial implementation, will have to find ways of embedding a learning culture within their framework. The need to devote time to the acquisition of factual knowledge remains as crucial today as it did in the relatively recent era of preclinical undergraduate teaching.

A pharmacology curriculum is of necessity a dynamic one. Having established a core postgraduate curriculum, we are mindful of the need for regular review. Having the document as a web‐based tool allows us to review its content easily. Where new agents appear to the market, the Basic Sciences Research Group weighs the need to subject the agent to immediate scrutiny for inclusion. The broader strategy for maintaining currency of content, however, is to re‐administer the Delphi on a 3 yearly basis based on abbreviated content of those new agents which have appeared in the time since the original study.

Future work

We have demonstrated the utility of a consensus methodology involving a large respondent panel to determine national curricular policy. During 2007 we are examining ways in which the core pharmacology curriculum can be administered in a more interactive web format,¹¹ mapped to learning outcomes as part of workplace‐based assessment, in order to allow trainees to both acquire and demonstrate knowledge acquisition in daily practice.