Construction and evaluation of a web-based interactive prescribing curriculum for senior medical students

Abstract

Aims

To develop and evaluate for the National Prescribing Service (NPS) a web-based interactive prescribing curriculum for Australian senior medical students based on the World Health Organization’s Guide to Good Prescribing.

Methods

Teachers of prescribing from all Australian medical schools in 2000 wrote 12 case-based modules which were converted to on-line format. Objective evidence was provided for selecting first-line medicines from available alternatives by comparing efficacy, safety, convenience and cost. The curriculum was made available to final year students in 2001 and was evaluated by measuring use from web statistics and by semistructured interviews with 15 teachers (2003) and on-line surveys of 363 students over 2003 and 2004.

Results

By 2004 the curriculum was used by nine of 11 possible medical schools. Uptake increased each year from 2001 and all 12 modules were accessed consistently. Student access was significantly (P < 0.001) greater when prescribing was an assessable part of their course. Teachers’ evaluations were uniformly supportive and the curriculum is seen as a valuable resource. Student responses came from a small proportion of those with password access but were also supportive. Over half of student respondents had created their own evidence-based formulary.

Conclusions

A collaborative venture initiated by the NPS with Australian medical schools has been successfully implemented in most courses. Teachers find the resource of high quality. Student respondents find the curriculum valuable in developing their own prescribing skills. It is best delivered by self-directed study followed by tutorial discussion of prescribing decisions.

Introduction

Medical school curricula dedicate time to the teaching of pharmacology and clinical pharmacology but prescribing often receives scant attention and commonly only the legal and regulatory aspects are taught – almost as though the translation of pharmacological knowledge into an appropriate prescription for a patient were an automatic process requiring no additional skills. However, when medical interns have been asked about their most feared tasks at the commencement of clinical practice, aspects of prescribing loom large and, particularly, the fear of harming a patient by a poor prescribing decision [1].

In 1997, the then Minister for Health and Family Services of the Commonwealth of Australia announced the creation and funding of a National Prescribing Service (NPS) to implement the national Quality Use of Medicines programme, the fourth arm of the National Medicines Policy [2, 3]. The committee charged with setting up this service conducted a nationwide survey of all those with a stake in the venture with a request to identify the priority areas for the NPS. One of the highest priorities was given to improving the quality of prescribing education, with particularly strong views coming from junior doctors, especially those training for general practice.

As a consequence, a curriculum committee of NPS was set up with the remit of bringing together a national programme to improve the teaching of prescribing in the (at that time) 10 national medical schools which might flow on to work on intern, other junior medical officer and, ultimately, clinical specialist training. This paper reports our experience in working through this task for medical school curricula and in evaluating the outcome.

Methods

At the outset it was agreed that a partnership with medical school teachers would be essential. The Australasian Society for Clinical and Experimental Pharmacologists and Toxicologists (ASCEPT) is the national professional society for clinical pharmacologists and the partnership concept was readily accepted by the Society.

As we were proposing a national curriculum, it became important to have representation from all the medical schools on the initial curriculum committee. Several regional and, later, national meetings were held to determine the overall plan for the curriculum. The annual ASCEPT scientific meeting proved an invaluable forum for such meetings. It was agreed that the programme should be designed for senior medical students towards the end of their clinical training and be delivered in a flexible way with a preference for a web-based interactive programme suitable for individual or group study, an approach supported in the literature [4, 5]. It should be based on the therapeutic topics that interns identified as most important to them in their first year of clinical practice. It should be capable of providing reports back to tutors and evaluators on usage by a medical school or by an individual student.

As it had been evaluated formally [6], it was agreed to adopt the overall framework of the World Health Organization (WHO) Guide to Good Prescribing (GGP) [7], which had been available for 4 years at that time, and to adapt this to local circumstances.

Defining the topics

Two groups of interns from two medical schools, who were close to the end of their first year of practice, were asked to suggest those topics which were most relevant to the intern year and in which they would have liked more competence as they progressed through their own first year as doctors. A set of 12 were agreed to for the first version of the curriculum. These were chronic obstructive pulmonary disease (COPD), peptic ulcer, hypertension, the confused patient, seizures, acute chest pain, otitis media in a child (in order to bring in at least one reference to children and to raise issues about use of antibiotics and analgesics in this condition), heart failure, anticoagulation, postoperative pain and vomiting, polypharmacy and an intern orientation module which included aspects of prescribing behaviour, hospital protocols and regulations. Conveniently, there were sufficient topics to enable at least one to be allocated to teachers from each medical school.

Constructing the modules

The NPS curriculum committee, comprising clinical pharmacologists involved in student education, provided overall direction for construction of the modules.

Teachers were paid to develop their modules as paper problems using the headings of the GGP, which include:

• defining the problem from the clinical vignette provided

• specifying therapeutic objectives

• assessing the place for nonpharmacological options

• selecting first-line medicines from an inventory of available products on the basis of comparative efficacy, safety, convenience (often related to the kinetic properties of the particular medicine) and cost.

Each module also incorporated the writing of an on-screen prescription, the provision of information relevant to this and consideration of follow-up arrangements and monitoring the therapy.

References were incorporated into each module, wherever possible available on-line through a public domain website.

General specifications for the on-line interactive format

NPS called for tenders for the conversion of the paper-based modules into interactive modules suitable for use on individual or classroom computers. The specifications were for an intuitive, self-directed, problem-based on-line learning environment with links to Australian as well as other reputable sources of objective information about medicines.

Australian sources granted us permission to include selected passages from the Australian Medicines Handbook (formulary) and Therapeutic Guidelines. Other sources included Australian Prescriber and the regularly updated Pharmaceutical Benefits Scheme listings of government-subsidized medicines with their costs.

The facility to create a personal, evidence-based, formulary for individual users was also needed, so that students could build their selections of preferred medicines (‘P-drugs’ to use the WHO term). In a later development the personal formulary may now be downloaded into a MS Windows CE^TM-powered portable, hand-held device and be available without reference to a desk-top machine.

A system of reporting was needed to provide the NPS with summary usage statistics for each medical school with details of activity by student, teaching module and site. A South Australian-based consortium comprising the University of Adelaide, Flinders University, the South Australian Centre for Rural and Remote Health and Knowledge South Pty Ltd (a subsidiary of the University of South Australia) won the bid. They converted the paper-based modules to an on-line format.

Security

Security was designed to provide for a hierarchy of levels from students to authors to developers. The site enables students and tutors to access all learning functions, authors to access modules under development, curriculum committee members to access all modules and the project team to access the whole curriculum and its infrastructure. Random number-generated passwords are provided by the developers to the NPS, which allocates them to participating medical schools to protect student privacy.

Visitors to the site (http://nps.unisa.edu.au) have free access to the COPD module, in which they may execute all the functions up to adding a drug to the formulary and prescribing. Mock-ups of the latter processes are provided.

Pilot testing

Two of the modules were pilot-tested by a group of 25 medical students in one medical school. Students completed the modules on-line and provided feedback about navigating around the modules as well as comments on content and its appropriateness. Resulting modifications were made before the programme was made generally available, free of charge, to all Australian medical schools in 2001.

Evaluation methods

Process

This measured implementation of the curriculum within each of the medical schools monitored by website statistics using a database maintained by NPS. Each password that is created has a profile which identifies each user. This enabled actual use of the curriculum within each medical school to be monitored.

Impact

This measured general perceptions of the value of the curriculum by students and teachers. Feedback from teaching staff has been collected through a series of semistructured interviews with 15 staff from nine of the medical schools in 2003, while feedback from students in 2003 and 2004 has been obtained through two on-line surveys using five-point rating scales for response to the questions. (Copies of the on-line surveys are available from the authors or the Editorial Office of the Journal.)

Results

Implementation and uptake of the curriculum

Between January 2001 and the end of 2004, a total of 5811 passwords had been issued to medical students. The medical school which had not introduced the curriculum was a new school where the first intake of students arrive at their final year in 2005.

Sixty-six percent of students were in their final year. According to national medical workforce surveys [8], which provide the total number of students completing medical school each calendar year, it is estimated that approximately two-thirds of all Australian final year medical students have been provided with access to the curriculum.

Figure 1 (medical schools de-identified) shows the number of medical students that accessed the curriculum in each school between 2001 and 2004. Use of the curriculum was limited to just one school in 2001: uptake was widespread throughout the majority of schools from 2002 onwards. The exception was medical school 5, which implemented the curriculum in its small rural clinical school in 2004. The number of student passwords provided to each medical school remained steady but the absolute number of students accessing the curriculum increased each year overall.

Figure 1.

Medical student use of prescribing curriculum (total number that visit curriculum website in each year). 2001 (), 2002 (), 2003 (), 2004 (▪)

Figure 2 shows the number of students accessing the curriculum, as a proportion of all students provided with passwords in each medical school (to allow for the differing sizes of medical schools in Australia) between 2002 and 2004.

Figure 2.

Medical student use of prescribing curriculum, 2002–2004 (proportion of students provided with passwords who visited the curriculum website). 2002 (), 2003 (), 2004 (▪)

Figure 3 documents the extent of use at each school showing the total number of hits to the curriculum website. The larger medical schools (4, 6, 7 and 9) had the greatest number of hits but the data also demonstrate the overall growth of curriculum use since its inception. Total hits refers to hits to all files, and therefore a single user session may generate multiple hits.

Figure 3.

Total ‘hits’ to website at each medical school 2002–2004. 2002 (), 2003 (), 2004 (▪)

All topics were accessed by students and there was no particular preference apart from higher use of the COPD module, which was commonly used by tutors as an introductory demonstration for the whole programme. There was increasing use of all modules over time.

Feedback from teachers and students

The semistructured interviews conducted in 2003 obtained feedback from 15 members of teaching staff representing nine medical schools. Teaching staff from the two medical schools that had not implemented the curriculum at that time were also approached but did not participate. All respondents indicated that the curriculum was an extremely valuable resource. Each school reported introducing the curriculum as a tool for individual, self-directed learning. The majority instructed students to work through each of the 12 modules. All teachers indicated that some form of feedback to students was essential to maximize the value of the curriculum and six medical schools reported that they had extended the use of the curriculum into the small group setting to enable this to occur. Teachers from the remaining medical schools were all interested in learning from their peers as to how the curriculum could be integrated in this way.

Teachers from six medical schools (four of which had incorporated small group learning) reported they assessed students’ learning in prescribing through case-based examination questions. The proportion of students that accessed the curriculum from schools which assessed learning (see Figure 2, Medical Schools 4, 6, 7, 8, 9 and 11) was 34% higher (95% confidence interval 30, 39) than of those that did not ( = 207; P < 0.001).

The most significant barrier to implementing the curriculum was perceived to be effectively engaging other staff members to use it. In most schools, the original author was responsible for championing use of the curriculum. However, teachers reported that when authors moved on to other positions or schools, this influential role in encouraging and maintaining uptake might be lost.

The on-line student surveys conducted in 2003 (n = 129 representing five medical schools) and 2004 (n = 234 representing 10 medical schools) yielded a response rate of 6% and 13%, respectively. Of respondents, 97% (in 2003) and 91% (in 2004) said they had used the curriculum.

The majority of users (91% in 2003 and 92% in 2004) thought that the therapeutic content within each module was appropriate to their needs and, when asked if they felt better equipped to prescribe having used the curriculum, 86% of respondents in 2003 and 78% in 2004 either agreed or strongly agreed that this was the case.

All respondents who had used the curriculum were asked if they had developed their own personal formulary and just over half (53% in 2003 and 56% in 2004) had done so. Respondents were then asked to indicate whether the curriculum had been of assistance in developing their personal formulary. The majority of respondents either strongly agreed or agreed with this statement (93% in 2003 and 89% in 2004).

Qualitative feedback from both surveys suggested the curriculum could be improved by broadening its scope, providing greater complexity within particular modules and expanding the number of modules available. Many students commented on the need for feedback on their prescribing decisions, thus emphasizing the need for the curriculum to be extended to the small group/tutorial setting.

Discussion

The size of personal formulary, written or remembered, from which an experienced doctor commonly prescribes seldom exceeds 70 medicines, although ‘competent familiarity’ may be needed for up to 100 more for less common conditions [9, 10]. If a medical student has worked through a prescribing programme such as the one we describe here, they are already more than half way to owning a personal formulary. This can be modified over time but, as it has been selected with a sound evidence-base using defensible processes, the method will still remain valid when many of the initial medicines selected are obsolete and ready for deletion or replacement.

The WHO Guide to Good Prescribing programme was first evaluated in a seven-medical school randomized controlled trial [6] and shown to be able to inculcate an evidence-based approach to prescribing in senior medical students from both developed and developing countries. The endpoints of this study were all based on ‘intention to prescribe’ and actual prescribing was not a feasible outcome.

In another randomized trial, the approach has been shown to be applicable to the training of other health professionals and to make actual prescribing more rational [11]. We felt that this evidence supported the wider introduction of the programme into the education of senior medical students. Flexible delivery of educational material is needed when students may be based either in a city or in rural placements. Most have access to the internet either through personal computers or through the hospitals in which they work and we opted for a web-based programme which will run on most systems and does not require special computer hardware.

The free availability of an on-line curriculum does not mean that all medical schools will use it. NPS had no mandate to enforce change in undergraduate medical curricula. While a marketing exercise informing medical schools of the impending availability of the programme and asking for support in facilitating its entry into their curriculum was undertaken at the outset, local ‘champions’ were responsible for much of the uptake. However, even they have had problems in influencing a curriculum or year committee to incorporate this programme, particularly when other courses and programmes are waiting to be adopted into an already overcrowded timetable.

A recent challenge is the creation of four new Australian medical schools and our wish to see them recognize the importance of instruction in prescribing in their new curricula. Active ‘marketing’ is clearly needed if the reach of the programme is to be as wide as possible.

Students will study what they know is to be assessed and there was little surprise in our finding of the higher utilization of the prescribing programme in schools where this was the case.

Evaluation was based on web statistics together with student surveys with a voluntary response of small percentages (6% and 13%), a total of 363 students in all. The relatively low response rate may be due to the lack of incentives provided to students, combined with the timing of the survey, which was at the end of the academic year, when many students are undertaking clinical placement. While the survey responses are adequate for the conclusions drawn, it is possible that the silent majority hold differing views on the usefulness of the programme. However, we are reassured that the curriculum is of value by the increasing uptake of use with time and the strong support it enjoys from tutors. This feedback may not be fully representative, as teaching staff from the two medical schools that had not implemented the curriculum did not participate.

In contrast to some other topics, prescribing choices and the information on which they are based are changing continuously and we have already needed to revise the modules on two occasions, especially the links to objective sources of information such as the formulary or therapeutic guidelines, to keep pace with these changes. This considerable task has been overseen by one of the module authors (Dr S. Shakib, University of Adelaide), who is familiar with the total programme.

Prescribing becomes a reality when young doctors are first obliged to make their own decisions during the intern/house officer year. Australian research has shown that the choices and decisions that they make unsupported are many fewer than the prescriptions they write under instruction from more senior colleagues [12]. Nevertheless, the preparation provided by the prescribing curriculum should help alleviate the stress of the first year of clinical practice and, ideally, should be in continuity with a similar, more sophisticated programme for junior doctors. NPS is developing such a programme based on a limited number of full modules of a similar format to those for students, but at a more advanced level. The programme also includes shorter modules suitable for taking to the bedside and working though with a more senior colleague as tutor. The challenge here will undoubtedly be the unstructured life of the average intern and simply finding the time to work through these resources, even though the concept is strongly supported by the Confederation of Postgraduate Medical Education Councils in Australia.

We have not measured actual prescribing, as this could not be done in a student population. Once they have graduated, students become hospital interns and immediately come under the influence of the hospital environment, although in Australia they are protected to some extent in their first year as prescribers from advertising by the pharmaceutical industry. The need to conform to peer and senior’s behaviour is very strong in this most vulnerable year and the adoption of imitative behaviour is a common survival mechanism. An intern’s evidence-based approach to the selection and use of common medicines may occasionally conflict with the choices of more senior hospital staff, but a prescribing programme specifically for the intern, now under development, may help reinforce appropriate prescribing behaviour.

While it is a difficult task, there is clearly a need to develop methods to track the prescribing of junior doctors who have been exposed, or not, to this particular form of training as senior students and establish whether the influence of the student prescribing curriculum can survive the rigours of the intern period. We hope to be able to report further on the impact of this student programme, and of the pending intern curriculum, in the future.

In conclusion, this curriculum, put together by a representative team from all the medical schools in Australia, has been implemented progressively over a 4-year period. It is enthusiastically supported by tutors and by the majority of students who responded to a voluntary evaluation. While it has primarily been used as a self-study programme, the preferred mode of use appears to be in combination with small group tutorials which provide feedback to students on their personal prescribing decisions.