Abstract
Objectives
To identify the tasks that should constitute the work of preregistration house officers to provide the basis for the development of a self evaluation instrument.
Design
Literature review and modified Delphi technique.
Setting
Northern Deanery within the Northern and Yorkshire office NHS executive.
Subjects
67 educational supervisors of preregistration house officers.
Main outcome measures
Percentage of agreement by educational supervisors to tasks identified from the literature.
Results
Over 61% of communication items, 70% of on call patient care items, 75% of routine patient care items, 45% of practical procedure items, and over 63% of self management items achieved over 95% agreement that they should be part of the house job of preregistration house officers. Poor agreement was found for the laboratory and clinical investigations that house officers could perform with or without supervision.
Conclusions
The tasks of house officers were identified but issues in using this method and in devising a universally acceptable list of tasks for preregistration house officers were apparent.
Key messages
More than 100 activities were identified as potential tasks for house officers, and 11 personal abilities were identified as self management skills
The ability of preregistration house officers to perform all of the tasks independently would be restricted by their experiences and therefore may depend on the specialty in which they work
The deliberation over what are and are not “shared tasks” was evident; some educational supervisors wanted the house officer to be capable of, but not practise, some tasks whereas others did not believe these tasks were within the remit of the house officer
The Delphi technique is a useful method for gaining the autonomous opinions of individuals from a large group of geographically distant members
Introduction
The year’s post of house officer is being recognised as a critical transitional period.1,2 The content of the house job is, however, difficult to analyse because of its complexity and constant development. We aimed to identify the tasks of preregistration house officers and to devise a method on the basis of these results for following future changes.
The range of work performed by house officers has been investigated by various methodologies including interviews,3 direct observation,4 diary keeping,5 postal questionnaire,6,7 and multiple data sources.8 However, there remains no complete guide to the tasks that should make up a house job. Previous studies have focused principally on what house officers do and not on what they should do. Identifying what doctors should do during their house jobs was necessary, as our study was the first stage in developing a self evaluation instrument for house officers to be used to support the planning of a personal education strategy.
The Delphi technique is a consensus method used to determine the extent of agreement on an issue. The technique involves asking a panel of experts—in this instance educational supervisors—to take part in a series of rounds to identify, clarify, refine, and finally to gain consensus on the particular issue.9–11 As the panel do not meet, individuals can express their opinion without being influenced by others. To reduce the number of rounds in our study, the tasks were generated from the literature rather than from an initial round of the Delphi technique.
Participants and methods
The panel
Our panel was derived by a two stage process. Firstly, we identified educational supervisors eligible for inclusion in the study, then we identified those within the eligible group who were willing to take part.
We asked all 18 clinical tutors whose NHS hospital trusts were responsible for the training and employment of preregistration house officers in the Northern Deanery to propose educational supervisors for the panel. The clinical tutors were asked to include those who had at least 2 years’ experience of supervising preregistration house officers, and who were considered to have particular insight into the educational as well as the service function of the preregistration year. We then invited eligible educational supervisors to become part of the panel.
A list of tasks was identified from the literature3–5,7,12–18 and collated under the section headings: 1, communications; 2, on call patient care; 3, routine patient care; 4, laboratory investigations; 5, clinical investigations; 6, practical procedures; and 7, self management. We generated a datasheet of operational definitions, categories of tasks, and space for comments, and we posted this to members of the panel.
Task appraisal
The educational supervisors were asked to accept, reject, or question the inclusion of each task. They were invited to modify the statements and to add new tasks. Judgments were made on the basis of whether house officers would be able to perform the task by the end of their preregistration year. Tasks were defined as “any activity carried out by a preregistration house officer and deemed to be appropriate for that grade.”19 The panel were asked to include those tasks that are the “essence” of the educational experience of being a preregistration house officer, and those tasks that should be carried out by preregistration house officers and not tasks that are performed by them because there is no one else available to do them. The educational supervisors were also asked to include those tasks that might not be performed routinely by a house officer, but which the house officers might be called on to do.
Statements that gained over 95% agreement in the first round (round 1) were deemed accepted and were not resubmitted in the second round (round 2). The remaining task statements from round 1 were modified in line with the comments of the educational supervisors. When several suggestions for one task were given, we used the most commonly suggested modification.
In round 2, the modified task statements were resubmitted to the educational supervisors along with all additional tasks suggested by individual consultants. We also included with this a summary of the results of round 1.
The educational supervisors were asked to return the completed datasheets within 4 weeks. We coded the responses and analysed them by frequency of response with SPSS for windows (version 6.0). The Delphi technique was conducted between April and June 1997. Our results therefore reflect the jobs of house officers as they existed at that time.
Results
Of the 113 educational supervisors (68 physicians and 45 surgeons) proposed by the clinical tutors, 10 (9%) refused to take part and seven (6%) did not respond. Of the remaining 96 (85%), 75 agreed to take part, and 21 asked to see the work before deciding. Overall, we sent out 96 forms (60 physicians and 36 surgeons) in round 1. Two consultants withdrew and so we sent out 94 forms (59 physicians and 35 surgeons) in round 2.
Overall, 74 forms were returned in round 1 of which 64 (67%) were processed (42 physicians, 22 surgeons), and 72 forms were returned in round 2 of which 67 (71%) were processed (45 physicians, 22 surgeons). We did not process datasheets returned after 4 weeks. Forty physicians and 18 surgeons replied to both rounds.
Teaching versus non-teaching hospitals
Analysis of non-respondents in both rounds by employment showed no significant difference between teaching hospitals and non-teaching hospitals (6 of 20 (30%) v 26 of 93 (28%) respectively). Eighteen specialties were represented on the panel.
Tables 1–6 show the task statements from round 2 and those that achieved over 95% agreement in round 1. The items identified from the literature under section 7 (self management) were skills and not tasks. However, as the data from this section were dealt with by the same procedure as the others, we included them here (table 7).
Table 1.
Section 1: communication tasks. Values are numbers (percentages) of panel accepting statement
| No | Task | Panel response |
|---|---|---|
| 1.1 | Establishing and maintaining good working relationship with other staff | 64 (100)* |
| 1.2 | Liaising with senior doctors | 63 (98)* |
| 1.3 | Liaising with nurses | 64 (100)* |
| 1.4 | Liaising between staff off and on ward, for example, diagnostic departments | 64 (100)* |
| 1.5 | Communicating information between hospital and community, for example, general practice | 67 (100)† |
| 1.6 | In consultation with senior doctor, disclosing information about patients to appropriate authorities | 64 (96)† |
| 1.7 | Talking to patients, explaining to patients, liaising with patients, informing patients | 62 (97)* |
| 1.8 | At discretion of senior doctor, breaking bad news to patients | 58 (87)† |
| 1.9 | Talking to relatives, explaining to relatives, liaising with relatives, informing relatives | 62 (97)* |
| 1.10 | At discretion of senior doctor, breaking bad news to relatives | 62 (93)† |
| 1.11 | Giving advice on individual patient care to other non-medical professionals, for example, physiotherapists | 58 (87)† |
| 1.12 | Gaining informed consent for minor frequently executed procedures with which preregistration house officer is familiar, for example, chest drains | 67 (100)† |
| 1.13 | In consultation with senior colleague, gaining consent for postmortem examination | 63 (94)† |
| 1.14 | Case presentation on ward rounds | 67 (100)† |
| 1.15 | Performing effective “hand over,” for example, between doctors on shifts | 67 (100)† |
| 1.16 | In consultation with senior doctor, making referral to coroner | 65 (97)† |
| 1.17 | Completing death certification | 65 (97)† |
| 1.18 | Giving health promotion advice to patients | 58 (87)† |
| 1.19 | Handling difficult patient interactions, for example, self discharge, complaints | 40 (60)† |
| 1.20 | Communicating with management and administration | 42 (63)† |
| 1.21 | Reporting adverse drug reactions | 62 (93)† |
Task accepted in round 1 (64 responders).
Task accepted in round 2 (67 responders).
Table 6.
Section 6: practical procedures. Values are numbers (percentages) of panel accepting statement
| No | Task | Panel response | ||||
|---|---|---|---|---|---|---|
| 6.1 | Bladder catheterisation (in presence of chaperone): | |||||
| a. Male | 67 (100)† | |||||
| b. Female | 62 (93)† | |||||
| 6.2 | Inserting venflon | 62 (97)* | ||||
| 6.3 | Gaining arterial access for blood samples | 63 (94)† | ||||
| 6.4 | Simple skin sutures | 65 (97)† | ||||
| 6.5 | Injecting: | |||||
| a. Subcutaneously | 61 (95)* | |||||
| b. Intramuscularly | 66 (99)† | |||||
| c. Intra-articularly | 20 (30)† | |||||
| 6.6 | Inserting chest drains under supervision | 64 (96)† | ||||
| 6.7 | Insertion of fine bore feeding nasogastric tube and checking its position | 61 (91)† | ||||
| 6.8 | Mixing intravenous drugs | 66 (99)† | ||||
| 6.9 | Assembling: | |||||
| a. Pumps | 54 (81)† | |||||
| b. Intravenous infusions | 62 (93)† | |||||
| 6.10 | Administering intravenous drugs | 66 (99)† | ||||
| 6.11 | Assisting more senior staff with procedures unfamiliar to preregistration house officer | 64 (96)† | ||||
| 6.12 | Wound management: | |||||
| a. Opening infected wounds | 33 (49)† | |||||
| b. Aspirating haematomas | 40 (60)† | |||||
| c. Removing sutures | 61 (91)† | |||||
| 6.13 | Subcutaneous infusions | 62 (93)† | ||||
| Unsupervised | Supervised | Rejected | ||||
| 6.14 | Central line insertion | 2 (3) | 38 (57) | 23 (34) | ||
Tasks accepted in round 1 (64 responders).
Tasks accepted in round 2 (67 responders).
Table 7.
Section 7: self management skills. Values are numbers (percentages) of panel accepting statement
| No | Task | Panel response*
|
|
|---|---|---|---|
| Developed | Developing | ||
| 7.1 | Efficient use of time or time management | 64 (100)† | — |
| 7.3 | Knowing when it is necessary to contact more senior doctor for help | 64 (100)† | — |
| 7.4 | Understanding roles of others | 64 (100)† | — |
| 7.2 | Efficient use of resources | — | 66 (99)‡ |
| 7.5 | Teaching others, for example, medical students | — | 64 (96)‡ |
| 7.6 | Managing their own education | — | 66 (99)‡ |
| 7.7 | To deal with death, dying, and emotionally distressing events | — | 66 (99)‡ |
| 7.8 | Identifying priorities, for example, when on call knowing who to attend to first and what to do first | 26 (39) | 34 (51) |
| 7.9 | Clear knowledge of “one’s own competencies” | 20 (30) | 39 (58) |
| 7.10 | Working as part of multidisciplinary ward team | 20 (30) | 40 (60) |
| 7.11 | Understanding his or her own role and responsibilities as doctor and employee | 15 (22) | 44 (66) |
Those who accepted skill but failed to indicate developed or developing not included.
Skills accepted in round 1 (64 responders).
Skills accepted in round 2 (67 responders).
Round 2
As round 1 of the Delphi technique was concerned principally with refinement of the task statements, we focus on the data from round 2.
Section 1
In section 1 (communications), 13 (62%) statements achieved over 95% acceptance, and 16 (76%) achieved over 90% acceptance. Suggestions were given on how to alter three of the five tasks (1.18, 1.19, 1.20) that achieved less than 90% acceptance (table 1).
Comments indicated that task 1.18 would have gained more acceptance if presented as “Giving simple health promotion advice to patients.” Comments on task 1.19 suggested that handling complaints, other than in limited instances, was not the duty of the house officer. No clear guidance for change was given for task 1.20 except that it needed to be made more “explicit.”
Section 2
Section 2 (on call patient care) generated few comments from consultants, with over 95% acceptance for 7 (70%) task statements and over 90% acceptance for all but task 2.3b. One comment on task 2.3b was that it was not a house officer task.
Section 3
In section 3 (routine patient care), 15 (75%) task items achieved over 95% acceptance with only three (15%) items under 90% acceptance. Consultants’ comments suggested that task 3.13 should be limited to hand written discharge letters and therefore supported task 3.10b “Completing hand-written discharge forms,” which was accepted in round 1. Comments suggested that altering task 3.15 to “Create a provisional problem list and management plan” might have made this task statement more acceptable.
Sections 4 and 5
For sections 4 and 5 (laboratory and clinical investigations respectively) venous blood sampling, electrocardiography, and simple respiratory function tests all achieved over 70% acceptance as unsupervised tasks whereas lumbar punctures and urinalysis achieved over 70% acceptance as supervised tasks. No investigation achieved over 95% acceptance in either unsupervised or supervised categories, although venous blood sampling came near. Those educational supervisors who accepted the task but did not indicate whether supervised or unsupervised are not included in the results.
For some investigations, acceptance and rejection rates were similar for both unsupervised and rejected categories—for example, urine sampling and sputum sampling. Some rejected a task because they considered it to be a nursing activity whereas others believed the house officer should be capable of performing it. There was parity in responses over all categories for some investigations, for example, Doppler arterial assessment. Comments by consultants stated that experience provided by a job influenced whether the house officer could perform the task independently or not, for example, “... depends on experience, for example, skin biopsy in dermatology ward, Doppler arterial assessment in vascular job, urine microscopy—renal job.”
Some investigations in tasks 4.2 and 5.2 were not seen as within the remit of the house officer. This was also true for tasks 5.3b (computed tomography scan) and 6.5c (injecting: intra-articularly).
Sections 6 and 7
For tasks 6.1 to 6.13 (practical procedures), 9 (47%) achieved over 95% acceptance and 4 (21%) under 90% acceptance. In section 7 (self management skills) of round 1, some of the panel were unhappy to accept some skills as “fully developed” by the end of the preregistration year. Therefore in round 2 for all newly submitted skills the panel were asked to decide whether the skill should still be developing or fully developed by the end of the preregistration year.
Discussion
The findings
The panel believed that the house officer should be able to perform the majority of the identified tasks independently by the end of the year. However, our study also showed those tasks that consultants considered the house officer should perform only after consultation with the consultant or under direct senior medical supervision, or both.
Results suggest that in some jobs the house officer would gain enough experience to execute tasks independently whereas in others they would not. This may indicate that unsupervised execution of all but a few investigations may be dependent upon the specialty into which the preregistration house job is placed.
In round 2, the educational supervisors agreed that all the self management skills were required by preregistration house officers, but opinion differed on how well developed these skills should be by the end of the year. This response may be indicative of the differing values held by individual educational supervisors rather than reflecting judgments on the basis of their knowledge of the house job. It may also indicate that an increase in the number of response choices reduces the chance of agreement being achieved.
The Delphi technique
Although a significant number of tasks achieved a high level of agreement, literature on the Delphi technique does not stipulate at what level consensus can be deemed to have been reached. Therefore we set an arbitrary decision of 95% in round 1. To use this level of acceptance in round 2 would have removed from the final list all laboratory and clinical investigations and those tasks the house officer is said to find demanding, for example, breaking bad news.
The panel
The constitution of the panel depended on the clinical tutors selecting individuals who they believed to be well informed. Although this was thought the most appropriate way of identifying the “experts,” it is acknowledged that this, together with the non-responders, may have caused hidden bias. Insufficient data were available to perform analysis by specialty, and no statistically significant associations were found when analysis was performed by physician versus surgeon classification.
Conclusion
The Delphi technique was useful in gaining the opinions of educational supervisors on the tasks that should be included in the preregistration year, and this technique may prove a useful tool in monitoring future changes to the job. Further work on the items identified by our study could be undertaken to differentiate between the tasks and skills that educational supervisors want the house officer to experience within the preregistration year and those the house officer must perform competently to achieve registration.
For the self evaluation instrument, those items scoring below 50% acceptance will be rejected as tasks for house officers and those with over 90% acceptance will be accepted. Items ranging from 50% to 90% will be further modified in the light of the comments in round 2. The practical and laboratory investigations are undoubtedly part of some house officer jobs and not others. The instrument will record whether the house officers perform these tasks, and educational supervisors will be left to decide whether this is acceptable within their discipline.
The information gained from the self evaluation instrument will be used to encourage discussion between educational supervisors and house officers about the appropriateness, completeness, and quality of the educational programmes set for them. It could also form a potential feedback loop for assessing the effectiveness of the programmes.
Table 2.
Section 2: on call patient care. Values are numbers (percentages) of panel accepting statement
| No | Task | Panel response |
|---|---|---|
| 2.1 | As preregistration house officer could be faced with any one of wide range of conditions when on call, for any case tasks of house officer are to: | |
| a. Differentiate between simple and complex causes of symptoms | 61 (91)† | |
| b. Initiate general supportive measures | 67 (100)† | |
| c. In complex cases seek help from senior persons | 66 (99)† | |
| d. In simple cases begin treatment | 66 (99)† | |
| e. Monitor patient’s condition | 67 (100)† | |
| 2.2 | To perform cardiopulmonary resuscitation using: | |
| a. External cardiac massage | 64 (100)* | |
| b. Airway management (not including intubation) | 67 (100)† | |
| 2.3 | To perform cardiopulmonary resuscitation as part of cardiac arrest team using: | |
| a. Defibrillator | 62 (93)† | |
| b. Common drug treatments | 60 (90)† | |
| 2.4 | Verify death | 65 (97)† |
Tasks accepted in round 1 (64 responders).
Tasks accepted in round 2 (67 responders).
Table 3.
Section 3: routine patient care. Values are numbers (percentages) of panel accepting statement
| No | Task | Panel response |
|---|---|---|
| 3.1 | Taking history | 64 (100)* |
| 3.2 | Examining patient | 64 (100)* |
| 3.3 | Recording information | 64 (100)* |
| 3.4 | Interpreting information held in case notes | 63 (98)* |
| 3.5 | For tests, laboratory investigations, treatments, and referrals, preregistration house officers are expected to organise with clerical support: | |
| a. Administration and paperwork | 65 (97)† | |
| b. Sequencing and timing | 62 (92)† | |
| 3.6 | With clerical assistance, collate all patient information for ward rounds | 67 (100)† |
| 3.7 | With clerical assistance, collate all patient information for theatre lists | 42 (63)† |
| 3.8 | With supervision, provide ongoing clinical care for inpatients, for example, daily visits and monitoring | 67 (100)† |
| 3.9 | Prescribing drug regimens for: | |
| a. Infection (non-complex cases) | 66 (99) † | |
| b. Pain relief (non-complex cases) | 67 (100)† | |
| c. Pain control for terminally ill (with supervision) | 65 (97) † | |
| d. Sedation (non-complex cases) | 64 (96)† | |
| 3.10 | Discharge procedure: | |
| a. Writing or signing home prescription forms | 63 (98)* | |
| b. Completing hand written discharge forms | 62 (97)* | |
| 3.11 | Writing technically correct drug prescription, for example, fulfilling British National Formulary guidelines | 67 (100)† |
| 3.12 | Calculating appropriate drug dosage | 67 (100)† |
| 3.13 | Dictating discharge letter to general practitioner | 34 (51)† |
| 3.14 | Summarising past records | 60 (90)† |
| 3.15 | Creating problem list and management plan | 61 (91)† |
Tasks accepted in round 1 (64 responders).
Tasks accepted in round 2 (67 responders).
Table 4.
Section 4: laboratory investigations. Values are numbers (percentages) of panels’ response in round 2
| No | Task | Panel response | ||
|---|---|---|---|---|
| 4.1 | For laboratory investigations preregistration house officers are expected to perform following tasks: | |||
| a. In liaison with more senior doctor make decision on which laboratory investigations are required for individual patients | 66 (99)* | |||
| b. Understand significance of reported findings, for example, whether they suggest immediate consultation with more senior doctor | 66 (99)* | |||
| 4.2† | Preregistration house officer may be called upon to take the following samples: | Unsupervised | Supervised | Rejected |
| Venous blood sampling | 63 (94) | 1 (2) | 0 | |
| Urine sampling | 35 (52) | 1 (2) | 28 (42) | |
| Sputum sampling | 28 (42) | 1 (2) | 36 (54) | |
| Lumbar puncture | 7 (10) | 53 (79) | 2 (3) | |
| Joint aspiration | 3 (5) | 31 (46) | 25 (37) | |
| Pleural biopsy | 1 (2) | 20 (30) | 39 (58) | |
| Pleural aspiration | 20 (30) | 42 (63) | 0 | |
| Skin biopsy (non-malignant) | 9 (13) | 22 (33) | 27 (40) | |
| Liver biopsy | 0 | 9 (13) | 54 (81) | |
| Knee aspiration | 3 (5) | 32 (48) | 26 (39) | |
| Fine needle aspiration | 1 (2) | 13 (19) | 48 (72) | |
| Needle biopsy of prostate | 0 | 1 (2) | 60 (90) | |
| 4.3 | Physical preparation of patient for investigation, for example, when applicable lying patient in correct position, swabbing area | 60 (90)* | ||
| 4.4 | Prepare syringes, specimen bottles, and labels for samples | 61 (91)* | ||
Accepted.
Those who accepted task but failed to indicate supervised or unsupervised not included.
Table 5.
Section 5: clinical investigations. Values are numbers (percentages) of panels’ response in round 2
| No | Task | Panel response | ||
|---|---|---|---|---|
| 5.1 | For clinical investigations preregistration house officers are expected to perform following tasks: | |||
| a. In liaison with more senior doctor make decision on which clinical investigations are required for individual patients | 65 (97)* | |||
| b. Understand significance of reported findings, for example, whether they suggest immediate consultation with more senior doctor | 66 (99)* | |||
| 5.2† | A preregistration house officer may be called upon to perform the following tests: | Unsupervised | Supervised | Rejected |
| Urinalysis | 0 | 48 (72) | 15 (22) | |
| Urine microscopy | 21(31) | 7 (10) | 33 (49) | |
| Electrocardiography | 61(91) | 0 | 1 (2) | |
| Abdominal paracentesis | 12 (18) | 45 (67) | 5 (8) | |
| Central venous pressure measurement with line in situ | 34 (51) | 13 (19) | 11(16) | |
| Echocardiography | 0 | 0 | 66 (99) | |
| Exercise stress test | 3 (5) | 3 (5) | 58 (87) | |
| Simple respiratory function test—that is, spirometry, peak flow rate | 47 (70) | 4 (6) | 7 (10) | |
| Sigmoidoscopy | 0 | 30 (45) | 31(46) | |
| Upper gastrointestinal endoscopy | 0 | 1 (2) | 66 (99) | |
| Flexible cystoscopy | 0 | 0 | 67 (100) | |
| Proctoscopy | 10 (15) | 26 (39) | 27 (40) | |
| Doppler arterial assessment | 19 (28) | 18 (27) | 25 (37) | |
| Barium enema | 0 | 0 | 67 (100) | |
| Abdominal ultrasound | 0 | 0 | 67 (100) | |
| 5.3 | In emergency situations to take decision to order: | |||
| a. Plain radiography for chest, abdomen, and skull | 66 (99)* | |||
| b. Computed tomography scan | 15 (22)* | |||
| c. Ventilation-perfusion scan | 32 (48)* | |||
Accepted.
Those who accepted task but failed to indicate supervised or unsupervised not included.
Acknowledgments
We thank the Northern region educational supervisors, clinical tutors, and postgraduate centre managers.
Footnotes
Funding:Medical and Dental Education Levy and the Northumberland Health Authority.
Competing interests: SJS was chairman of the Local Medical Workforce Advisory Group while the study was being conducted.
References
- 1.General Medical Council. Tomorrow’s doctors. London: GMC; 1993. [Google Scholar]
- 2.General Medical Council. The new doctor. London: GMC; 1997. [Google Scholar]
- 3.Dowling S, Barrett S. Doctors in the making: the experience of the pre-registration year. Bristol: Saus; 1991. [Google Scholar]
- 4.Leslie PJ, Williams JA, McKenna C, Smith G, Heading RC. Hours, volume and type of work of pre-registration house officers. BMJ. 1990;300:1038–1041. doi: 10.1136/bmj.300.6731.1038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Turnbull NA, Miles NA, Gallen IW. Junior doctors’ on-call activities: differences in workload and work patterns among grades. BMJ. 1990;301:1191–1192. doi: 10.1136/bmj.301.6762.1191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Dent THS, Gilliard JH, Aarons EJ, Smyth-Pigott PJ. Variations in clinical experience of pre-registration house officers: the effect of London. Health Trends. 1995;27:22–26. [PubMed] [Google Scholar]
- 7.British Medical Association. BMA cohort study of 1995 medical graduates: second report. The pre-registration year. London: BMA; 1997. [Google Scholar]
- 8.McKee M, Black N. Junior doctors’ work at night: What is done and how much is appropriate? J Public Health Med. 1993;15:16–24. doi: 10.1093/oxfordjournals.pubmed.a042815. [DOI] [PubMed] [Google Scholar]
- 9.Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311:376–380. doi: 10.1136/bmj.311.7001.376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Duffield C. The Delphi technique. Aust J Adv Nurs. 1988;6:41–45. [PubMed] [Google Scholar]
- 11.Williams PL, Webb C. The Delphi technique: a methodological discussion. J Adv Nurs. 1994;19:180–186. doi: 10.1111/j.1365-2648.1994.tb01066.x. [DOI] [PubMed] [Google Scholar]
- 12.Conference of Postgraduate Medical Deans and Directors of Postgraduate Medical Education of Universities in the UK. The pre-registration house officer experience: implementing change. London: COPMED; 1994. [Google Scholar]
- 13.Gillard JH, Dent THS, Aarons EJ, Smyth-Pigott PJ, Nicholls MWN. Pre-registration house officers in eight English regions: survey quality of training. BMJ. 1993;307:1180–1184. doi: 10.1136/bmj.307.6913.1180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Mitchell IA, Teale GR. The practical house officer. Oxford: Blackwell Scientific; 1992. [Google Scholar]
- 15.Calman KC, Donaldson M. The pre-registration house officer year: a critical incident study. Med Educ. 1991;25:51–59. doi: 10.1111/j.1365-2923.1991.tb00026.x. [DOI] [PubMed] [Google Scholar]
- 16.Dent THS, Gilliard JH, Aarons EJ, Crimlisk HL, Smyth-Pigott PJ. Pre-registration house officers in the 4 Thames regions. 1: survey of education and workload. BMJ. 1990;300:713–716. doi: 10.1136/bmj.300.6726.713. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Firth-Cozens J, Morrison LA. Sources of stress and ways of coping in junior house officers. Stress Med. 1989;5:121–126. [Google Scholar]
- 18.Elizabeth JE, Hughes S. An assessment of the pre-registration year experience. BMJ. 1986;293:1559. doi: 10.1136/bmj.293.6561.1559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Dornbusch SM, Scott WR. Evaluation and the exercise of authority. San Francisco: Jossey-Bass; 1975. p. 71. [Google Scholar]