Abstract
First published in 2006, the first European core syllabus in Adult Respiratory Medicine was developed with the intention of harmonising education and training throughout Europe. Internationally recognised by the European Union of Medical Specialists and identified as the first document of its kind in respiratory medicine, it has provided a comprehensive guide for both local and national institutions in the development of adult respiratory training programmes.
Like all fields in education, respiratory medicine is an ever-changing area and as such, respective syllabi, curricula and training programmes must adapt and diversify in line with the evolution of core medical concepts. Given the proven importance of the Adult Respiratory Medicine syllabus from both a national and international standpoint, it is of equal importance that said syllabus remains abreast of emerging trends so as to sustain the synchronisation of respiratory medicine in Europe. In order to develop an updated programme, a comprehensive review process of the current syllabus is a necessary endeavour and a step that the European Respiratory Society (ERS) has undertaken through the process of a needs assessment.
Short abstract
Updated ERS international Adult Respiratory Medicine syllabus for postgraduate training http://ow.ly/Phyx30hSlyD
Project rationale
Common standards for training and certification were needed within Europe to move forward with the mobility of medical professionals and to guarantee patient safety [1]. The task force, which developed the first syllabus draft, was composed of experts representing different regions of Europe as well as representatives from each of the following organisations: UEMS (European Union of Medical Specialists), EBAP (European Board for Accreditation in Pneumology), CERS (Conference of European Respiratory Societies) and the ERS Early Career Members’ Committee (ECMC) [2]. The core syllabus was the first of the consensus-based documents to be produced and published under the HERMES (Harmonised Education in Respiratory Medicine for European Specialists) initiative. Focusing on knowledge, the development of learning outcomes was the next step undertaken and was successfully fulfilled with the publication of the curriculum in 2008. Based on these documents, the ERS HERMES examination was established to standardise the certification of knowledge following the completion of national training programmes in adult respiratory medicine [3]. The ERS HERMES summer schools were established and educational resources, such as the ERS Handbook of Respiratory Medicine, were produced to provide preparatory support for the exam. Furthermore, fundamental criteria required to provide quality training in adult respiratory medicine were defined for training centres in collaboration with EBAP. The target audience was postgraduate trainees.
This document highlights the work accomplished during 2015–2017. There have been a number of considerations including some updates and changes to the original content as well as detailed discussions on target audience. This publication is a key milestone in outlining a consensus-based document of the knowledge, skills and attitudes required of respiratory medicine specialists at trainee level. Further development will continue to consider the content required of the trained specialist for the purpose of continued professional development [1].
What is a needs assessment?
Needs assessment, often defined as a situational analysis, is a systematic process aimed at identifying and prioritising needs; collecting and analysing information; making informed, needs-based decisions; allocating resources; and implementing actions to resolve problems underlying important needs [4, 5]. Ury et al. [6] outline that needs assessment is a vital component in the infancy stages of any curriculum design, forming part of the foundational platform on which a working curriculum is built; however, it does not end after implementation. Instead, it is an ongoing process “which continues to provide important and useful information and feedback that ensures the sustainability of the curriculum” [6]. Using a range of different methods, such as questionnaire surveys, focus groups, interviews and the notable Delphi method, a needs assessment has come to be recognised as a necessary and worthwhile endeavour in curriculum planning and development. Given the proven merits of the Delphi method, it was the intention of the ERS task force to employ this form of needs assessment to both inform and prioritise certain needs based upon the results of the situational analysis as well as ascertaining the specifics of what should be included within an Adult Respiratory Medicine training programme.
Project background
The objective of this study was to perform a needs assessment aimed at addressing changing developments in the field, focusing on reviewing and refining the content of the Adult Respiratory Medicine curriculum as required.
A curriculum update project was first proposed and initiated during meetings at the ERS International Congress in Barcelona in 2013. Between October 2013 and January 2015, a designated task force focused on reviewing the curriculum currently in operation. In summary, the following issues were identified for analysis.
Gaps were identified in the content related to lifestyle, environment, occupation and public health measures (e.g. indoor and outdoor pollution, respiratory hazards in the workplace).
Proposed inclusion of exempted items as well as new items.
Sequential arrangement of certain modules.
Target audiences
When assessing and reviewing the Adult Respiratory Medicine syllabus, careful consideration of the relevant target audiences was necessary. The organisation and structure of Adult Respiratory Medicine in general is complex and the role of ERS at an international level requires careful consideration of the range of different audiences that will use the syllabus in different capacities. Different target audiences include trainees in respiratory medicine, trained specialists within a range of specific fields as well as health professionals who play a key role in this specialty area. The different levels of training and education required for each target audience must also be considered along with existing national educational structures for training and certification.
The Delphi methodology
This study was conducted using a modified Delphi technique, a consensus-based methodology that is favoured by medical educationalists worldwide [7].
Delphi is a “group communication process” that focuses on gathering information from a panel of experts, aiming to reach a convergence of opinion regarding a particular issue or set of issues [8]. Unlike other data collection methods, the Delphi method employs a number of iterations or rounds, with three rounds being deemed sufficient by relevant experts, until a consensus of opinion is reached [8]. To begin with participants are supplied with a prepared questionnaire, which is followed by a second questionnaire based on the results garnered from the first. Subsequent surveys refine and define the central issues at heart of the study, taking into account the opinions provided by the participants [9].
Determining the level of consensus must be carefully considered when developing the Delphi rounds. There is no universal consensus proportion agreed upon for the Delphi method; different experts have advocated percentage measures ranging between 50% [10], 70% [11] and 80% [12]. For this particular study, the ERS task force opted to apply an 80% cut-off rate.
The process
The task force
Invitations to join the task force were sent in December 2014. Gernot Rohde (the Netherlands), the ERS Education Council Chair, was chosen to lead the group. The other members were active ERS members selected to represent the eleven ERS assemblies in existence at that time and the ECMC (table 1).
Table 1.
Task force members
Task force member | Country represented |
Konrad Bloch | Switzerland |
Brendan Cooper | UK |
Ernst Eber | Austria |
Felix Herth | Germany |
Ildiko Horvath | Hungary |
Deborah Jarvis | UK |
Christian Kaehler | Austria |
Gilbert Massard | France |
Anne-Pascale Meert | Belgium |
Stylianos Ofanos | Greece |
Gernot Rohde | Germany |
Szymon Skoczynski | Poland |
Frank Smeek | The Netherlands |
Study participants
Prior to commencing round one of data collection, focus issues were ascertained and participants were identified during a plenary meeting of the ERS International Congress in Amsterdam in September 2015. Following a review, the draft syllabus containing 128 items divided over 22 modules, was finalised before being forwarded to all associated parties.
The participants involved in the Delphi process formed two separate groups, an expert group and a public group. The expert group was comprised of members of the task force, the Education Council, Assembly heads and secretaries, group chairs and secretaries and a number of national respondents (one national representative per country was nominated by CERS). The public group consisted of ERS assemblies: clinical; respiratory intensive care; basic and translational sciences; clinical physiology and sleep; airway diseases; epidemiology and environment; respiratory infections; and thoracic oncology.
Data collection
The study was performed between December 2015 and February 2017, and was designed as a three-round Delphi iterative process. The first round took place between December 2015 and January 2016, the second round between August 2016 and September 2016, and the third round between November and December 2016. The time frame for the first two rounds was 4 weeks, while the third and final round ran for 5 weeks.
The Delphi rounds
For this study, the Delphi rounds were divided into three different stages: 1) a qualitative round involving all task force members, which was achieved through a facilitated focus group; 2) a semi-qualitative round with the task force members and chair approving the online survey; and 3) a quantitative round, repeated for three consecutive rounds (Delphi 1, 2 and 3), with members of both the expert and the public groups taking part in the online questionnaire.
The results and feedback collected from each survey were then discussed and analysed during the biannual task force discussion meetings.
Delphi Round 1
All participants were asked to determine whether certain modules were “mandatory”, “optional” or “excluded”. In addition, group members were required to vote on the inclusion of individual syllabus items using a Likert scale [13], with the scale ranging from 1 (strongly disagree) to 5 (strongly agree). Furthermore, they were also able to add comments or suggestions for module content.
In January 2016, feedback from Delphi round 1 was collated and discussed by task force members. From the 128 items surveyed, only 12 items failed to reach the approval rate of 80% and were therefore deemed controversial. Following extensive debate as to whether certain items should be retained, edited or excluded, the revised syllabus draft was then prepared for the second round of surveys.
Delphi Round 2
Round 2 took place in August 2016 with participants invited to rate the updated draft syllabus. Following discussion of the results, it was decided by task force members to launch a third Delphi round focusing only on the non-consensual items.
Delphi Round 3
The final round had two aims: to provide the respondents with a detailed overview of the results from the previous rounds; and to offer them a final opportunity to review and comment on the syllabus. Focusing only on the items that did not reach a consensus of above 80% in previous rounds, participants were required to vote “yes” or “no” as to whether include them within a certain module. In February 2017, the Delphi 3 results were discussed and the syllabus, comprising of 20 modules and 136 items, was finalised.
Statistical analysis
Descriptive statistics for all variables were provided, describing group responses by measures of central tendency (mean), dispersion (sd) and frequency distribution (percentages). The interquartile range, the mode and the median were also described for a more robust analysis. The first set of indices was employed to obtain an indication of consensus.
To ensure the consistency of the responses between the rounds, the Delphi round 2 questionnaire was similar to that used in Delphi round 1. The answers in the top two Likert measures (agree and strongly agree) were compared, from the respondents who participated in Delphi 1 and 2 to describe the group stability (see the online supplementary table). The McNemar test for correlated variables was applied. The percentage obtained gives an indication as to the stability of the responses. The items which scored <80% are those considered controversial, which is consistent with the study undertaken by ERS.
Results and discussion
Over 80% of the syllabus items under analysis reached a consensual agreement of 80% and above, resulting in a minimal number of items deemed “controversial”. The additional comments provided by the participants were reviewed and where possible integrated within the final content for further consideration in the Delphi process. Table 2 outlines the final content of the Adult Respiratory Medicine syllabus for trainees.
Table 2.
Adult Respiratory Medicine – Syllabus for trainees
Module 1. Structure and function of the respiratory system |
1 Anatomy and development (including malformations) |
2 Immunology and defence mechanisms |
Module 2. Physiology and pulmonary function testing |
1 Ventilation (theory) |
2 Circulation (theory) |
3 Control of breathing (theory) |
4 Control of ventilation (theory) |
5 Respiratory mechanics (diagnostics) |
6 Gas exchange (diagnostics) |
7 Arterial blood gas and acid–base status assessment (diagnostics) |
8 Exercise testing (diagnostics) |
9 Bronchial challenge testing (diagnostics) |
10 Lung function tests (interpretation) |
Module 3. Diagnostic approach |
1 Symptoms |
2 Signs |
3 Syndrome based approach to diagnosis and differential diagnosis |
Module 4. Diagnostic procedures |
1 Endoscopic techniques items |
1.1 Bronchoscopy |
1.2 Endobronchial ultrasound (EBUS) and oesophageal ultrasound (EUS) |
1.3 Thoracoscopy |
2 Thoracocentesis including closed pleural biopsy |
3 Imaging techniques |
3.1 Chest radiography |
3.2 Thoracic ultrasound |
3.3 Computed tomography (CT) scan |
3.4 Positron emission tomography (PET) CT |
4 Assessment of airway inflammation |
4.1 Exhaled nitric oxide fraction (FeNO) and nasal nitric oxide (nNO) |
4.2 Sputum assessment |
5. Basic microbiological methods |
Module 5. General principles of treatment modalities and prevention measures |
1 Pharmacotherapy |
1.1 Inhaled drug therapy |
1.2 Systemic pharmacotherapy |
2 Immunotherapy |
2.1 Allergen-specific immunotherapy |
2.2 Lung cancer immunotherapy |
3 Respiratory physiotherapy |
4 Pulmonary rehabilitation |
5 Palliative care |
6 Oxygen therapy |
7 Preventive measures |
8 Smoking prevention and cessation |
9 Assisted ventilation |
10 Chest tube insertion |
11 Advanced treatment modalities |
11.1 Endobronchial interventions |
11.2 Lung volume reduction (surgical and endoscopic lung volume reduction (ELVR)) |
11.3 Lung transplantation |
Module 6. Respiratory emergencies |
1 Evaluation |
2 Differential diagnosis |
3 Immediate management steps |
4 First-line treatment |
5 Specific conditions |
Module 7. Airway diseases |
1 Upper airway diseases |
2 Asthma |
3 Bronchitis |
4 Chronic obstructive pulmonary disease (COPD) and emphysema |
5 Bronchiolitis |
6 Bronchiectasis |
7 Rare airway diseases |
8 Congenital airway disease |
Module 8. Respiratory infections |
1 Upper respiratory tract infections |
2 Lower respiratory tract infections |
3 Pleural infections |
4 Lung abscess and other infections |
5 Influenza, pandemics and severe acute respiratory syndrome (SARS) |
6 Infections in the immunocompromised host |
7 Aspiration pneumonitis |
Module 9. Mycobacterial diseases |
1 Pulmonary tuberculosis (TB) including multidrug-resistant (MDR)/extensively drug resistant (XDR)-TB |
2 Extrapulmonary TB |
3 Latent tuberculous infection |
4 Non-tuberculous mycobacterial diseases |
Module 10. Thoracic tumours |
1 Lung tumours |
1.1 Lung cancer (include paraneoplastic syndromes) |
1.2 Other lung tumours |
1.3 Metastatic pulmonary tumours |
1.4 Solitary nodules |
2 Pleural tumours |
2.1 Mesothelioma |
2.2 Other pleural tumours |
3 Chest wall tumours |
4 Mediastinal tumours |
Module 11. Sleep and control of breathing disorders |
1 Obstructive sleep apnoea syndrome |
2 Central sleep apnoea syndrome |
3 Hypoventilation syndromes |
4 Dysfunctional breathing/hyperventilation syndrome |
Module 12. Respiratory failure |
1 Acute respiratory failure |
2 Chronic respiratory failure |
Module 13. Diffuse parenchymal lung diseases |
1 Hypersensitivity pneumonitis |
2 Sarcoidosis |
3 Idiopathic interstitial pneumonias |
4 Cryptogenic organising pneumonia of unknown aetiology/bronchiolitis obliterans organising pneumonia |
5 Connective tissue disease related interstitial lung disease |
6 Langerhans cell histiocytosis |
7 Lymphangioleiomyomatosis (LAM) |
8 Pulmonary alveolar proteinosis |
9 Amyloidosis |
10 Drug-induced disease |
11 Radiation-induced disease |
12 Nonasthmatic eosinophilic bronchitis |
13 Acute and chronic eosinophilic pneumonia |
14 Hypereosinophilic syndrome |
Module 14. Pulmonary vascular diseases |
1 Thromboembolic disease |
2 Pulmonary hypertension |
3 Vasculitis and diffuse pulmonary haemorrhage |
4 Arteriovenous malformation |
Module 15. Diseases of the chest wall and respiratory muscles including the diaphragm |
1 Chest wall deformities |
2 Neuromuscular disorders |
3 Diaphragmatic disorders |
Module 16. Pleural and mediastinal diseases (excluding tumours) |
1 Pleural effusion |
2 Pneumothorax |
3 Mediastinitis |
Module 17. Respiratory consequences of systemic/ extrapulmonary conditions |
1 Primary immunodeficiency syndromes |
2 Secondary immunodeficiency syndromes/immunosuppression |
3 Cardiac disease |
4 Gastrointestinal, liver and kidney disease |
5 Haematological disease |
6 Obesity |
7 Connective tissue diseases |
Module 18. Genetic disorders |
1 Cystic fibrosis |
2 Primary ciliary dyskinesia |
3 α1-antitrypsin deficiency |
4 Birt–Hogg–Dubé syndrome |
Module 19. Occupational diseases |
1 Respiratory hazards associated with occupational factors |
2 Acute inhalation injuries and their possible sequelae |
3 Occupational respiratory infections including in healthcare workers |
4 Work-related asthma (including occupational asthma and work-exacerbated disease) |
5 COPD and work |
6 Interstitial lung disease (ILD) caused by inorganic agents |
7 ILD caused by dusts of biological origin (including extrinsic allergic alveolitis) |
8 Asbestos-related conditions other than bronchopulmonary cancer (but including mesothelioma) |
9 Occupational causes of bronchopulmonary cancer |
Module 20: Epidemiology, environment and lifestyle |
1 Epidemiological and statistic methods for critical appraisal |
2 Indoor pollution |
3 Outdoor pollution |
4 High-altitude and diving |
5 Lifestyle (smoking, e-cigarettes, nutrition and activity) |
Items that did not reach consensus amongst participants were debated by the task force. A summary of the discussions surrounding controversial items is provided below.
EBUS/EUS
Endobronchial ultrasound (EBUS) and oesophageal ultrasound (EUS) were two separate items in Delphi 1 and 2. EBUS, an endoscopic technique used to diagnose and stage lung cancer, reached the approval rate of 80% in Delphi 1 and 79% in Delphi 2. However, EUS, used to diagnose and stage oesophageal, pancreatic, gastric, colorectal and bile duct cancers, only scored 65.48% (Delphi 1) and 52.22% (Delphi 2). Feedback from the respondents suggested that these techniques were too specialised and only available in select medical centres. Additionally, EUS is not implemented in all hospitals and seen as complementary to EBUS. The task force considered these comments and decided to merge EUS and EBUS as only knowledge about the technique, the indications and the basic technical aspects will be required by a number of pulmonologists. The ERS recognises, however, the rapidly growing need for EBUS and EUS expertise and the major technical advances within the interventional area and thus assume that these techniques will be included in the syllabus in the future.
Thoracoscopy
Thoracoscopy is a procedure employed in the diagnosis and treatment of pleural diseases. This procedure permits exploring the pleural cavity and conducting biopsies of pleural lesions [14]. Only 68.8% (Delphi 1), 67.22% (Delphi 2) and 59.04% (Delphi 3) of the respondents agreed to include this procedure within the syllabus, as this expertise is not integrated within most European training centres. However, they were in agreement that the knowledge and indications were a necessary inclusion. The task force decided to retain this item, including only knowledge of the procedure, its indications and limitations. The ERS, however, recognise and support the further development of medical thoracoscopy performed by pulmonologists.
Immunotherapy
Immunotherapy is defined as “the prevention or treatment of disease with substances that stimulate the immune response” (Oxford Dictionary). According to some respondents, immunotherapy was deemed a specialised item that should not be included in the general Adult Respiratory Medicine syllabus. However, as this item is seen as an important, up-and-coming topic, it was decided to retain this item in the syllabus.
Epidemiology, environment and lifestyle module
Reaching a consensus regarding the module on environmental diseases has proven challenging for the task force. Following Delphi 1 and 2, the title and position of the module within the syllabus was edited with “Smoking” the only unanimously accepted item to be included.
Challenges and limitations
Over 12 years the development and redevelopment of internationally recognised training standards for respiratory medicine have met with many challenges and, at times, limitations for further development. On the one hand, this ERS task force has respected and supported the wide variations of standards and regulations of each individual country in their best efforts to provide trained and certified respiratory medicine specialists. On the other hand, the mobility of medical professionals and the increased mobility of patients have placed increased pressure on medical facilities to provide and meet minimum international standards of quality patient care. Attaining and managing this balance has been no easy feat. The work of this international task force and the Delphi process has ensured that the individual voice of each represented country has been heard, taking into account the various needs and respective perspectives of training currently in operation (table 1). These inherent challenges have called for a number of considerations.
1) To what degree have the ERS international standards for training of respiratory professionals been accepted and used?
To date, there is little consensus on how widely the 2006 published European core syllabus in respiratory medicine [2] has been used for training across countries. This is very much dependent on the training structure of each individual country and their established training programmes. The uptake of an international syllabus is fraught by local and national regulations for training, use of English as an international language versus national and regional languages [15], and how disease and approaches to medical action differ across countries [16]. The use of the 2006 European core syllabus in respiratory medicine across countries will require further investigation to assess the frequency and level of use.
2) Is this study overly reliant on the active participation of those associated with ERS?
The ERS task force and national delegates involved in the development of this updated syllabus have been invited through the ERS membership database. Invitations were also sent to CERS, which is also affiliated with ERS. While the task force has tried to ensure equal geographic and professional participation from respiratory specialists, it must be noted that most of the respondents held an academic position or were a consultant in a public hospital, which may result in an overly ambitious syllabus not truly representative of the respiratory medicine community.
3) How can this document have a measurable impact on national and international training for respiratory medicine specialists?
Probably the greatest limitation of this project is the inability of an international task force to provide a full training programme for local institutions which will consider the essential components of knowledge, skills and attitudes required for practice. The scope of a programme developed and delivered through the international community begins and ends at the publication of internationally agreed upon standards with some supporting initiatives which may assess knowledge with limited assessment of skills. It must be noted that this project was never intended to replace local, regional or national training for respiratory specialists, but rather to act as an aid to support and reference current training, assessment and actualisation programmes.
Future considerations
The true sustainability of any efficient syllabus and curriculum lies in the ability of those at the helm to recognise the necessity of reviewing and subsequently adapting educational structures so as to meet the ever-changing needs of the respective learners. To ensure the efficiency of the Adult Respiratory Medicine syllabus, the undertaking of the Delphi method of needs assessment proved an extremely worthwhile endeavour. Informed by the feedback garnered from the Delphi rounds, the intention of the new, updated Adult Respiratory Medicine syllabus is to ensure that emerging fields in respiratory medicine, as well as updates in diagnosis, management and treatment practices since the initial 2006 publication, are addressed and reflected within the new publication. Comprising a simplified structure with fewer modules, the reviewed syllabus is designed to act as a guide for educational design, the planning of teaching and learning, and assessment structures as well as resource activities, supporting the ERS and other institutions using the ERS syllabus. Given the scope of this project, the needs assessment process was not without its limitations, although its merits certainly outweighed the aforementioned challenges. In reviewing and learning from past accomplishments, through carrying out a needs assessment, ERS is continuing to make strides towards a healthy future in Adult Respiratory Medicine.
Key points
Needs assessment, often designed as a situational analysis, is a systematic process aiming at identifying and prioritising needs, collecting and analysing information.
Needs assessment has been recognised as necessary in curriculum planning and development in medical education.
The Delphi method is a “group communication process” that focuses on gathering information from a panel of experts, aiming to reach a convergence of opinion regarding a particular issue or set of issues [8].
The Delphi technique was used as a needs assessment approach.
Self-assessment questions
- 1 All of the following are characteristics of the Delphi methodology except:
- a) It is a consensus-based methodology
- b) It employs a number of iterations or rounds until a consensus of opinion is reached
- c) The universal consensus proportion agreed upon for the Delphi method is 80%
- d) It is a group communication process that focuses on gathering information from a panel of experts
- 2 Which of the following statements regarding a needs assessment is/are correct?
- a) It is a vital component of any curriculum design
- b) It is always composed of questionnaire surveys, focus groups and interviews
- c) It is a systematic process aimed at identifying and prioritising needs, collecting and analysing information
- d) It is used only in the initial stages of curriculum design
Suggested answers
1) c.
2) a. true; b. false; c. true; d. false.
Supplementary material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material EDU-0193-2017_supplementary_material (699.5KB, pdf)
Footnotes
Supplementary material This article has supplementary material available from breathe.ersjournals.com
Conflict of interest N. Tabin is an employee of the European Respiratory Society.
Conflict of interest S. Mitchell is an employee of the European Respiratory Society.
Conflict of interest E. O’Connell has nothing to disclose.
Conflict of interest D. Stolz is the Chair of the Education Council for the European Respiratory Society 2017–2020.
Conflict of interest G. Rohde reports personal fees from Pfizer, Boehringer Ingelheim, Solvay, GSK, Essex Pharma, MSD, Roche and Novartis for lectures including service on speakers bureaus and/or consultancy during advisory board meeting; and personal fees from GSK for travel accommodation/meeting expenses, all outside the submitted work.
References
- 1.Mitchell S, Rohde G. Introducing an updated international ERS syllabus for postgraduate training Setting standards for training and certification in respiratory medicine. Breathe 2017; 13: e53–e55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Loddenkemper R, Séverin T, Eiselé J-L, et al. HERMES: a European Core Syllabus in Respiratory Medicine. Breathe 2006; 3: 59–69. [Google Scholar]
- 3.Loddenkemper R, Haslam PL. Multiple choice and the only answer: the HERMES examination. Breathe 2008; 4: 242–248. [Google Scholar]
- 4.Altschuld J, Kumar D. A generic needs assessment model and steps. In: Needs Assessment: An Overview. London, SAGE Publications 2010; pp. 29–56. [Google Scholar]
- 5.Ratnapalan S, Hilliard RI. Needs assessment in postgraduate medical education: A review. Med Educ Online 2002; 7: 8. [DOI] [PubMed] [Google Scholar]
- 6.Ury WA, Reznich CB, Weber CM. A needs assessment for a palliative care curriculum. J Pain Symptom Manage. 2000; 20: 408–416. [DOI] [PubMed] [Google Scholar]
- 7.Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000; 32: 1008–1015. [PubMed] [Google Scholar]
- 8.Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Eval 2007; 12.10: 1–8. [Google Scholar]
- 9.De Villiers MR, de Villiers PJ, Kent AP. The Delphi technique in health sciences education research. Med Teach 2005; 27: 639–643. [DOI] [PubMed] [Google Scholar]
- 10.McKenna HP. The Delphi technique: a worthwhile research approach for nursing? J Adv Nurs 1994; 19: 1221–1225. [DOI] [PubMed] [Google Scholar]
- 11.Sumsion T. The Delphi technique: an adaptive research tool. British Journal of Occupational Therapy 1998; 61: 153–156. [Google Scholar]
- 12.Green B, Jones M, Hughes D et al. Applying the Delphi technique in a study of GP’s information requirements. Health Soc Care Community 1999; 7: 198–205. [DOI] [PubMed] [Google Scholar]
- 13.Likert R. A Technique for the Measurement of Attitudes. Arch Psychol 1932; 140: 1–55. [Google Scholar]
- 14.Oyonarte WM. Módulo pleuroscopía Toracoscopía. Revista chilena de enfermedades respiratorias 2008; 24: 35–39. [Google Scholar]
- 15.Baethge C. The Languages of Medicine. Dtsch Arztebl Int 2008; 105: 37–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Wolf-Maier K, Cooper RS, Kramer H, et al. Hypertension treatment and control in five European countries, Canada, and the United States. Hypertension 2004; 43: 10–17. [DOI] [PubMed] [Google Scholar]
Further reading
- •.Heiko A. Consensus measurement in Delphi studies: review and implications for future quality assurance. Technol Forecast Soc Change 2012; 79.8: 1525–1536. [Google Scholar]
- •.Kitzinger J. Qualitative research: introducing focus groups. BMJ 1995; 311: 299–302. [DOI] [PMC free article] [PubMed] [Google Scholar]
- •.Lalloo D, Demou E, Kiran S, et al. International perspective on common core competencies for occupational physicians: a modified Delphi study. Occup Environ Med 2016; 73: 452–458. [DOI] [PMC free article] [PubMed] [Google Scholar]
- •.Lieff S. Evolving curriculum design: A novel framework for continuous, timely, and relevant curriculum adaptation in faculty development. Acad Med 2009; 84: 127–134. [DOI] [PubMed] [Google Scholar]
- •.Loddenkemper R, Haslam PL, Séverin T, et al. European Curriculum Recommendations for Training in Adult Respiratory Medicine. Breathe 2008; 5: 80–93. [DOI] [PubMed] [Google Scholar]
- •.Loddenkemper R, Séverin T, Eiselé J-L, et al. HERMES: a European core syllabus in respiratory medicine. Breathe 2006; 3: 59–70. [Google Scholar]
- •.Mack MJ, Aronoff RJ, Acuff TE, et al. Present role of thoracoscopy in the diagnosis and treatment of diseases of the chest. Ann Thorac Surg 1992; 54: 403–409. [DOI] [PubMed] [Google Scholar]
- •.Miller GE. The assessment of clinical skills/ competence/performance. Acad Med 1990; 65: 63–67.2302301 [Google Scholar]
- •.Nayahangan L, Clementsen PF, Paltved C, et al. Identifying technical procedures in pulmonary medicine that should be integrated in a simulation-based curriculum: a national general needs assessment. Respiration 2016; 91: 517–522. [DOI] [PubMed] [Google Scholar]
- •.Junod Perron N, et al. Needs assessment for training in interprofessional skills in Swiss primary care: a Delphi study. J Interprof Care 2014; 28.3: 273–275. [DOI] [PubMed] [Google Scholar]
- •.Norman GR, Shannon SI, Marrin ML. The need for needs assessment in continuing medical education. BMJ 2004; 328: 999–1001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- •.Ravonne A. The Delphi Technique in Educational Research. Sage Open 2014: 1–8. [Google Scholar]
- •.Smith CF, Finn GM, Stewart J, et al. Anatomical Society core regional anatomy syllabus for undergraduate medicine: the Delphi process. J Anat 2016; 228: 2–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- •.Stitt-Gohdes WL, Tena BC. The Delphi technique: A research strategy for career and technical education. J Career Techn Educ 2004; 20. [Google Scholar]
- •.Yousuf MI. Using experts’ opinions through Delphi technique. Pract Assess Res Eval 2007; 12.4: 1–8. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material EDU-0193-2017_supplementary_material (699.5KB, pdf)