Table 3. Summary of the literature on consensus methods for solving problems in health care.

Consensus method	Characteristics/Advantages	Disadvantages
Delphi first introduced in 1948 [37]	Uses expert panels.	Members drop out often from fatigue [37].
	Requires surveys by questionnaire and/or electroniccommunication (e-mail) for multiple rounds.	Decisions are limited by group members and their pastexperience or work in the field [32].
	Inexpensive data collection method, relyingon repeated rounds of commentsfrom experts.	Criticized for being less representative than the RAND-UCLAappropriateness multidisciplinary panels [52].
	Reliability increases with the size of the group and thenumber of rounds [37].	There is the potential for bias [32] and not having inter-raterreliability testing [31].
	After each round data are analysed and collated into onedocument in preparation for the next round [32].	Is generally inferior to the nominal group technique, albeitto a small degree [24].
	The outcome is a combined opinion achieved in astructured and anonymous way [32].	Difficulties relate to practical rather than theoreticalconsiderations and more research is needed to clarify theconcept expertise.
	The Delphi has been modified [31].
Nominal Group Technique firstdescribed in 1971 by Delbecq andVan de Ven [53]	Is used to create a structured environment in whichexperts are given the best available information forconsidering solutions that are more justifiableand credible than may bethe case otherwise [37].	Face-to-face consensus methods place more responsibilityon the leader than is the case for the Delphi technique,and the NOMINAL GROUP TECHNIQUE therefore requires objectiveand skilled leaders [37].
	Is used for obtaining consensus in an orderly manner frompersons closely associated with a problem area, andis based on the National Institutes of Health (NIH) andthe Glaser approach to consensus [37].	Jones and Hunter (1995) modified the NOMINAL GROUPTECHNIQUE by having a different mix of participants infurther rounds as there is a potential for bias in theselection of experts.
	Is useful to establish agreement on controversialsubjects [37].
	There is no hard and fast rule about the number ofexperts to include in a nominal group but 9–12 arerecommended and lay persons can beincluded [24].
	The modified NOMINAL GROUP TECHNIQUE is facilitated byan expert or credible non-expert while another person takesthe role of non-participant observer collecting qualitativedata from the discussion but is not concerned with analysisof the group process [24].
Consensus conference used by theNational Institutes of Health (NIH)since 1977 [37]	Consists of expert multidisciplinary member panels andoften involves national task forces and committeesand national and international leadersin the field.	Resource intensive.
	Is useful where there is clinical uncertainty [54].	Includes pre-conference preparation of questions andanswers by experts in the field.
	Conference proceedings last from 1.5 to 2.5 daysfollowed by dissemination and evaluation ofrecommendations [54].
RAND-UCLA appropriatenessmethod developed in 1984 by theHealth Services UtilizationStudy [55]	A systematic method combining expert multidisciplinaryclinical opinion and evidence [56].	Resource intensive.
	A rough screening test for specific medical andsurgical procedures [52].	Patient preferences are often neglected [55].
	Measures appropriateness of health services andappropriateness of health settings for quality andcost considerations [55].	There is concern about the method’s subjectivity andunreliability [52].
	Can have a 9–12 member multidisciplinary expertpanel [57].
	Evidence of good reproducibility [52].
	A modified RAND appropriateness model combinedcharacteristics of both the Delphi and nominal grouptechnique [58].
	Discussion rounds can be scored using continuousinteger scales of 1–9 [57].