Table 1.

Summary of consensus development methods.

	Implicit Methods	Explicit Methods
Approaches	Informal consensus development, consensus development conferences, unstructured research agenda setting	Delphi, NGT, modified Delphi (or RAND/UCLA), structured research agenda setting
General methodology	Qualitative approach and/or simple structure of majority vote, usually of experts	Structured interaction (for NGT and modified Delphi) with explicit rules for aggregation of judgments with complex statistical methods
Pros	Easy to administer, rapid assembly of experts and dissemination	Results are likely more valid, transparency in methods
Cons	Opaque process, hidden dissent, unclear structure of interaction, potential greater influence or dominance of interaction due to participant status, long lists without prioritization	Logistically more difficult to administer, time consuming
Examples	Most clinical practice guidelines released by professional societies; roundtable US NIH discussions to set research priorities in EM	Consensus on ED quality indicators, ED crowding measures, research priorities in syncope
When to use	Research agenda setting where there are multiple research priorities, speed is a priority, logistically infeasible to do explicit methods, lack evidence for most clinical/research scenarios	When not limited by time, logistics, and evidence

Abbreviations: NGT, nominal group technique; ED, emergency department; EM, emergency medicine; US NIH, United States National Institutes of Health