Skip to main content
. 2015 Aug 13;10(8):e0135162. doi: 10.1371/journal.pone.0135162

Table 1. RANK ORDER of the Dependency of Consensus Indices’ on the NUMBER OF QUESTIONS (6–40) in a Delphi Survey.

Delphi Consensus Index Dependence on the Number of Questions (0.000–1.000)*
Fleiss’ Kappa (KF) .025
Conger’s Kappa (KC) .019
Clustered Mode (CM) .008
Extremity CPWA (XCPWA) .005
Clustered PWA (CPWA) .004
Mode (M) .004
Pair-wise Agreement (PWA) .002
De Moivre index (DM) .000
Interquartile Range (IQR) .000

*The dependency value ranges from 0.000 to 1.000. A value of “0.000” shows complete independence of the Consensus Index from the Delphi survey characteristic examined (e.g., the number of questions) whereas a value of “1.000” shows complete dependence. The dependency value is the maximum numeric difference observed for each consensus index when the number of questions in a simulated Delphi survey varied from 6 to 40.

All Delphi consensus indices (the left column) typically take a value ranging from 0.000 to 1.000, except the Interquartile Range (IQR). For example, in the case of Fleiss’ Kappa, a maximum difference of 0.025 can be anticipated when the number of Delphi survey questions vary from 6 to 40.

For the Interquartile Range, the dependency data were normalized by dividing the difference observed in simulations by the maximum possible difference (9.000), i.e., the length of the Likert scale from 1 to 10 used in the simulations.