Table 5.
Details of validity checks and qualitative methods
| Item | Category | 1990–2000 | 2001–2008 | 2009–2012 | Current: 2013–2017 | ||||
|---|---|---|---|---|---|---|---|---|---|
| N = 34a | (%)b | N = 114a | (%)b | N = 179a | (%)b | N = 301a | (%)b | ||
| External validity tested | Yes | 0 | (0) | 1 | (1) | 2 | (1) | 7 | (2) |
| No | 34 | (100) | 113 | (99) | 177 | (99) | 294 | (98) | |
| Internal validity tested | Non-satiation (dominated questions) | 15 | (44) | 56 | (49) | 36 | (20) | 50 | (17) |
| Transitivity (a > b, b > c then c > a) | 3 | (9) | 5 | (4) | 2 | (1) | 2 | (1) | |
| Sen’s expansion and contraction | 0 | (0) | 2 | (2) | 2 | (1) | 2 | (1) | |
| Internal compensatory (1 attribute) | 12 | (35) | 36 | (32) | 30 | (17) | 18 | (6) | |
| Other | N/C | N/C | N/C | N/C | N/C | N/C | 102 | (34) | |
| Not clearly reported/not tested | N/C | N/C | N/C | N/C | N/C | N/C | 189 | (63) | |
| Type of qualitative method used | Interviews | N/C | N/C | N/C | N/C | N/C | N/C | 151 | (50) |
| Focus groups | N/C | N/C | N/C | N/C | N/C | N/C | 54 | (18) | |
| Other | N/C | N/C | N/C | N/C | N/C | N/C | 53 | (18) | |
| No qualitative method used | N/C | N/C | N/C | N/C | N/C | N/C | 43 | (14) | |
| Rationale using qualitative methods | Attribute selection | 6 | (18) | 79 | (69) | 90 | (50) | 160 | (53) |
| Level selection | 6 | (18) | 38 | (33) | 73 | (41) | 134 | (44) | |
| Pre-testing questionnaire | 16 | (47) | 36 | (32) | 73 | (41) | 113 | (38) | |
| Understanding results/responses | 0 | (0) | 5 | (4) | 14 | (8) | 12 | (4) | |
| Not clearly reported/other | N/C | N/C | N/C | N/C | N/C | N/C | 5 | (2) | |
N/C not collected (data were not collected for this specific category)
aNumbers of individual studies might not add up to total Ns as some studies addressed multiple topics
bPercentages might not add up to 100% because some studies addressed multiple topics and because of rounding error