Table 4.
Quality assessment of methodology in work-environment instruments
| Author, year | Quality of instrument development | n | Structural validity | Internal consistency | Other measurement properties | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Meth. quality | Rating | Meth. quality | Rating | Yes/no | Specification | Meth. quality | Rating | |||
| Abraham and Foley [36] | Inadequate | 153 | Doubtful | ? | No | |||||
| Adams, Bond [37] | Adequate | 834 | Doubtful | - EFA loadings NR | Very good | - α 0.92–0.66 | Yes | Reliability measurement error | Doubtful inadequate | - Pearson r 0.90–0.71? NR |
| Aiken and Patrician [38] | Inadequate | 2027 | Doubtful | ? Α 0.79–0.75 | Yes | Reliability hypothesis testing |
Inadequate Inadequate | ? NR? NR | ||
| Appel, Schuler [54] | OP | 1163 | Adequate | - EFA loadings SV 0.86–0.36; LV NR |
Very good | - α: LV 0.87–0.60 SV 0.80–0.63 | No | |||
| Berndt, Parsons [39] | Doubtful | 160 | Adequate | - EFA loadings 0.87–0.45 | Very good | + α 0.92–0.88 | Yes | Hypothesis testing |
Very good | + OOM |
| Bonneterre, Ehlinger [55] | Doubtful | 4085 | Adequate | - EFA loadings NR | Very good | - α 0.89–0.56 | Yes | Reliability hypothesis testing | Doubtful | - Spearman’s r 0.88–0.54? KG |
| Clark, Sattler [71] | Doubtful | 520 | Adequate | - EFA loadings 0.79–0.47 | Very good | + α 0.94 | No | |||
| Duddle and Boughton [40] | Doubtful | 119 | Adequate | - EFA loadings 0.88–0.61 | Very good | + α 0.93–0.78 | No | |||
| Erickson, Duffy [56] | Inadequate | 849 | Adequate | - EFA loadings 0.87–0.31 | Very good | + α 0.88–0.78 | No | |||
| Erickson, Duffy [41] | Inadequate | 1550 (2x775) | Doubtful | ? EFA loadings 0.87–0.34 | Very good | + α 0.88–0.81 | No | |||
| Estabrooks, Squires [42] | OP | 752 | Adequate | - EFA loadings 0.86–0.34 | Very good | - α 0.91–0.54 | Yes | Hypothesis testing |
Very good | + OOM |
| Flint, Farrugia [43] | Inadequate | 195 (EFA) 938 (CFA) | Very good | - EFA loadings 0.95–0.38 CFA for each factor Range CFI 0.99–0.919 range RMSEA 0.08–0.06 |
Very good | + α 0.87–0.81 | No | |||
| Friedberg, Rodriguez [57] | Inadequate | 601 | Very good | + EFA and CFA loadings 0.95–0.38 CFI 0.97 RMSEA 0.04 |
Very good | + α 0.96–0.78 | No | |||
| Gagnon, Paquet [58] | Inadequate | 3142 | Very good | + CFA CFI 0.98 RMSEA 0.05 | Very good | - α 0.91–0.64 | Yes | Hypothesis testing |
Inadequate | ? KG |
| Ives-Erickson, Duffy [44] | Inadequate | 390 | Adequate | - EFA loadings 0.88–0.42 | Very good | + α 0.93–0.84 | No | |||
| Ives Erickson, Duffy [45] | Inadequate | 874 | Adequate | - EFA loadings 0.85–0.51 | Very good | + α 0.92–0.82 | No | |||
| Jansson von Vultée [59] | Inadequate | 435 | Inadequate | - NR | Inadequate | NR | No | |||
| Kalisch, Lee [46] | Doubtful | 1758 | Very good | - EFA and CFA: loadings 0.69–0.41; CFI 0.88 RMSEA 0.05 |
Very good | + α 0.85–0.74 | Yes | Reliability criterion validity hypothesis testing |
Doubtful very good Doubtful | + ICC2 > 0.84 + Pearson r 0.76 + KG |
| Kennerly, Yap [47] | Inadequate | 340 | Very good | - EFA and CFA: loadings 0.90–0.51; CFI 0.94 RMSEA 0.06 |
Very good | - α 0.93–0.60 | No | |||
| Klingle, Burgoon [60] | Inadequate | 1829 | Doubtful | - NR | Very good | ? α 0.87–0.81 | Yes | Hypothesis testing |
Doubtful | ? KG |
| Kobuse, Morishima [61] | Doubtful | 2924 | Adequate | - EFA loadings 0.87–0.28 | Very good | + α 0.82–0.75 | Yes | Hypothesis testing |
Inadequate | ? KG |
| Kramer and Schmalenberg [48] | Adequate | 3602 | Adequate | - EFA loadings 0.83–0.34 | Very good | - α 0.94–0.69 | Yes | Reliability hypothesis testing |
Doubtful very good |
? r range 0.88–0.53 + KG |
| Lake [49] | Inadequate | 2299 | Adequate | ? EFA loadings: 0.73–0.40; | Very good | + α 0.84–0.71 | Yes | Reliability hypothesis testing |
Inadequate very good |
+ ICC1 0.97–0.86 + KG |
| Li, Lake [50] | OP | 2000 | Adequate | - EFA loadings > 0.70 | Very good | + α 0.92–0.84 | No | |||
| Mays, Hrabe [51] | Inadequate | 210 | Adequate | - EFA loadings 0.87–0.57 | Doubtful | + α 0.89–0.75 | Yes | Hypothesis testing |
Doubtful | ? KG |
| McCusker, Dendukuri [62] | Inadequate | 121 | Adequate | - EFA loadings 0.79–0.40 | Very good | - α 0.88–0.64 | Yes | Hypothesis testing |
Adequate | ? OOM |
| McSherry and Pearce [63] | OP | 98 | Doubtful | - EFA loadings 0.92–0.17 | Doubtful | + α 0.78–0.71 | No | |||
| Pena-Suarez, Muniz [64] | Inadequate | 3163 | Very good | - EFA and CFA: loadings 0.77–0.41; CFI 0.85 RMSEA 0.06 | Doubtful | - α total scale 0.97 | Yes | Cross-cultural validity | Inadequate | - DIF NR |
| Rafferty, Philippou [65] | Adequate | 1705 | Adequate | - EFA loadings 0.87–0.40 | Very good | + α 0.93–0.70 | No | |||
| Reid, Courtney [52] | OP | 639 | Very good | - EFA and CFA: loadings 0.88–0.40; CFI 0.91 RMSEA 0.06 | Very good | - α 0.89–0.66 | Yes | Hypothesis testing |
Doubtful | ? KG |
| Saillour-Glenisson, Domecq [66] | Inadequate | 859 | Doubtful | - EFA and CFA: loadings, CFI and RMSEA NR | Very good | - α 0.91–0.53 | Yes | Reliability | Inadequate | - ICC range NR |
| Schroder, Medves [67] | Doubtful | 111 | Inadequate | - CFA for each factor range CFI 0.99–0.94 range RMSEA 0.13–0.04 | Very good | - α 0.89–0.67 | No | |||
| Siedlecki and Hixson [68] | Inadequate | 1332 | Adequate | - EFA loadings 0.91–0.71 | Very good | + α 0.89–0.73 | Yes | Hypothesis testing |
Inadequate | ? KG |
| Stahl, Schirmer [72] | Adequate | 1692 | Adequate | - EFA loadings 0.80–0.30 | Very good | - α 0.90–0.50 | Yes | Hypothesis testing |
Inadequate | ? OOM |
| Upenieks, Lee [69] | Doubtful | 464 | Very good | + CFA: CFI = 0.98 RSMEA 0.06 | Yes | Hypothesis testing |
Very good | ? OOM Pearson r 0.52–0.72 |
||
| Whitley and Putzier [53] | Inadequate | 245 | Inadequate | - NR | Very good | + α 0.87–0.72 | No | |||
| Wienand, Cinotti [70] | Doubtful | 8681 | Adequate | - EFA loadings 0.78–0.38 | Very good | + α 0.95–0.76 | Yes | Hypothesis testing |
Very good | ? KG |
NR: Not reported, KG: known groups, OOM: other outcome measurement, OP: other publication, LV: long version, SV: short version, EFA: exploratory factor analysis, CFA: confirmative factor analysis, CFI: comparative fit index, RMSEA: root-mean-square error of approximation, DIF: differential item functioning and ICC: intraclass correlation coefficient.