Table 4.
T1E and Pow for unadj tests, multiplicity adj through the multivariate normal-based closed test (adj) and Holm tests, observed in 50,000 simulation runs for scenarios 4–6. Rows labelled ‘Any’ refer to the probability of rejecting the null hypothesis for at least one included parameter (i.e. family-wise T1E or Pow). Further rows show the rejection probability for each specific parameter included in the parameter set. The sample size in the simulation was 200 subjects per group. Values are presented as per cent.
| Scenario | Set | Parameter | T1E unadj | T1E adj | T1E Holm | Pow unadj | Pow adj | Pow Holm |
|---|---|---|---|---|---|---|---|---|
| 4 | 5 | Any | 6.41 | 2.46 | 1.74 | 94.6 | 88.4 | 85.2 |
| S 1 | 2.39 | 0.94 | 0.73 | 66.8 | 62.6 | 61.3 | ||
| S 2 | 2.61 | 1.05 | 0.73 | 83.5 | 77.1 | 74.9 | ||
| S 3 | 2.56 | 1.01 | 0.76 | 83.0 | 77.0 | 75.3 | ||
| Score test | 2.45 | 0.99 | 0.70 | 91.6 | 85.0 | 81.7 | ||
| 6 | Any | 6.24 | 2.31 | 1.60 | 94.5 | 88.1 | 84.8 | |
| cloglogS 1 | 2.31 | 0.88 | 0.66 | 66.1 | 61.7 | 60.4 | ||
| cloglogS 2 | 2.55 | 0.99 | 0.69 | 83.2 | 76.5 | 74.1 | ||
| cloglogS 3 | 2.47 | 0.93 | 0.69 | 82.7 | 76.5 | 74.7 | ||
| Score test | 2.45 | 0.98 | 0.69 | 91.6 | 84.9 | 81.6 | ||
| 7 | Any | 3.27 | 2.40 | 1.12 | 92.8 | 90.9 | 84.7 | |
| avgHR | 2.47 | 1.98 | 1.04 | 90.6 | 89.3 | 83.8 | ||
| RMST | 2.50 | 2.00 | 1.04 | 90.0 | 88.9 | 83.9 | ||
| Score test | 2.45 | 1.97 | 1.07 | 91.6 | 90.2 | 84.5 | ||
| 5 | 5 | Any | 6.30 | 2.58 | 1.79 | 83.9 | 71.7 | 65.9 |
| S 1 | 2.58 | 1.10 | 0.80 | 63.6 | 54.9 | 51.9 | ||
| S 2 | 2.49 | 1.08 | 0.76 | 64.6 | 55.3 | 51.3 | ||
| S 3 | 2.60 | 1.11 | 0.79 | 54.0 | 46.9 | 44.2 | ||
| Score test | 2.45 | 1.07 | 0.74 | 76.7 | 65.3 | 59.7 | ||
| 6 | Any | 6.06 | 2.37 | 1.63 | 83.6 | 71.1 | 65.2 | |
| cloglogS 1 | 2.51 | 1.04 | 0.75 | 63.1 | 54.2 | 51.1 | ||
| cloglogS 2 | 2.46 | 1.04 | 0.72 | 64.3 | 54.7 | 50.7 | ||
| cloglogS 3 | 2.39 | 0.93 | 0.66 | 53.5 | 46.1 | 43.4 | ||
| Score test | 2.45 | 1.07 | 0.73 | 76.7 | 65.3 | 59.5 | ||
| 7 | Any | 3.53 | 2.40 | 1.26 | 79.9 | 75.1 | 65.2 | |
| avgHR | 2.49 | 1.88 | 1.12 | 71.6 | 69.6 | 63.3 | ||
| RMST | 2.49 | 1.88 | 1.03 | 77.0 | 73.4 | 64.1 | ||
| Score test | 2.45 | 1.85 | 1.01 | 76.7 | 73.0 | 64.1 | ||
| 6 | 5 | Any | 7.16 | 2.58 | 1.97 | 98.1 | 94.6 | 93.2 |
| S 1 | 2.42 | 0.84 | 0.68 | 90.4 | 85.4 | 84.5 | ||
| S 2 | 2.63 | 0.95 | 0.73 | 88.0 | 82.2 | 81.1 | ||
| S 3 | 2.67 | 1.03 | 0.78 | 56.8 | 54.1 | 53.6 | ||
| Score test | 2.51 | 0.93 | 0.73 | 96.7 | 92.7 | 91.0 | ||
| 6 | Any | 6.58 | 2.17 | 1.65 | 98.1 | 94.4 | 93.0 | |
| cloglogS 1 | 2.35 | 0.77 | 0.64 | 90.1 | 84.8 | 83.8 | ||
| cloglogS 2 | 2.57 | 0.89 | 0.68 | 87.7 | 81.7 | 80.6 | ||
| cloglogS 3 | 2.11 | 0.65 | 0.48 | 56.4 | 53.4 | 52.9 | ||
| Score test | 2.51 | 0.92 | 0.72 | 96.7 | 92.6 | 91.0 | ||
| 7 | Any | 3.36 | 2.44 | 1.16 | 98.0 | 97.3 | 94.7 | |
| avgHR | 2.52 | 2.03 | 1.10 | 97.5 | 97.0 | 94.6 | ||
| RMST | 2.54 | 2.01 | 1.04 | 97.4 | 96.8 | 94.3 | ||
| Score test | 2.51 | 1.97 | 1.05 | 96.7 | 96.3 | 94.3 |
T1E: type 1 error rate; Pow: power; unadj: unadjusted; adj: adjusted; Holm: Bonferroni–Holm adj; HR: hazard ratio; avgHR: average HR; RMST: restricted mean survival time; S 1: scenario 1; S 2: scenario 2; S 3: scenario 3; cloglogS 1: complementary log–log scenario 1; cloglogS 2: complementary log–log scenario 2; cloglogS 3: complementary log–log scenario 3.