Table 1: Application Datasets:
Dataset | COLA (Matthews Corr. [23]) | RTE (F1 Score) | CYDET (F1 Score) | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Param Inc. |
Overall (std) | Slice Lift | Param Inc. |
Overall (std) | Slice Lift | Param Inc. |
Overall (std) | Slice Lift | ||||
Max | Avg | Max | Avg | Max | Avg | |||||||
Vanilla | - | 57.8 (±1.3) | - | - | - | 67.0 (±1.6) | - | - | - | 39.4 (±-5.4) | - | - |
HPS [7] | 12% | 57.4 (±2.1) | +12.7 | 1.1 | 10% | 67.9 (±1.8) | +12.7 | +2.9 | 10% | 37.4 (±3.6) | +6.3 | −0.7 |
Manual | 12% | 57.9 (±1.2) | +6.3 | +0.4 | 10% | 69.4 (±1.8) | +10.7 | +4.2 | 10% | 36.9 (±4.2) | +6.3 | −1.7 |
MoE [17] | 100% | 57.2 (±0.9) | +20.0 | +1.3 | 100% | 69.2 (±1.5) | +10.9 | +3.9 | 100% | OOM | OOM | OOM |
Ours | 12% | 58.3 (±0.7) | +19.0 | +2.5 | 10% | 69.5 (±0.8) | +10.9 | +4.6 | 10% | 40.9 (±3.9) | +15.6 | +2.3 |