Table 2: Model characterizations:
Method | Slice-aware | No manual tuning |
Weighted slice info. |
Avoids copies of model (M params) |
Num. Params |
---|---|---|---|---|---|
Vanilla | ✓ | ✓ | O(M+r) | ||
HPS | ✓ | ✓ | ✓ | O(M+kr) | |
Manual | ✓ | ✓ | ✓ | O(M+kr) | |
MoE | ✓ | ✓ | ✓ | O(kM+kr) | |
Ours | ✓ | ✓ | ✓ | ✓ | O(M+krh) |