Table 2.

Communication and computation delays in each layer.

Layer (Input Size)	Operator	Stride	PWC+DWC		PWC ${}^1$
			Communication Delay (us)	Computation Delay (us)	Communication Delay (us)	Computation Delay (us)
224 × 224 × 3	Conv2d	2	N/A	N/A	24.56	405.81
112 × 112 × 32	Bottleneck	1	31.40	586.67	24.56	404.49
112 × 112 × 16	Bottleneck	2	34.01	1296.53	65.02	270.47
		1	30.65	600.87	108.00	430.33
56 × 56 × 24	Bottleneck	2	30.65	522.08	106.08	129.47
		1	24.07	265.79	127.01	150.92
28 × 28 × 32	Bottleneck	2	24.07	171.47	33.04	78.68
		1	16.55	209.50	65.68	144.35
14 × 14 × 64	Bottleneck	1	16.55	209.47	65.79	216.36
		1	24.71	412.38	98.43	314.83
14 × 14 × 96	Bottleneck	2	24.71	341.67	28.08	152.12
		1	21.43	309.20	46.32	244.99
7 × 7 × 160	Bottleneck	1	21.43	309.17	46.85	489.72
7 × 7 × 320	Conv2d	1	N/A	N/A	19.65	670.02
1280 × 1000	Conv2d	1	N/A	N/A	9.77	1345.96

¹ Include pooling or concatenation if necessary.