| 1: | Set , and . |
| 2: | Distribute the initial settings to the workers |
| 3: | repeat |
| 4: | for all in parallel do |
| 5: | |
| 6: | Send to master |
| 7: | end for |
| 8: | Collect and from workers and broadcast the update v |
| 9: | |
| 10: | for all in parallel do |
| 11: | |
| 12: | end for |
| 13: | |
| 14: | until acceptance of stopping criterion¶ |
The iteration is terminated if .