Table 3. Popular workflow management systems.
| Comparison aspect | Swift/T [15] | NextFlow [67] | Galaxy [69] | Kepler [70] |
|---|---|---|---|---|
| Nature | WL† and execution engine | WL and execution engine | Web interface | WL and execution engine |
| Support community standard WL? | No | No | CWL | No |
| User interface | CLI | CLI, REPL [75], IDE [76] |
GUI | GUI, CLI, Jupyter notebooks |
| Programming paradigm [77] | Dataflow | Dataflow | Sequential [78] | Sequential, dataflow, process network or continuous time [79] |
| Containerization support | None | Docker, Singularity |
Docker, Singularity |
Docker |
| Scalability [80] | Extreme scale [81] | Yes | Complicated ‡ [69] | Yes |
| Checkpointing and caching | No | Yes | Yes | Yes |
| Portability ¶ | Cray aprun, LSF | LSF, NQSII, HTCondor, Kubernetes, Ignite, DNAnexus |
LSF, HTCondor, Galaxy Pulsar [82] XSEDE Jetstream [83] |
Open stack, Google cloud, Apache Mesos |
| Distributed execution | MPI-based | Apache Ignite/ MPI | Spark [84], Hadoop [85] | Spark, Hadoop |
| Supported compute architecture | Homogeneous | Homogeneous or heterogeneous | Not clear | Homogeneous or heterogeneous |
| Compute resource allocation | Reserved a priori | Reserved a priori | Multiple deployment strategies [86] | Allocated dynamically |
† WL = workflow language; REPL = Read-Eval-Print-Loop console; CLI = Command Line Interface; GUI = graphical user interface.
‡ Recent optimizations of Galaxy for User interface scalability and Server scalability enable analysis of large datasets for many users.
¶ All these workflow management systems can run on a single server, on clusters managed by PBS, Grid Engine, Slurm, and on AWS.