[1] https://github.com/common-workflow-language/common-workflow-...
* looks like yaml
* has curly braces to look programmery
* whitespace might be meaningful
* has pipes like a bash script
https://github.com/common-workflow-language/common-workflow-...
mea culpa: The above was based on a first look at something titled "A DSL for parallel and and scalable computational pipelines"a as opposed to "Java workflow manager with Groovy language scripting." The presented screenshot still looks to me like an unholy union of yaml, js, py and sh. If that sounds groovy to you; have fun.
https://github.com/common-workflow-language/common-workflow-...
Also, whenever these conversation of flow-based / piplining tools come up, I always like to point people to Common Workflow Language to remind people that there is an attempt at standardizing workflow descriptions so that they can be used with different packages:
Where each container is described by what types of things it takes as inputs and what it outputs -- ie as functions.
In practice we've built hundreds of these and composed them in endless variety for a large range of genomics tasks, processing petabytes of data at Seven Bridges Genomics.
The big win I've found with Nextflow is that once you've written a workflow, you have a lot of flexibility in the execution environment: Have all the tools already installed on your workstation or large compute instance? Use the local executor to saturate the box with concurrently running jobs. Don't have or want all those tools installed? Use the local executor with Docker images. Have access to a traditional compute cluster (e.g. LSF, SGE, Torque, etc.)? Use the cluster executor with Docker images.
A couple other resources worth checking out:
Toil workflow engine https://github.com/BD2KGenomics/toil
Common Workflow Language (CWL) specification https://github.com/common-workflow-language/common-workflow-...
How does this compare to Arvados (http://arvados.org)?
You might be interested in the common workflow language effort which among other goals is developing a multi-vendor standard for wrapping analysis tools using Docker (https://github.com/common-workflow-language/common-workflow-...)