What does HackerNews think of dagster?

An orchestration platform for the development, production, and observation of data assets.

Language: Python

#124 in Python
Show HN: Airflow is cool but have you tried this for data pipelines? | Sep 2022
This is cool, but looks like https://github.com/dagster-io/dagster

The issue with less popular data pipeline projects is that they’re less stable in production

Introduction to Apache Airflow | May 2020
Expand Context ↕
There are too many different tools in the space. I've been heavily researching workflow / ETL frameworks this week, and even after culling the ones that seemed like poor fits, I'm still left with:

- https://github.com/getpopper/popper

- https://docs.pachyderm.com/

- https://github.com/lyft/flyte

- https://aws.amazon.com/step-functions/

- https://github.com/spotify/luigi

- https://docs.metaflow.org/

- https://github.com/dagster-io/dagster

- https://github.com/argoproj/argo

- https://github.com/prefecthq/prefect
Dolphin Scheduler | Dec 2019
The only other one I can think of off the top of my head is dagster: https://github.com/dagster-io/dagster

It’s made by Nick Schrock of graphql fame, among others. I’m sure there are 100s of these projects though.

Metaflow, Netflix's Python framework for data science, is now open source | Dec 2019
How does it compare to dragster.io?

https://github.com/dagster-io/dagster