I am from McKinsey. I do not work on Kedro, but am a data scientist who has used it. I have mixed feelings on Kedro.

Pros:

* Forces data scientists to produce an end product that is not poorly organized Jupyter notebooks.

* Data Catalog is good for well structured systems

* Pipeline visualization stack is great (Kedro viz)

* Config options are pretty good

* Seems stable. Dev team is pretty good on this and avoiding breaking changes.

Cons:

* Data catalog is kind of bad for any non structured setup with flat file data with manual file movement (which is bad to begin with but sometimes that’s life)

* Productivity of making brand new data science code seems to drop when data scientists leave notebooks and

* Most of the time I get brought into a client context because the client doesn’t know anything about data scientist. A lot of data scientists, from both parties, come from academic backgrounds and aren’t great at code. The nice thing about notebooks is that they run. Kedro requires you to create pipeline and node objects to wrap around your code before it runs. It requires some familiarity with Kedro to understand, run, or modify. This makes it seem like a bad idea to dump on a novice client. If the data scientist on their side inheriting it doesn’t really get it, or leaves, there’s unlikely to be enough internal knowledge to maintain it. I try to avoid pushing any new tech stacks on my clients where I can for this reason.

So… I like it but don’t love it for consulting work, which is ironic.

A few years ago, I started working as a data scientist at a big financial firm and reviewed all workflow orchestrator available tools (including Kedro). I didn't like that all of them forced me to re-write my Jupyter code into their frameworks (they're supposed to make me more productive, not less).

True, notebooks have their issues but they can be fixed (I don't buy that "Jupyter is only for prototyping argument"). So, long story short, I started a project with a friend that makes us more productive by fixing the problems that notebooks' problems. https://github.com/ploomber/ploomber