SQL is having somewhat of a moment in the bigdata world, thanks in part to 'modern datastack' and new age datawarehouses like snowflake,bigquery.

However there are a lot of pushback from 'traditional' dataengineers who were trained on spark/scala. Its bit of hardsell to go from a highly typed language to a free for all text based logic.

I think the following is needed for sql to be finally accepted as 'serious' contender.

create compiled sql language ( not pandas)

1. that compiles to sql and addresses some of the issues bought up in the post like nested aggregations.

2. make code reusable. Eg: apply year over year growth to a table that has the requisite columns. Compiler should check this in ide.

3. make materializations first class concept in the language. No seperate dbt layer.

4. crate a way to package and distribute libraries that you can import into your project .

5. a unit testing framework that makes it easy to test the logic without having to setup test tables in the database.