I have bewilderingly tried to discern why software development continues to grow more and more complex.

It wasn’t always like this. There was a time when we talked about languages and OSes and libraries as if they made a difference on how much you could get done with as little people and cognitive load as possible (the claims were very much overrated, but the point was we acted like it mattered).

And then it started ballooning. It seems to me that much of that coincides with a lot of new money being dumped into the economy, and software moving past the necessary-evil-so-how-do-I-drive-down-my-costs and on to the gold rush of you must have a web app for your service that rivals a video game in complexity and visual finesse. It seems to me that as long as their is so much free money sloshing around venture capital funding, that it was inevitable that the process of making software would complexify to soak up the extra cash. After all, you can only add so many levels and varieties of managers to add value. After that comes the variegation in roles of software development.

That’s my working theory anyway.

My theory is we need to add "software history" to computer science education programs. How many developers whose careers began after 2000 have ever hand written or know the power of a Makefile? I've worked at research labs and major animation studios that back in the 90's the entire infrastructure was fully automated thru Makefiles, and it purred like a well fed cat. There are hundreds of thousands of forgotten perfectly fine software tools, ignored simply because our industry has absolutely no respect for it's own history or lessons painfully learned beyond pulling up disparaging stories for new marketing angles.

As projects grow the Makefile becomes pretty much a big bash script, but with a specific syntax that I always struggle with.

Unrelated to this topic, I invite you take a look at my project which a tool purpose-built to be a better version of what your makefile became:

Run: Task runner that helps you easily manage and invoke small scripts and wrappers

https://github.com/TekWizely/run

Defining commands feels like make, but comes with a bunch of extras targeted at the needs of a task-runner.

I hope you'll check it out!