Make is an absolutely wonderful, wonderful tool.

Most of the common criticisms of Make are actually criticisms of autoconf, which I agree is a hideous tool. (On the other hand, autoconf is a tool intended to address a hideous problem, so perhaps that's inevitable).

A lot of the more recent build tools I see are largely reinventions of Make. Make is very widely supported (most basic projects can make do[0] with portable Makefiles, though GNU Make is also available for most systems), and its syntax is actually very easy to grasp and manipulate[1].

[0] no pun intended

[1] Most projects only need a very small subset of what Make has to offer, anyway, and that can be learned in a matter of minutes.

Make is a great tool for the intended uses. Unfortunately, many uses of Make-like tools also require the unintended uses; my favorite example is the auto-dependencies, which is required by most compilation tasks and still tedious to get it right [1]. It also (mostly) lacks modern programmable interfaces, which led to Makefiles laden with hacks and mumbo-jumbos. At least for the compilation tasks, I see Make is clearly being outdated.

[1] Something like http://mad-scientist.net/make/autodep.html

The auto-dependency problem is a perfect example of how there is room for improvement in make-land. When something that basically everyone wants to do takes a long web page to describe a complicated method that sort of works, that is a compelling indicator that there must be a better way.

Incidentally, I am working on a make-replacement that is intended to address this and similar needs. It's still in very early stages, but I think I'm onto something.

The idea is to write a very low-level tool that leaves out most of Make's higher-level abstractions. In my tool there are no recipes, no implicit rules, no variables or variable substitutions, no conditionals, etc. The input to my tool is just the precise specifications of the commands you need to run, their inputs and outputs (so a precise dependency graph can be calculated), and with everything fully-expanded already.

The idea, then, is that whatever higher-level abstractions you want (if any) you build into a higher-level tool. The higher-level tool just spits out a file describing the list of tasks. Then the higher-level tool can worry about the higher-level structure, policy, configuration, etc. of your project. So instead of writing things like implicit rules in Makefiles, you just write some code that explicitly generates tasks.

For example, with Make, you might write an implicit rule like this:

     %.o : %.c\n             $(CC) -c $(CFLAGS) $(CPPFLAGS) $< -o $@\n
\nThen Make magically decides which output files match this implicit rule. My idea is that, instead of this, explicitly apply your rules to your inputs. Your build system could instead be a Ruby/Python/etc. script that looks something like this:

    for c_file, o_file in files:\n      tasks.append(Task(\n        target=o_file,\n        source=c_file,\n        command="gcc -c %s -o %s", c_file, o_file\n      ))\n\n    print(tasks)\n
\nI think this is much more convenient than having to program in Make and learn its quirky abstractions.

I have an elegant solution for the auto-dependency problem (unproven, but I think it's promising). The idea is that your dependency-calculating tasks are still just tasks, but the tool knows how to integrate the calculated dependencies back into the overall dependency graph. These dependency-generating tasks depend on the files they are generating dependencies for, so it is all part of the unified dependency graph.

If you're interested, star my project and follow my blog (where I will make any announcements about it):

https://github.com/haberman/taskforce

http://blog.reverberate.org/

You should check out djb's redo [1], specifically the implementation by apenwarr [2]. It's close to the separation of concerns you're looking for: programs written in your language of choice enumerate the dependency graph, an external tool collects them and executes its way through the dependency graph towards your targets.

I agree with you that Make's pattern matching facilities are ugly and could be improved upon. I tried something of the sort [3] with redo, and it worked well, but felt too alien.

From what I've seen at big companies with huge builds, the "dependency graph itself changed, now what do we rebuild" problem is the killer problem. I bet you've seen instances of that at Google.

[1] http://cr.yp.to/redo.html

[2] https://github.com/apenwarr/redo

[3] https://news.ycombinator.com/item?id=7464047

See also remake. It's a bit simpler, I think.

https://github.com/rocky/remake