Thinking over this, it just (finally) struck me that awk (and sed, grep, etc.) are basically just libraries (more like DSLs), and the shell is just another scripting language with a REPL/notebook environment.
When that's the case, what makes them particularly useful compared to some other language with a uniform syntax and possibly better forms of "glue" -- say Python, or lisp, or whatever? Why not allow people to acquire proficiency in just one language, and use it consistently to communicate with their computer? I don't think ubiquity/portability is a compelling answer, since (for many/most people who care about this) it is relatively easy to install the environment of their choice on their personal computing device. BTW, we must also strive to maintain that flexibility in the next generation of computing devices.
As an aside, folks who do like the task-focused nature of shell DSLs might appreciate language-oriented programming with Racket.
Command line tools are very terse, heavily optimized (ie fast), interactive, and easily chainable. I would never want to write a python program for quick analysis of a log file when I can grep/sed/awk/sort/uniq my way to a solution.
The other day my boss asked me to find out if there was any second where we processed more than 100 transactions a second on our server in the last 90 days. I needed to go through 100s of GB of data which mawk managed in no time and in a relatively simple one liner. I imagine my idiomatic python script would still be running
Often times, analysis is an evolving thing. As you uncover one answer, new questions arise. It’s far easier imo to tweak my command line pipelines than to modify program flow.
At the end of the day, general purpose programming languages are jacks of all trades and masters of none. They’re inherently held back by their flexibility when compared to DSLs. Text processing is so easy in awk b/c it’s a text processing domain specific language. If you really think that we should use on language for everything, you’re ignoring the value that specialization can provide you, whether in speed, power, or ease of use
Yeah but I would wager very few people actually know awk. I would also wager 90+% of awk programs are pasted directly from StackOverflow.
That's why I like the idea of pawk[1] - it's like awk, except with python syntax instead of some esoteric syntax that nobody knows.