Last week I threw out AWK and replaced it with Ruby (Could've been Python, Perl or PHP even).

Because AWK is not suited for CSV. Please prove me wrong!

I had to parse 9million lines. Some of which contain "quoted records", others, same column, are unquoted. Some contain comma's, in the fields, most don't. CSV is like that: more like a guideline than actual sense.

Two hours of googling and hacking later, I gave up and rewrote the importer in Ruby, in under 5 minutes.

Lesson learned: I'll stay clear of AWK, when I know a oneliner of Ruby (or Python) can solve it just as well. Because I know for certain the latter can deal with all the edgecases that will certainly pop up.

To simplify working with CSV data using command line tools, I wrote csvquote ( https://github.com/dbro/csvquote ). There are some examples on that page that show how it works with awk, cut, sed, etc.