To visualize data coming in from a pipe, can pipe it to
https://github.com/dkogan/feedgnuplot
Very useful in conjunction with other tools to provide filtering and manipulation. For instance (the first one is mine):
https://github.com/dkogan/vnlog
https://www.gnu.org/software/datamash/
https://csvkit.readthedocs.io/
https://github.com/johnkerl/miller
https://github.com/eBay/tsv-utils-dlang
https://github.com/BatchLabs/charlatan
https://github.com/dinedal/textql
https://github.com/BurntSushi/xsv
https://github.com/dbohdan/sqawk
https://github.com/dkogan/vnlog
It's tsv-utils-like, but is strictly a wrapper around existing tools. So filtering and transformations are interpreted literally as awk (or perl) expressions. And the various cmdline options match the standard tool options because they ARE the standard tools. So you get a very friendly learning curve, but something like tsv-utils is probably faster and probably more powerful. And it looks like tsv-utils references fields by number instead of by name. Many of the others (mine included) use the field names, which makes a MAJOR usability improvement.
Other tools in no particular order:
https://csvkit.readthedocs.io/
https://github.com/johnkerl/miller
https://github.com/eBay/tsv-utils-dlang
https://github.com/BatchLabs/charlatan
https://github.com/dinedal/textql
https://github.com/BurntSushi/xsv
https://github.com/dbohdan/sqawk
- https://github.com/dkogan/vnlog
- https://csvkit.readthedocs.io/
- https://github.com/johnkerl/miller
- https://github.com/BurntSushi/xsv
- https://github.com/eBay/tsv-utils-dlang
- https://stedolan.github.io/jq/
- https://github.com/BatchLabs/charlatan
- https://github.com/dinedal/textql
- https://github.com/dbohdan/sqawk
(disclaimer: vnlog is my tool)
$ ps aux | sqawk -output table \
'select user, round(sum("%mem"), 2) as memtotal
from a
group by user
order by memtotal desc' \
header=1
┌────────┬────┐
│dbohdan │67.1│
├────────┼────┤
│ root │3.5 │
├────────┼────┤
│ avahi │0.0 │
├────────┼────┤
│ daemon │0.0 │
├────────┼────┤
│message+│0.0 │
├────────┼────┤
│ nobody │0.0 │
├────────┼────┤
│ ntp │0.0 │
├────────┼────┤
│ rtkit │0.0 │
├────────┼────┤
│ syslog │0.0 │
├────────┼────┤
│ uuidd │0.0 │
├────────┼────┤
│whoopsie│0.0 │
└────────┴────┘
It parses DSV data like Awk does, runs SQL queries against it and formats the output in one of several ways. An example I am particularly fond of is using this tool as a poor man's libxo (https://github.com/Juniper/libxo):
$ ps | sqawk -output json,indent=1 'select PID,TTY,TIME,CMD from a' trim=left header=1
[{
"PID" : "3947",
"TTY" : "pts/2",
"TIME" : "00:00:07",
"CMD" : "zsh"
},{
"PID" : "15951",
"TTY" : "pts/2",
"TIME" : "00:00:00",
"CMD" : "ps"
},{
"PID" : "15952",
"TTY" : "pts/2",
"TIME" : "00:00:00",
"CMD" : "tclkit-8.6.3-mk"
}]
The rec format is interesting because it is a specification for what others have used before without one. It seems quite readable and does not suffer from the ",,,,,," problem of CSV.
[1] Here's the list of ones I have run into so far:
* https://github.com/harelba/q
* https://github.com/tobimensch/termsql
* https://github.com/samuel/squawk
* https://github.com/tjunier/sqawk
I've also written one: https://github.com/dbohdan/sqawk. It turned out I wasn't the only one to come up with the name.
I've also written an Awk-like tool in Tcl that speaks SQL and has table joins - https://github.com/dbohdan/sqawk.