What does HackerNews think of visidata?
A terminal spreadsheet multitool for discovering and arranging data
For my toolbox I include jq, gron, miller, VisiData, in addition to classics like sed, awk, and perl.
- https://github.com/saulpw/visidata - http://visidata.org/
Also there is a great introduction: - https://jsvine.github.io/intro-to-visidata/ "Intro to VisiData Tutorial" by Jeremy Singer-Vine
It is not specific to any tool, so you can use command line tools, the TUI data explorer VisiData[2] or whatever data tools you want to solve the puzzle in your terminal.
[1] https://hanukkah.bluebird.sh/5783/ [2] https://github.com/saulpw/visidata
If I'm trying to understand some new json data, I start with visidata, and then I poke around to find the data I want. If it is data I want to extract again, I use jq to get the interesting bits. Sometimes I use that in combination with sqlite-utils to store that data so I can query it. (I haven't tried some of the other tools that will create a sqlite database for you.)
- https://github.com/saulpw/visidata - https://github.com/simonw/sqlite-utils
There are some recent Unix shells, like nushell for working with structured data, but I haven't used them.
- https://github.com/saulpw/visidata
- https://github.com/johnkerl/miller
- https://github.com/nushell/nushell
I use to be very comfortable using awk/sed/perl/sort/uniq/tr/tail/head from the CLI for the sort of data cleaning this article is talking about. However, over the past year I've found I use VisiData https://github.com/saulpw/visidata for interactive work.
If I need to clean up the data first, I'll use mlr or jq as input to Visidata. If my data is too dirty for mlr, then I'll use Unix toolbox tools mentioned as input to mlr, jq or VisiData.
VisiData provides some ability to script, but when possible I prefer to have the shell do the scripting with all the tools mentioned as input to Visidata.
https://github.com/sharkdp/fd
https://github.com/sharkdp/bat
https://github.com/sharkdp/diskus
https://github.com/sharkdp/hyperfine
For viewing csv's as tabular, paged data from the terminal, check out VisiData (python) https://github.com/saulpw/visidataYou can totally join two 'sheets' by primary key and save and replay that workflow, I think you may have to write a python function to iterate over the sets of columns to look for differences. Not sure if there's a cleaner way