Always bugged me that highly repetitive logs take up so much space!
I'm curious, are there any managed services / simple to use setups to take advantage of something like this for massive log storage and search? (Most hosted log aggregators I've looked at charge by the raw text GB processed)
Check out https://github.com/parseablehq/parseable ... we are building a log storage and analysis platform in rust. Columnar format helps a lot in reducing overall size but then you have little computational overhead to deal with conversion and compression. This trade off will be there but we are discovering ways to minimise it with rust