Datasette is pretty cool.

But AFAICT, it just doesn’t scale whatsoever. That SQLite db is both the dataset index and the dataset content combined, right? So you're limited by how big that SQLite db can realistically be. The docs say "share data of any shape or any size", but AFAICT it can't handle large datasets containing large unstructured data like images and video and multi-billion data point datasets are hard to store in a single machine/file.

Not really a criticism, but more wondering if there are scale optimizations in Datasette I'm not aware of since the docs do say any shape or size.

You're right, Datasette isn't the right tool for sharing billion point datasets (actually low-billions might be OK if each row is small enough).

I think of Datasette as a tool for working with "small data" - where I define small data as data that will fit on a USB stick, or on my phone.

My iPhone has a TB of storage these days, so small data can get you a very long way!

Using it for unstructured image and video would work fine using the pattern where those binary files live somewhere like S3 and the Datasette instance exposes URLs to them. I should find somewhere in the documentation to talk about that.

But yes, I should probably take "of any size" off the homepage, it does give a misleading impression.

I decided to just drop "any size" but keep "any shape".