What does HackerNews think of absurd-sql?
sqlite3 in ur indexeddb (hopefully a better backend soon)
Something relevant is absurd-sql, but unfortunately that's not even close to production ready :/
The article briefly mentioned about Dream [2] web framework but sadly, the author has stopped maintaining the library and noone has stepped up yet. So the whole web story with OCaml isn’t good yet.
[1]: https://github.com/jlongster/absurd-sql [2]: https://aantron.github.io/dream/
However, the new File System Access apis (https://developer.mozilla.org/en-US/docs/Web/API/File_System...) that are landing in browsers will fix this. One of the things it does is enable very efficient block level read/write access to a privet sandboxed filesystem for the websites origin, perfect for persistent sqlite. There is more here: https://web.dev/file-system-access/#accessing-files-optimize...
The other API worth knowing about is the more direct File System Access API, which is the one that allows direct access now: https://developer.mozilla.org/en-US/docs/Web/API/File_System...
wrt SQLite, icmyi: https://github.com/jlongster/absurd-sql
Assuming this is normal WASM SQLite any persistence will be by flushing to LocalStorage/IndexedDB. There won't be any ACID compliance.
There is a project called "Absurd SQL" to back SQLite with a custom block based FS on top of IndexedDB. It is somewhat absurd but it works incredibly well!
https://github.com/jlongster/absurd-sql
I think combining WASM SQLite with the session extension (https://www.sqlite.org/sessionintro.html) would be a super interesting way to build an eventually consistent datastore for web apps. You could do all transactions locally and sync back and fourth just the changes.
The one I am most excited by is for persistent SQLite with proper acid transaction in the browser, not having to load the whole db into memory. Absurd SQL [0] currently does this by creating a VFS on top of IndexedDB. This would let it do it properly, and is likely to be upstreamed to SQL.JS which is the main SQLite WASM project.
0: https://github.com/jlongster/absurd-sql
Many new in browser DB engines are going to get built on top of this. Others that I could see happing are:
- Relm from MongoDB being ported to WASM and use this for storage.
- If I were Supabase I would be looking to create a “Mini Supabase” for mobile, and make it work in browser too.
- Couchbase Mobile as an alternative to PouchDB
I see there is concern about Asyncify. I believe in the browser there is a way around this using Atomics and a shared buffer (Absurd SQL[0] uses it), but suspect this isn't possible in CloudFlare Workers as it is working across the worker/main thread boundary? Is it possible to launch another worker from a CloudFlare worker, in the same process, like a web worker?
I am somewhat hoping that CloudFlare have something up their sleeve to make these sorts of projects easer, maybe even some sort of distributed block store.
Combine it with the various JavaScript ORMs and you have a nice developer UX.
I’m waiting for someone with more time than myself to build a syncing feature on top of SQLites Sessions[2] so that changes locally are synced back to the server.
(Feel like I’m a cheer leader posting this comment every week)
SQL.js[0] and the incredible “Absurd SQL”[1] are making it possible to build PWAs and hybrid mobile apps with a local SQL db. Absurd SQL uses IndexedDB as a block store fs for SQLite so you don’t have to load the whole db into memory and get atomic writes.
Also I recently discovered the Session Extension[2] which would potentially enable offline distributed updates with eventual consistency!
I can imagine building a SAAS app where each customer has a “workspace” each as a single SQLite db, and a hybrid/PWA app which either uses a local copy of the SQLite db synced with the session extension or uses a serveless backend (like CloudFlare workers) where a lightweight function performs the db operations. I haven’t yet found a nice way to run SQLite on CloudFlare workers, it need some sort of block storage, but it can’t be far off.
It's worth noting that this loads the entire db into memory, and only saves to the filesystem when you tell it to (not on each transaction) so you could loose changes on a crash.
There is a brilliant project to add true transactional flushing and a block like storage backend to WASM Sqlite.js called "Absurd SQL", worth checking out. It's currently built on top of IndexedDB but they are working with the WG designing the FileSystemAccess APIs to ensure it has suitable block level support and locking for this type of tool.
0: https://github.com/jlongster/absurd-sql
1: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
2: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
If you are building an offline enabled web app for the cost of a 1mb wasm download it’s worth it for full sql support and the speed increase.
I think there is a good chance Absurd-SQL will get folded into SQLite.js.
Based on the series of blog posts / documentation opinion pieces that have been posted so far, I'm quite interested in playing around with this despite being mostly in the relational SQL camp. Everything I've read is thoughtful, well reasoned, and rather practical and the author is exploring a rather interesting problem space.
I'd love to see a mashup of RxDB[1] and absurd-sql[2] that brings a distributed SQL datastore to the browser.
I'm excited about https://github.com/jlongster/absurd-sql though.
There is also alasql[3] which is implemented in js, and lovefield[2] from Google which seems like an experiment that is now abandoned.
First, you could implement a REST/GraphQL cache in SQL. This would require maintaining mappings of your API response fields to SQL tables.
Going further, you could implement your backend API on the client, and have requests to it go directly to your browser SQL db. The benefit of this is you write your API once, and you get full offline support. If you don't need "local-first" then it's just a matter of figuring out when a query becomes invalid on the server. I could show instant results from local db, and then send a query to the server to ask if the data I already have in my cache can fulfill this request. Could optimize this on the server by listening to DML queries via WAL/LISTEN. WebSockets would be used to maintain real-time updates to only the data you are viewing.
You could also just use SQL directly as your UI API and then trigger these queries on the backend (respecting security of course).
What's doesn't feel optimal though is subscribing to sqlite.js updates in the browser. This makes me feel like we need an SQL db written in JS.
Also, if our DB is running in the same execution environment as our frontend and has one consumer, we could store each row as a JS object, and then reference this directly in our views, and subscribe to updates on a row-by-row basis. So if you are rendering a large table, if a column in a single row changes, when this row is updated in the database, instead of re-rendering the entire table manually, (or smartly detecting if the query was invalidated), you just bind to updates on the row instance, which is what the db is actually storing. I think this would reduce huge amounts of code.
Local-first is a little more difficult. There is some good exploration here[4].
I think writing a db in JS on the backend is not such a bad idea either. The hot paths would be written as Rust addons, but the majority of stuff people want today is just implementing the right logic to automatically handle denormalizations and subscriptions which we already hack together in our application layer using JS et al.
[1]: https://github.com/jlongster/absurd-sql
[2]: https://github.com/google/lovefield
[3]: https://github.com/agershun/alasql
[4]: https://actualbudget.com/blog/