What does HackerNews think of clj-3df?

Clojure(Script) client for Declarative Dataflow.

Language: Clojure

#15 in Clojure
Mainly it is because the tar pit paper proposed the relational algebra in its functional relational programming.

I do think for some it might be easier to reason mechanically about dataflow through collection-oriented operators. But I suppose it's subjective.

clj-3df [1] to my understanding does something like Relic for datomic datalog using differential dataflow.

Disclaimer: I now work on XTDB [2], so datalog is somewhat now my day job.

[1] https://github.com/sixthnormal/clj-3df

[2] https://xtdb.com/

I'm hedging my bets by using Datalog and the Datomic protocol wherever I can: https://tonsky.me/blog/datomic-as-protocol/

And keeping a close eye on Differential Dataflow: https://github.com/TimelyDataflow/differential-dataflow/

DD is commercialized in Materialize: https://materialize.io/

The solution is likely to arrive in the Clojure/ClojureScript ecosystem before it arrives anywhere else because tools like Datascript lend itself to this type of thinking, e.g. 3DF: https://github.com/sixthnormal/clj-3df/

The closest thing we have to streaming updates right now is FactUI, which uses the Clara rules engine, but it's IMO hampered by limitations of Clara and Cljs namespacing: https://github.com/arachne-framework/factui

The RETE algorithm is a less general solution to incremental updates. Zach Oakes is doing some interesting work with O'Doyle Rules: https://github.com/oakes/odoyle-rules

1. You have to register specs for each kind of document. For example, from the frontend you can subscribe to a user document of a given ID with `{:table :users :id #uuid "some-uuid"}`. The backend will: (1) look up the document with the given ID, (2) verify that you've registered a spec for the `:users` "table" and verify that the document meets that spec, (3) run the authorization rule to make sure the client has access to that user document.

2. It is inherently difficult. The closest thing like this that exists for datalog is I believe clj-3df[1]. Hence I was very excited when Materialize was launched publicly a few months ago. A version for datalog would be cool too, but SQL is good enough I think.

[1] https://github.com/sixthnormal/clj-3df

Any discussion about incremental UI should include Yaron Minsky's talk about Jane Street's OCaml framework, incr_dom from 2016: https://www.youtube.com/watch?v=R3xX37RGJKE

Incremental is essentially an implementation of Adapton from 2014: https://www.cs.umd.edu/~hammer/adapton/

There is a big upset coming in the UX world as we converge toward a generalized implementation of the "diff & patch" pattern which underlies Git, React, compiler optimization, scene rendering, and query optimization.

Two sleeper startups in this space that are going to make a lot of money are Frank McSherry's Materialize Inc. and Nathan Marz's Red Planet Labs.

incr_dom source: https://github.com/janestreet/incr_dom

Differential Dataflow: https://github.com/TimelyDataflow/differential-dataflow

Declarative Dataflow Client: https://github.com/sixthnormal/clj-3df

Crux adapter for Declarative Dataflow: https://github.com/sixthnormal/clj-3df

This could be cool, I was sad when nothing came of the old dataflow project in Clojure contrib. Readers might also be interested in:

https://github.com/sixthnormal/clj-3df

Which offers Datalog queries that deliver you fast, minimal diffs of data upon updates from external sources (e.g. Kafka). Build on differential dataflow, which is very cool.