BTW, Citus license is GNU Affero General Public License (github lists “conditions: same license”) and hydra is Apache. How is that possible if the latter is a fork? There’s probably something about these licenses I’m not aware of and I’m curious.
[1]: https://github.com/citusdata/citus
[2]: https://blogs.microsoft.com/blog/2019/01/24/microsoft-acquir...
As far as open-source add-ons for true horizontal scaling, I think [Citus](https://github.com/citusdata/citus) is the most well-known and sophisticated, but I don't have enough experience with it in production to have a particularly strong opinion yet. It might work quite well, but I still fundamentally doubt that it will ever be as good as a system that was designed to support horizontal sharding from the ground up.
https://github.com/citusdata/citus
(Citus engineer)
As your using postgres have you looked at citus[0] at all?
It borrows all the best practices from PostgreSQL the naming of variables and functions are more self-explaining in general.
I also believe that the practices around PRs and code reviews are also good examples.
I knew someone here would misinterpret why I mentioned its age. As I said, it's not what it was designed for. Perhaps it has bolted on things in the 20+ years since it was first conceived, but it wasn't purpose-built for scale.
> To conflate project age with current robustness is to indulge in fallacious thinking
It only makes sense to talk about "robustness" in the context something was designed to work in. I can't abuse a system and call it unrobust. Of course Postgres is robust. You're twisting what I was saying.
> There are good examples of PostgreSQL running at scale, including at companies operating at scale such as Instagram
I'd actually be keen to learn more about how Postgres works at Instagram scale without their own customisations to make it work in that setting.
If Postgres works well in that setting why do things like this exist? https://github.com/citusdata/citus
https://github.com/citusdata/citus
Which is why Postgres is creeping up on both scylladb and cassandra.
Plus PG 10 will have declarative partitioning built in. Pretty cool.
* Postgres Professional roadmap ( Pluggable storages, Multimaster cluster with sharding, Effective partitioning, Adaptive query planning, Page-level data compression, Connection pooling, Native querying for jsonb with indexing support, ....) https://wiki.postgresql.org/wiki/Postgres_Professional_roadm...
* EnterpriseDB database server roadmap ( Parallelism, Replication, Vertical Scalability, Performance ) https://wiki.postgresql.org/wiki/EnterpriseDB_database_serve...
====
And "Scalable PostgreSQL for real-time workloads https://www.citusdata.com " --> https://github.com/citusdata/citus
Github: https://github.com/citusdata/citus
HN: https://news.ycombinator.com/item?id=11353322 "Citus Unforks from PostgreSQL, Goes Open Source (citusdata.com)" ( 24th March, 2016 )
"What is Citus?
- Open-source PostgreSQL extension (not a fork)
- Scalable across multiple hosts through sharding and replication
- Distributed engine for query parallelization
- Highly available in the face of host failures "
"Citus provides users real-time responsiveness over large datasets, most commonly seen in rapidly growing event systems or with time series data . Common uses include powering real-time analytic dashboards, exploratory queries on events as they happen, session analytics, and large data set archival and reporting." https://www.citusdata.com/blog/17-ozgun-erdogan/403-citus-un...