What does HackerNews think of aeron?

Efficient reliable UDP unicast, UDP multicast, and IPC message transport

Language: Java

#34 in C
#19 in C++
#20 in Java
Aeron – Proof of the benefits of open development | Aug 2023
Not to be confused with the famous Aeron chair, this Aeron is "efficient reliable UDP unicast, UDP multicast, and IPC message transport."

https://github.com/real-logic/aeron

State machine replication, and why you should care | Apr 2022
If you're interested in this topic, I highly suggest some of the talks and papers about the LMAX Disruptor [0] and Martin Thompson's latest project Aeron [1]. It targets the JVM, but the lessons are generally applicable since I don't think Concord/Aria are open source.

[0] https://lmax-exchange.github.io/disruptor/ [1] https://github.com/real-logic/aeron

Accidentally making a language, for an engine, for a game | Apr 2022
Expand Context ↕
I assume your live subscribers are external people over the internet?

The trading platforms I've worked on tend to order things into a single stream that you can act upon. This helps with testing, race conditions, auditability. You will want to be able to replay an exact series of events to recreate conditions.

LMAX Disruptor[1] and Aeron[2] are two open source examples of something widely used, either using the libraries themselves or as concepts.

Some things that trading systems (generally?) need which you may not, which might simplify your architecture:

- Every message must be delivered and in order. UDP is usually used over TCP, as the platform will likely want more control over handling missed messages. - It will be common to have read/write applications that need to add a message as a response. Writing is difficult as you need to be quick, otherwise a subsequent message from another application might invalidate your message.

Do subscribers need older data? How do dropped packets impact the system? Can they just be forgotten? What latency requirements do you have? Do subscribers also write to the same stream?

- [1] https://lmax-exchange.github.io/disruptor/ - [2] https://github.com/real-logic/aeron
Homa, a transport protocol to replace TCP for low-latency RPC in data centers | Aug 2021
It targets the properties that Aeron protocol aims for too (https://github.com/real-logic/aeron).
Emerging Architectures for Modern Data Infrastructure | Oct 2020
Expand Context ↕
>> Almost no one provides tooling and platforms that address it

I think this is due to the nature of the mentioned companies are not being too common (yet?). There are tools and systems that you can use, especially from high frequency trading which has somewhat similar challenges. KDB+ and co. would be my first stop to check if there is something that I could use. The question is the financial structure and scaling of the problem, to determine if these tools are in game. There are other interesting projects in the space:

- https://github.com/real-logic/aeron

- https://lmax-exchange.github.io/disruptor/

Of course these are not exactly what you need, long term storage and querying (like KDB) is largely unsolved.

The other tools that you might be referring to by "most of the opensource platforms" indeed are not capable doing this. I spent the last 10 years on optimizing such platforms but it is not even remotely close to what you need, you (or anybody who thinks these could be optimized) are wasting your time.