What does HackerNews think of trio?

Trio – a friendly Python library for async concurrency and I/O

Language: Python

#31 in Python
> I am interested in how concurrency can be represented elegantly and efficiently

This was the main driving force behind the development of Polyphony. I've been exploring all the different ways of expressing concurrent operations: callbacks, promises, async/await. Each of these solutions has its drawbacks: callbacks make you split your app logic across different functions; promises are a somewhat better API but do not solve the split logic problem; async/await potentially leads to two types of functions (the "what color is your function" problem). All three kind of break Ruby's exception handling model.

In contrast, Polyphony relies on Ruby fibers, which provide a base for implementing cooperative concurrency, where each fiber does its work, yielding execution whenever it performs a blocking operation, such as reading from a socket, or waiting for a timeout. In addition, fibers are arranged into a tree where each fiber is considered the child of the fiber from which it was spawned, and is guaranteed to stop executing before its immediate parent stops executing. Any uncaught exception raised by a fiber will be propagated up the fiber tree. This implements a pattern known as Structured Concurrency [1]. Fibers can communicate by sending each other messages, in a similar fashion to Erlang/Elixir processes.

There's no direct equivalent in Polyphony to the program you included, which seems to implement two generators running asynchronously. But the following does something similar.

    def number_generator
      n = 0
      loop do
        client = receive
        client << (n += 1)
      end
    end

    task1 = spin { number_generator }
    task2 = spin { number_generator }
    
    10.times do
      task1 << Fiber.current
      puts "next number from task1: #{receive}"
      task2 << Fiber.current
      puts "next number from task2: #{receive}"
    end

 So in the above program, two fibers are spun, each running the same number generator, which repeatedly waits for a message which is actually the fiber of the "client", and sends the next number to the client. In Python there's the Trio library [2], which implements structured concurrency, and which was actually one of the main inspirations for developing Polyphony.
[1] https://en.wikipedia.org/wiki/Structured_concurrency [2] https://github.com/python-trio/trio
I'll +1 the Trio shoutout [1], but it's worth emphasizing that the core concept of Trio (nurseries) now exists in the stdlib in the form of task groups [2]. The article mentions this very briefly, but it's easy to miss, and I wouldn't describe it as a solution to this bug, anyways. Rather, it's more of a different way of writing multitasking code, which happens to make this class of bug impossible.

[1] https://github.com/python-trio/trio

[2] https://docs.python.org/3/library/asyncio-task.html#task-gro...

It's similar to manual memory management: you have to remember to do the other side of the thing you're doing.

Structured concurrency is one approach to solving this problem. In a structured concurrency a promise would not go out of scope unhandled. Not sure how you would add APIs for it though in Javascript.

See Python's trio nurseries idea which uses a python context manager.

https://github.com/python-trio/trio

I'm working on a syntax for state machines and it could be used as a DSL for promises. It looks similar to a bash pipeline but it matches predicates similar to prolog.

In theory you could wire up a tree (graph if you have joins) of structured concurrency with this DSL.

https://github.com/samsquire/ideas4#558-assign-location-mult...

Not complete - doesn't include Task Groups [1]

In fairness they were only included in asyncio as of Python 3.11, which was released a couple of weeks ago.

These were an idea originally from Trio [2] where they're called "nurseries" instead of "task groups". My view is that you're better off using Trio, or at least anyio [3] which gives a Trio-like interface to asyncio. One particularly nice thing about Trio (and anyio) is that there's no way to spawn background tasks except to use task groups i.e. there's no analogue of asyncio's create_task() function. That is good because it guarantees that no task is ever left accidentally running in the background and no exception left silently uncaught.

[1] https://docs.python.org/3/library/asyncio-task.html#task-gro...

[2] https://github.com/python-trio/trio

[3] https://anyio.readthedocs.io/en/latest/

I think that what you're describing is how Trio's cancellation scopes work [1]. Trio is an alternative async library for Python [2] (as opposed to asyncio). It's pretty neat.

[1] https://vorpus.org/blog/timeouts-and-cancellation-for-humans...

[2] https://github.com/python-trio/trio [2]

With respect to async, I'm partial to trio [0], which is a spiritual successor to curio [1].

The model is similar to Golang in many ways, e.g. communication using channels [2] and cancellation [3] reminiscent of context.WithTimeout, except that in Golang you need to reify the context passing.

The author has written some insightful commentary on designing async runtimes [4] and is actively developing the library, so I'm optimistic about its future. There were plans to use it for requests v3 until the fundraiser fiasco [5].

[0] https://github.com/python-trio/trio

[1] https://vorpus.org/blog/announcing-trio/

[2] https://trio.readthedocs.io/en/stable/reference-core.html#us...

[3] https://trio.readthedocs.io/en/latest/reference-core.html#ca...

[4] https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

[5] https://vorpus.org/blog/why-im-not-collaborating-with-kennet...

I just watched your talk, and found it very interesting, thanks for sharing it!

I hope you can do the data flow analysis. That would be so cool and SO useful.

Python certainly has the ability to collect the data. A couple existing tools make use of this.

For example, MonkeyType and Birdseye observe the values that are passed around by tracing execution during test runs (or even during a production run, but the performance impact can be substantial). https://github.com/Instagram/MonkeyType https://github.com/alexmojaki/birdseye

Even more information can be gleaned from the gc module (see https://mg.pov.lt/objgraph/ for a tool using it).

These tools make good progress, but I'd be very interested to see what a software-visualization expert would come up with.

I'd also love to see how a concurrent execution tree can be visualized. For example, the wonderful Trio concurrency library is built on a tree of concurrent tasks. It would be so cool to see which events are happening at the same time. I've never seen a visualization of how it'd work. (The Trio team is also extremely friendly on their Gitter chat.) https://github.com/python-trio/trio

Structured logging is yet another exciting area. Can we generate visualizations from logs in OpenTracing/OpenCensus format? Some existing work is https://github.com/jonathanj/eliottree

Gary Bernhardt's "A whole new world" talk https://www.destroyallsoftware.com/talks/a-whole-new-world proposes extracting data from logs and highlighting important lines from tracebacks and and slow lines from trace timings in the editor.

I haven't used Structurizr, but it seems interesting. Do you have thoughts on it? https://structurizr.com/ has a python port at https://github.com/sixty-north/structurizr-python

I hope some of the (new?) concepts from trio find their way into the standard lib.

trio:

https://github.com/python-trio/trio

trio compared to asyncio, goroutines, etc.:

https://stackoverflow.com/a/49485603/1612318

"Notes on structured concurrency, or: Go statement considered harmful":

https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

I'm curious of your take on Trio or Curio. Do either address the peeves you've outlined?

https://github.com/python-trio/trio https://github.com/dabeaz/curio

It's doesn't have to be complex though.

It's complex because the asyncio API is terrible. It exposes loop/task factory and life cycle way to much, and shows it off in docs and tutorials.

Hell, we had to wait for 3.7 to get asyncio.run() !

Even the bridge with threads, which is a fantastic feature, has a weird API:

    await = asyncio.get_event_loop().run_in_executor(None, callback)
Also, tutorials and docs give terrible advices. They tell you to run_forever() instead of run_until_complete() and forget about telling you to activate debug mode. They also completly ignore the most important function of all: asyncio.gather().

asyncio can become a great thing, all the foundational concepts are good. In it's current form, though, it's terrible.

What we need is a better API and better doc.

A lot of people are currently understanding this and trying to fix it.

Nathaniel J. Smith is creating trio, a much simpler, saner alternative to asyncio: https://github.com/python-trio/trio

Yury Selivanov is fixing the stdlib, and experiments with better concepts on uvloop first to integrated them later. E.G: Python 3.8 should have trio's nurseries integrated in stdlib.

Personally, I don't want to wait for 3.8, and I certainly don't want the ecosystem to be fragmented between asyncio, trio or even curio. We already had the problem with twisted, tornado and gevent before.

So I'm working on syntaxic sugar on top of asyncio: https://github.com/Tygs/ayo

The goal is to make the API clean, easy to use, and that enforced the best practices, but stay 100% compatible with asyncio (it uses it everywhere) and it's ecosystem so that we don't get yet-another-island.

It's very much a work in progress, but I think it demonstrate the main idea: asyncio is pretty good already, it just needs a little love.

There are other implementations, such as Curio (https://github.com/dabeaz/curio) or Trio (https://github.com/python-trio/trio). Both are simpler and more hand-holdy (and less footgun) than asyncio.

Disclaimer: I am a trio dev, though.