For people in a similar situation, I'd strongly suggest giving Phoenix (on Elixir) a look. It's a "boring" Rails style server side rendered web framework, but with a few extra powers:

* Elixir is a great functional programming language

* When your backend needs to do more complex or longer running work it's just more Elixir rather than complicated architecture involving task queues etc. (because you have the concurrency of the BEAM to manage it)

* You have options like LiveView for your frontend. You don't have to use it, but it's there.

I agree with what you said about testability for Lambda style architectures. How do you spin up an offline version of some or all of that to see if it works? The BEAM world is nice for me because you can run a similar architecture of separate things (albeit all in one language), but have a much better testing setup.

heipei

I'm curious, and I know this is a random thread to post this question, but how do you handle the fact that the Elixir process might be shut down with regards to necessary persistence of any message queues? Put differently: I consider both message queues and databases in my system to be part of the stateful workloads and the web app part of the stateless workloads (can-be-killed-at-any-time). How do you handle the case where your stateful workload (queueing) now lives within your Elixir process which might be killed at any instance?

bostonvaulter2

If you want a persistent task queue in Elixir then I would use a library like oban [1] that uses postgres to manage the jobs, a nice benefit of it is that you can enqueue your jobs as part of a database transaction (and Ecto.Multi gives you a nice interface to transactions). And just because you can have a message queue directly inside your application server doesn't mean you need to or that it's a good fit for every project.

1. https://github.com/sorentwo/oban