I use Postgres SKIP LOCKED as a queue.
I used to use SQS but Postgres gives me everything I want. I can also do priority queueing and sorting.
I gave up on SQS when it couldn't be accessed from a VPC. AWS might have fixed that now.
All the other queueing mechanisms I investigated were dramatically more complex and heavyweight than Postgres SKIP LOCKED.
Do you have an example of such a queue somewhere?
I think I have a rough idea about how it works because I implemented something similar about four years ago in PostgreSQL but kept getting locking issues I couldn't get out of:
- https://stackoverflow.com/questions/33467813/concurrent-craw...
- https://stackoverflow.com/questions/29807033/postgresql-conc...
Also, what kind of queue size / concurrency on the polling side are you able to sustain for your current hardware?
https://github.com/mbuhot/ecto_job for Elixir
https://github.com/timgit/pg-boss for Node.js