This is a great overview of a RAG workflow, and also composing a complex task down into simple tasks that fit within the context window!
I'm really interested if there is any software or open source projects that make this type of thing easier- specifically the idea of creating multiple "phases" or "tasks" that each has their own LLM prompt and validation rules (like each node in the flowchart.) I think something like that could be very helpful!
There's a few tools out there like AgentGPT (https://github.com/reworkd/AgentGPT, although it's a more conversational interface), and (https://github.com/logspace-ai/langflow) and others. I think most developers definitely prefer a code-first interface though like a library but haven't found one that's great yet. We've used them in the past but didn't have the best experience so would love to hear if anyone has worked with a library they found really flexible.