In the end I think LLM's will not be used on their own much, and instead more as glue for bringing together a whole bunch of other technology and allowing it to be coordinated with plain English.
This is the "Composability" concept being pushed by the people behind LangChain.