Is there a place that compares/scores all of these local LLMs?
Would be nice to know how close they are to the paid ones.
Edit: Found one here:
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...
It doesn't include a comparison to the paid ones, but it's a great place to look at different ones and see which ones you should experiment with and sink time into.
Found it a minute after i posted ;-)
Have you tried any of the high-scored ones?
I've been playing a lot with Llama 2 13B recently, and it's really not bad at all. With oobabooga[1] you get a proper UI for it and even get an OpenAI-compatible API, so you just change the endpoint in your OpenAI library and it all works. I've been using that to test changes to my bots.
As another poster mentioned though, it's nowhere near the level of GPT-4. It's close enough to GPT 3.5 though, you should try it out!