It appears there is this genre of articles pretending that LLAMA or its RL-HF tuned variants are somehow even close to an alternative to ChatGPT.
Spending more than a few moments interacting even with the larger instruct-tuned variants of these models quickly dispels that idea. Why do these takes around open-source AI remain so popular? What is the driving force?
> Why do these takes around open-source AI remain so popular?
I can only speak for myself, but I have a great desire to run these things locally, without network and without anyone being able to shut me out of it and without a running cost except the energy needed for the computations. Putting powerful models behind walls of "political correctness" and money is not something that fits well with my personal beliefs.
The 65B llama I run is actually usable for most of the tasks I would ask chatgpt for (I have premium there but that will lapse this month). The best part is that I never see the "As a large language model I can't do shit" reply.
How have you managed to run the 65B model? Cloud resources, or you have a very kitted-out homelab?
All you need is 2 3090s.
All you need is a few thousand dollars lying around to spend solely on your inference fun?
I don’t think that many people really qualify as such (though it’s probably true that many of them are on HN).
AFAIK, you are able to fine-tune the models with custom data[1], which does not seem to require anything but a GPU with enough VRAM to fit the model in question. I'm looking to get my hands on an RTX 4090 to ingest all of the repair manuals of a certain company and have a chatbot capable of guiding repairs, or at least try to do so. So far doing inference only as well.