First hit is always free.
Don’t forget what your dealing with here: The faceless, amoral, infinitely ravenous, maw of the most efficient personal data succubus in history. Make no mistake this is something like “goodwill capture” instead of “regulatory capture.”
I see no way that this diminishes Meta’s power in any way - arguably it strengthens it by making it easier to choose a Meta architecture instead of creating a competing FOSS architecture.
So arguably all this does is raise the FOSS bar technically further entrench Meta - AND with the most important thing, having thousands of developers prime their data architectures for Meta models to eventually serve from a Meta account.
And once it’s widespread enough to lock you in, those commercial terms, whoops they changed!
As opposed to simply being locked into openai api's as the only option?
A false dilemma, also referred to as false dichotomy or false binary, is an informal fallacy based on a premise that erroneously limits what options are available.[1]
These models cost millions to train. The only reason open-source LLMs have a heartbeat is they’re standing on Meta’s weights. The only third path is a public option.
Not necessarily.
RWKV, for example, is a different architecture that wasn't based on Facebook's weights whatsoever. I don't know where BlinkDL (the author) got the training data, but they seem to have done everything mostly independently otherwise.
https://github.com/BlinkDL/RWKV-LM
disclaimer: I've been doing a lot of work lately on an implementation of CPU inference for this model, so I'm obviously somewhat biased since this is the model I have the most experience in.