The key way I try to distinguish "Snake Oil" is: Is anyone actually using this routinely, and are the problems it solves self-evident to new non-zealot users after a demo?

Take blockchain for example; people spent years and millions of dollars looking for additional usages for the technology beside coins/coin-contracts; and aside from the forementioned usages it didn't really gain routine adoption. That was why the non-coin-blockchain always struct me as a "hype train" or "snake oil" because it was a solution in search of a problem. NFTs are even worse since they haven't found any purpose to exist yet (money laundering?).

Contrast that with the "GPT" offerings (ChatGPT/Bing/Copilot/Bard/etc). Multiple of my colleagues are actively using them routinely every workday and when you demo it to another person, they understand it, and they too start utilizing it in their workflow. Heck, my mom discovered it herself on Bing's homepage and was telling me I should check it out.

That's the opposite of "snake oil" or a hype train, it is arguably a competitive threat to search engines.

PS - Small disclaimer: "AI" is a nebulous term. I'm talking specifically about LLMs. Other types of "AI" have already been here for a long time making our lives better (e.g. computer vision).

Are people durably achieving a competitive advantage through the GPT?

I think we've seen some people striving for a competitive advantage, and willing to see one exist, but I don't know if we've seen any "AI company" turn a meaningful and durable profit yet (open to being wrong on this).

To me (so far), it seems like GPT use cases that apply it as a table stakes feature of some greater application (rather than an ecosystem or a platform), are the ones that are actually showing promise in terms of expected utility + value capture.

If GPT4+ level engines become as cheap to execute as a SQLite query, I think that's where things get interesting (you can start executing this stuff at the edge).

But I still can't see new companies (a la OpenAI, MidJourney, etc.) making a lot of money in this scenario, it seems to overwhelmingly favor companies that already have distribution.

We're likely way too early to call this one. Based on current hardware trends, GPT-4 will run on your phone within 6-10 years. Right now, folks are feeling out the edges of what makes sense and what doesn't make sense at extreme R&D and opex expense. In 10 years, we'd expect the winners of today to still be winners and have great margins due to declining compute costs.

Granted, if you are spending 10x the future value of the product your are offering ... then even a 10x decline in compute costs won't get you where you need to be.

> Based on current hardware trends, GPT-4 will run on your phone within 6-10 years

they quantized model from 16 bits to 4 bits which was low hanging fruit, and looks like they can't quantize it anymore to 2 bits..

CPU/GPUs are general purpose, if enough workload demand exists specialized Transformer cores will be designed. Likewise, its not at all clear that current O(N^2) self-attention is the ideal setup for larger context lengths. All to say, I'd believe we have another 8-10x algorithmic improvement in inference costs over the next 10 years. In addition to whatever Moore's law brings.

Pixel 6+ phones have TPUs (in addition to CPUs and an iGPU/dGPU).

Tensor Processing Unit > Products > Google Tensor https://en.wikipedia.org/wiki/Tensor_Processing_Unit

TensorFlow lite; tflite: https://www.tensorflow.org/lite

From https://github.com/hollance/neural-engine :

> The Apple Neural Engine (or ANE) is a type of NPU, which stands for Neural Processing Unit.

From https://github.com/basicmi/AI-Chip :

> A list of ICs and IPs for AI, Machine Learning and Deep Learning