A bit off topic but, the power of GPT (and DL in general) is in the data. Yet, we’ve allowed private enterprises to control what should be distinctly public goods. I don’t know where we took the wrong turn within the past decade but we desperately need to correct this mistake.

I am not getting the angle here. Anyone, including you, can write GPT-like code, train the model with public data and release it for free. It may cost a few million to train in GPU costs, but if what you say is that important, surely there are folks here(I am assuming a good chunk of HN folks have a decent amount of disposable income) who will donate it for the public good? If there are not, then they either don't consider it important or are just virtue signaling. Or the effort is actually hard to implement.

I am totally okay with OpenAI being worth $30 billion or whatever when compared to crypto scams being worth billions.

I'm interested in solving problems you mention in this space. For the sake of simplicity, I will also agree that the data and the models are free if you know how and where to look. The problem is what then? Who has the money and/or compute capacity to do the work at a scale that can compete with the industry behemoths?

I've been slowly building out a home lab to test mesh computing in this space. Perhaps there is a way to carve the workloads into chunks that can be deployed to a distributed mesh of trusted nodes that have a hardware specs suitable for the task. Then somehow aggregated the results and distributes the entire package back to the network of contributors of that compute capacity. In other words, I will agree to lend you my compute capacity in exchange for a copy of the model you are training. I'd love to collaborate with folks and grow this idea and get a legit open source project going.

Let's build the "Constellation". If anyone wants to geek out and make this happen i'd love to chat. art.aquino at compute dot tech

Building compute clusters and cool software is a passion of mine. So i'm looking to build a network of like minded folks without any commitment and just to help each other.