This is a nice release, but the title is a bit misleading as the released sizes (1.3B and 2.7B parameters) do not yet compare to the size of GPT-3 (175B), but rather GPT-2 (1.5B) instead (although future releases may have significantly more!).
Edit: title improved, thank you!
Yeah. They say they are doing a 10B release soon[1].
I suspect they have run into training issues since they are moving to a new repo[2]
[1] https://twitter.com/arankomatsuzaki/status/13737326468119674...