What does HackerNews think of the-algorithm?

Source code for Twitter's Recommendation Algorithm

Language: Scala

Twitter unambiguously separates out the feed of people you're following and the algorithmic feed ("For you") in separate tabs (plus you can pin separate lists as tabs). It also opens up the feed you last used when you open a new session. Plus, the algorithm for the "For you" feed is open source[0]

0. https://github.com/twitter/the-algorithm

Well they open sourced part of it, https://github.com/twitter/the-algorithm

and then I'm pretty sure Elon had a whiteboard session during his first days before firing everyone and had people explain it to him. Which he then posted on his twitter account. https://twitter.com/elonmusk/status/1593899029531803649?s=20

Hey! We did some texting on https://github.com/twitter/the-algorithm without issues(*), that was around 150K lines of code.

And it works great, I wish I knew more domain details about twitter to be able to really leverage it, but it quickly finds parts of your system and how to build features like "removing echo chambers from the-algorithm by recommending posts to a broader audience that is likely to have opposing views and interact with others."

I think one of our users tried indexing their internal code on about 200k LOC.

The outputs are however limited to 4k or 8k depending on what model you use.

*(initial indexing might be slow and re-indexing might need to be disabled)

Twitter's recommendation algorithm has long since been completely open sourced. It's available right here: https://github.com/twitter/the-algorithm
I mean, I don't care about the guy but man, 13 days later you're now looking like the fool since 'the claim' has now suddenly turned into a fact. [0] [1]

This also didn't age well at all.

[0] https://github.com/twitter/the-algorithm

[1] https://github.com/twitter/the-algorithm-ml

I think the relevant part of the code is in this other repo:

https://github.com/twitter/the-algorithm

Not sure if it has what you were looking for (and maybe you already checked this repo, too!), but it's more relevant than the linked repo imo

Did you also skim the accompanying (or rather, main) repo, https://github.com/twitter/the-algorithm ?

From a quick clone and line-count, it has:

  235 kLOC .scala
  136 kLOC .java
  22  kLOC .py
  7   kLOC .rs
So I don't think you did, since you posted so quickly and that's a LOT of code.

I also haven't skimmed this code except very superficially, but perhaps you should since you're out there making statements with your Princeton credentials.

(I posted this comment with the heads-up a few minutes after your comment above and then expanded it as you didn't respond.)

Of note for anyone confused and reading the comments now, the link was to an empty, but real, twitter repository at https://github.com/twitter/the-algorithm. Now it is a 404.
Will Norris who works on OSS at Twitter posted this[0]: "watch this space https://github.com/twitter/the-algorithm"

[0]: https://twitter.com/willnorris/status/1518694675909013504