This goes for a lot of services – why do YouTube recommendations take up the majority of its homepage? Why are Instagram account suggestions pinned to its sidebar? Why are there news article recommendations below the article that you just read? Suggestions are meant to help users discover content. It's like how there were curated link directories before search engines existed, to help users discover new websites on the internet. This is the same idea, but limited to a specific platform so that you spend time on only their share of the internet.

As an extreme example, what if all forms of suggestion/recommendation/curation features on all platforms were gone, and only a search bar exists? Then your engagement is limited to the scope of your own thoughts, and you would just leave when you have nothing in mind that you want to search (and thus see fewer ads, generate less revenue, etc). I think there is a balance to be achieved, but companies certainly error on the side of more opportunities to drive engagement.

Anyway, your best bet is probably to cook up a browser extension/script to hide what you don't want to see. Maybe it even exists already. Though of course, the DOM probably changes all the time.

I've been using an extension called Tweak New Twitter[1] and it's great. Removes most of the engagement driven bloat.

1: https://github.com/insin/tweak-new-twitter