This is a big frustration of mine, too. The other is that Spotify is extremely biased towards playlist listening and makes browsing and listening to a library of albums really painful.

The good news is that Spotify's SDK and API are actually powerful enough that you can build up an entire alternative interface, which is actually what I recently started doing: https://i.imgur.com/ar7VrYy.png.

It's still work in progress but actually works perfectly well already. It's not ready for public use yet and also isn't open source yet though. If you want to follow development I guess the best place to do that is my twitter: https://twitter.com/tom_j_watson.