Honest question: Why are most of the frameworks that I know of in bottom half? When does speed becomes relevant?
Three reasons as to the hierarchy:
1) Most popular frameworks you would encounter run on higher level dynamic languages such as PHP, Ruby or Python. These languages are a decent bit slower than performance optimized languages. The languages towards the top are all C, Java, Go, etc.
2) The frameworks towards the top are generally lower level - they do less automatic 'magic' than the more popular frameworks that handle things like validation automatically.
3) In general, it's more likely you will have heard of older frameworks than newer ones. The frameworks towards the top are generally all less than 2 years old (excluding Java servlets which have just always been decent). This means they make use of newer designs such as non-locking and asynchronous connection handling.
When does speed become relevant? It's actually always relevant as soon as your product gets traction. For example, 1 server at the top of that chart may be able to handle the same load as 10 servers towards the bottom. If you're using a framework at the bottom and paying heavily for 10 servers, you could increase your startups runway substantially by swapping. However, swapping may cost more in engineer-hours than the servers.
Basically, the relevance of speed is directly related to your margin. If your margin is high (investment banking maybe?), then speed is almost irrelevant, you can afford a whole building of servers. If you're competing heavily on mass market web apps, speed is fairly critical as you can out price competition.
To add to this, I'd encourage you to look at the source code used to implement the tests in the higher performance frameworks. Ryan's third point is very relevant: you may find that modern frameworks leverage both high-performance platforms and much of the pragmatic thinking that was introduced a few years back in the frameworks with which you are more familiar.
Source code link: https://github.com/TechEmpower/FrameworkBenchmarks/