There needs to be a standard for benchmarking performance of these solutions. Milvus qps seems to be in a completely different tier of performance than the rest.

You can take a look here: https://github.com/zilliztech/VectorDBBench

It lets you run you run the benchmarks using your own API keys. Although it is made by Zilliz (maintainers of Milvus), you can take a look and see what is going on and judge if its fair.