The Unix Haters Handbook is pretty interesting reading, but I remember that the chapter on X was one of the weakest. It doesn't help that it starts with the hoary old myth that X named the 'server' and 'client' backwards. More relevant however is that from the vantage point of 2017, it becomes clear that X is one of the most unreasonably successful software architectures of all time.
Think about it: how many pieces of software design have remained in heavy use worldwide for 30 years. The fact that no one bothered to replace it in all that time is proof enough that whatever warts it has can't be that painful. (Now, finally, it seems like it's going to happen with Wayland, but it's not there yet.)
Not only that, but computer display technology has changed massively over X's lifetime. I'm writing this in a composited window manager hardware accelerated via natively-3D hardware outputting to multiple monitors of different resolutions over as many different digital display links. How much of this was even imagined in the 1980s? Yet the X protocol via it's extension mechanisms has proven adaptable enough to take in massive changes in the technology, and still keeps on ticking.
> how many pieces of software design have remained in heavy use worldwide for 30 years.
Unix itself.
The Unix you're using today likely does not share a single line of code with the Unix you were using 30 years ago. It was rewritten.
On the contrary, there is a lot that is not rewritten at all.
For starters: The Unix that I (and quite a lot of other people) use today shares entire manual pages with the Unix of more than 30 years ago.
An example: The manual page for the ul command that was written by (then) Mark Horton is pretty much unchanged today in FreeBSD and NetBSD. For 34 years, it hasn't actually described the command correctly or fully.