Off topic, but I’m really interested in the opposite. ie alternative desktop environments to Mac OS X’s WindowServer.
There are none to be interested in, and never were.
I welcome correction on this, but TTBOMK there never has been a single one, from the original Lisa OS in 1983.
Apple OSes have always been designed and built as an integrated whole. That's one of the things Apple owners like about them. Rather than developer effort being split across dozens of competing rival implementations, there is one, and all effort goes into improving that single one.
Before I get called a fanboy: I wasn't an Apple owner until about 12 years after the Mac launched and I have never bought a new Apple product in my life. All mine were 2nd hand, or free, saved from disposal.
I would be glad to use Mac OS X applications and OS with a radically different windowing system like i3wm.
If you removed Quartz, there would be no GUI left.
If that's what you want, then there is Darwin out there.
https://github.com/apple/darwin-xnu
There have been various efforts to make it run as a standalone OS with X11:
• Pure Darwin: http://www.puredarwin.org/
• Open Darwin: https://archiveos.org/opendarwin/
• Next BSD: https://github.com/NextBSD/NextBSD
But they aren't much use.
If you just want tiling windows, then I use Rectangle:
For me that replaced Spectacle when that was discontinued: