With the same quality of discussion and excellent moderation? I'm sorry to say that one does not exist for those topics. YMMV, but both /lit/ and Goodreads still have some great discussion amongst all the deep-fried drivel.
Most 2000s-era communities have been swallowed by the reddit leviathan and are now made up of similarly low-quality, validation seeking type of discourse.
> With the same quality of discussion and excellent moderation? ... Most 2000s-era communities have been swallowed by the reddit leviathan and are now made up of similarly low-quality, validation seeking type of discourse.
There's nothing saying that one can't create a community on reddit and moderate it severely. Yes, it will get some flack from being one of the subs where the moderators delete everything.
The community and moderation evolve together and with a few exceptions of being able to edit titles after the fact or change where links point to - reddit offers similar moderation tooling.
What is needed is the "create a sub, and moderate it to match the vision."
Be it reddit or grabbing a reddit clone and hosting it yourself, or finding another format that works better - the problem is similar. And the core problem there is that volunteer moderation is a thankless task that people only complain about and rarely praise.
The core problem of using Reddit as hosting is that Reddit's site-global social gamification mechanisms incentivize things — posting jokes, hearsay, etc — that a "serious" community (e.g. /r/AskHistorians/) explicitly wants to disincentivize. With no ability for a subreddit community to opt out of these gamification mechanisms. So your moderation efforts are always fighting against the direction the system wants to push things.
Hosting a community on Reddit, is like having a shortest-path route between two buildings that cuts through a grassy field, and then sticking a sign up saying "keep off the grass" and writing tickets for anyone seen on the grass. Rather than being able to either pave the shortest-route path, or rearrange the buildings so that the route between them doesn't go through the field.
There's also no ability to enable any kind of "moderated mode" on a subreddit where only subscribers may post, or where posts/comments (of new users, recent subscribers, etc) must go through a moderation queue. So you're always getting drive-by posts from users who don't know where they are or what the rules are, nor are they going to stick around long enough to care to learn them. The classic "I just found this on /r/all" comment.
(You can make a subreddit invite-only, but then people can't even see posts/comments from the subreddit if they're not logged in — i.e. the subreddit is no longer a public website — which is rarely desired.)
Know of any software with different incentives?
(Do you think, for example, that HK works differently because of software, because of moderation, or both?)
A Group is its own Worst Enemy - https://www.gwern.net/docs/technology/2005-shirky-agroupisit...
Consider the lessons from LambdaMOO or Communitree in there.
The design of the software promotes and discourages certain types of interaction. Stack overflow famously doesn't have a great commenting system - and it was intentionally designed that way to make long running comment back and forth difficult and unpleasant because that was what Jeff wanted (incidentally, one of Jeff's mentors was Clay Shirky who wrote the above mentioned A Group and was on the board of directors of early Stack Overflow). And as much pain as comments on SO had, the alternative for moderation would have been much worse in the framework of Stack Overflow.
Likewise, there's code on HN (an algorithm) that down weights overheated discussions ( https://news.ycombinator.com/item?id=16020089 ).
The technological and social cannot be separated and need to be built, cultivated, and grown in tandem with intention.
And this is where I get back to reddit... you can do all these things on reddit, just that it takes a bit more creative use of the features within reddit and use of external tooling to enforce it.
If I was to try to build something, I'd honestly start from Discourse ( https://github.com/discourse/discourse ) and then work to build the additional features to encourage the type of community that I'd want there and tooling for moderation.
As it is, I'm not that interested in building on discourse and reddit works. If I had to deal with a larger sub where I can't read all the posts each day or was a target of trolling / low quality content, I'd likely start spinning up things like a program that rate limits people who haven't participated much in the sub initially... possibly borrowing some sentiment analyzers to try to get a heads up where there's flame war brewing.
If I wanted something more "real time", discord posts are rather interesting... with the associated discord moderation bots.
Think about what you want, the community that you want, and then the how you want the community to mold the technology and the technology to mold the community.