Coincidentally, I was just reading this the other day. I've been programming for the Nintendo NES. The most common high level language that people use for NES Homebrew development is C with cc65[1]. The output of the compiler is passable, but C is just not well suited to the 6502 processor. Eventually I'd like to write a compiler for a high level language that used these ideas.

[1] http://cc65.github.io/cc65

I've wanted to implement a better C compiler myself, and I think it would work better if it were built from the ground up instead of from Small-C, as cc65 was.

I would split the parameter and computation stacks, instead of using one as cc65 does. The parameter stack would work the same as cc65's, but the computation stack would be a FORTH-like stack indexed with the zp,X addressing mode.

Second, I would have it build an AST instead of having the parser generate the code. This would open doors to big optimizations, such as automatically detecting when to use 8-bit ops (complicated by C's promotion rules), etc.

Finally, I would implement far pointers for memory-banked systems--most prominently the NES, which cc65 doesn't handle banking on as far as I know.

Another think that could be done is using the zero page as a bank of 16/32/etc. registers and treat the 6502 like a RISC.

cc65 is an amazing project that's enabled many developers. It's Small-C pedigree is limiting though, and inefficient on a machine that it wasn't designed for.

You may be interested in PLASMA:

https://github.com/dschmenk/PLASMA