My dream version is this exact form factor but capable of running games similar to pico-8 games. Something that has an MMU and can load binaries from external sources but does not necessarily run a full OS.
You and me both. There is already a lot out there similar running linux. Odroid go and other things running open dingux. Pico-8 as it stands needs a lot of horsepower as it's all lua scripts. I've talked to Lex about making a dedicated pico-8 console he says it would be tough, you basically need fully PC hardware to be faithful to what the virtual console should be able to do. I'm certain a lot of carts could be cross compiled into binaries and run just fine though he wasn't excited about that approach.
I was wondering if Nim could be a good tool here by compiling to C, here is a PICO-8 inspired lib written in it: