As an alternative for purely local LLMs, I've been having fun with this setup: https://github.com/oobabooga/text-generation-webui
Neat, eventually would like to run a purely local LLM Emacs shell https://github.com/xenodium/chatgpt-shell. For now ChatGPT only, but working on making more generic/reusable.