What does HackerNews think of Anime4K?
A High-Quality Real Time Upscaler for Anime Video
Combine mpv with its yt-dlp [2] and SponsorBlock support [3] and you have a competent way to upscale your own YouTube (and other types of [4]) videos.
If you really want to upscale with AI (albeit not at runtime), I've found Topaz Video AI to be the best so far, much sharper than other solutions on the market.
[0] https://gist.github.com/agyild
[1] https://github.com/bloc97/Anime4K
[2] https://www.funkyspacemonkey.com/replace-youtube-dl-with-yt-...
[3] https://github.com/po5/mpv_sponsorblock
[4] https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites....
https://github.com/bloc97/Anime4K
Yes it "guesses" what the missing information is and populates it. Similar approaches will work for live action media too.
And successive video frames encode so much extra resolution that a single frame doesn't capture, so there's a lot of information to pull from.