That is essentially what VLF[1] does in Emacs. It reads in discrete chunks of the file at a time and doesn’t load the next one till you try to display it. Doesn’t require any fancy data structures, just some extra book keeping and mechanics.
You can edit gigabyte-large files with no trouble vlf.el https://github.com/m00natic/vlfi – though it's a bit of a separate universe, you have to use vlf-occur instead of occur to find matches over all gigabytes of the file
;; To have it offered when opening large files:
;; (require 'vlf-setup)
;; When such files are detected, the command `so-long' is automatically called,
;; overriding certain minor modes and variables with performance implications
;; (all configurable), in order to enhance performance in the buffer.Again alas, long lines is not the only large-file problem in Emacs. Though perhaps most of my woes pertain to Org-mode, but I had to look for a solution to edit large files in the past.
This is probably the most current implementation of a ‘view large files’ package: https://github.com/m00natic/vlfi
You know how "there's an app for that" used to be a thing? Well, there's an elisp package for that: https://github.com/m00natic/vlfi
I've had good luck on big (multi-GB) SQL dumps, which often have very long lines, with
I believe you can use the vlf package (https://github.com/m00natic/vlfi) for dealing with large files in Emacs. I haven't used it myself, so I am not sure how stable it is.
Depends on the machine. With https://github.com/m00natic/vlfi you can\nexplore arbitrarily large file in instant.