What does HackerNews think of bup?

Very efficient backup system based on the git packfile format, providing fast incremental saves and global deduplication (among and within files, including virtual machine images). Please post problems or patches to the mailing list for discussion (see the end of the README below).

Language: Python

if files are getting managed by blobs, git doesn't really scale well with that since it stores diffs.

`bup` notionally does this a lot better, or git-lfs.

https://github.com/bup/bup

https://raw.githubusercontent.com/bup/bup/main/DESIGN

https://git-lfs.com/

git really needs textual representation for any kind of meaningful commit, and binaries totally break that.

I combine git-annex with the bup special remote[1], which lets me still externalize big files, while benefiting from block level deduplication. Or depending on your needs, you can just use a tool like bup[2] or borg directly. Bup actually uses the git pack file format and git metadata.

I actually wrote a script which I'm happy to share, that makes this much easier, and even lets you mount your bup repo over .git/annex/objects for direct access.

[1]: https://git-annex.branchable.com/walkthrough/using_bup/

[2]: https://github.com/bup/bup

You may really like https://github.com/bup/bup if you want something a bit more modern but in the same style
Bup (Python + C) - https://github.com/bup/bup - de duplicated and compressed. Storage format is a fit repository, an interesting choice that lets you restore using just git tools, “cat” and some effort.
Interesting, I've been using bup[1] for years but it looks like bupstash is scratching the same itches I had.

[1]: https://github.com/bup/bup

bup (https://github.com/bup/bup) kinda pioneered it... and as others have advised, restic, borg-backup and tarsnap do exactly that.

According to wikipedia, bup was released in 2010, 3 years after Tarsnap started doing this. (And Tarsnap wasn't the first either.)