What does HackerNews think of restic?

I do backups to a local, external HD (that I need to manually plug and unplug, unfortunately), and to Backblaze. Depending on the importance of the data, it's being pushed daily to B2 and weekly to HD, or more seldom, or only to HD.

Technology-wise, I've been on restic for some years now, because of its support for cloud backends. (Otherwise, Borg is a fine tool, too.) Until last week, I've used crestic, but I just migrated to resticprofile, as that can do web hooks and pre-/post-commands, which I intend to use for SQL server dumps.

restic: https://github.com/restic/restic/ crestic: https://github.com/nils-werner/crestic resticprofile: https://github.com/creativeprojects/resticprofile/

If you prefer a similar approach, but as a single compiled binary, there's Restic: https://github.com/restic/restic

Update: yet another take on basically the same approach, also as a self-contained binary: https://github.com/kopia/kopia

Duplicates aren’t always bad… some files naturally exist in many places, and removing them from some of the places make that directory/app incomplete.

If you do want to save space by storing one copy of the bits/blocks, and still retain the index of all the original locations… you can store all your backups on a ZFS file system with Dedup turned on… (this uses memory and has performance implications)..

Or back everything up with restic:

https://github.com/restic/restic

…restic stores files encrypted in a tree by the hash, so it naturally stores 1 file but as many references to it as needed. It has lookup and list functions that can tell you what’s duplicated.

To simply find/report dups to be dealt with manually, you could quite easily md5/Sha1 your entire file tree, storing the output in a text file , which you can then pipe through sort , awk, and uniq to see which hashes occupy multiple lines … this is labor intensive… I just let my backup tools “compress” by saving one copy of each hash, and then it doesn’t matter as much (in my opinion).

If its pictures or some other specific file type that you want to focus on the most… I’d pick an app that’s intended for cataloging those. Example: Adobe Lightroom shows me my duplicate pics and I can deal with those easily there.