ZH version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
50% Positive
Analyzed from 1450 words in the discussion.
Trending Topics
#backup#data#storage#hard#devices#files#drive#nas#sync#offline
Discussion (25 Comments)Read Original on HackerNews
Sure it's bothersome to use the sneakernet once a month (manually copy data to USB stick and stash it somewhere), but when that ransomware hits (can happen on personal devices also), it literally can't access the offline copy, and try to encrypt that one also.
Immutable backup solutions are code and thus eventually defeatable from the remote attackers point of view. Offline requires physical access, and maybe a big enough wrench to get the USB stick's disk encryption PIN out of you.
Surrendering just a little bit of absolute control over the physical media can dramatically increase the likelihood that it survives whatever apocalyptic event. I don't know that I've ever heard of tapes being stolen from Iron Mountain. Even if this happened, what are the odds that they would get all the tapes. Your tapes? It probably looks like the warehouse from Raiders of the Lost Ark in there.
Yeah, but an attacker is unlikely to compromise both you and your storage provider at the same point. Many S3-compatible storage providers (e.g. Backblaze and Hetzner) support object lock, where you can lock objects for a certain number of days (and refresh locks if the objects are still used in recent backups). Typically object locks are implemented such that not even you yourself can remove the data from the account settings.
E.g. when I cancelled my Backblaze B2 account, I had to wait until the object locks expired before I could remove the remaining data and delete the account.
Arq on macOS has great support for object locks BTW.
Well overdue for a refresh.
Doesn't that depend entirely on the storage format? For example if you use thunderbird as a client it has database files that you can copy. Since you say that one of your devices is running arch you should check out some of the email options in the repos. There are at least a few of options that can act as a middleman to bridge between your external accounts and your clients.
> So having a plan to roll back to a stable state might be helfpul. I installed and set up Timeshift for this
Snapshots make this trivial. Particularly in the case of btrfs if you configure your bootloader to always use the default subvolume then rollback becomes just `btrfs subvolme set-default`.
I use the wonderful https://github.com/garethgeorge/backrest as a web UI around restic.
Every night, I back the data up to a Hetzner storage box https://www.hetzner.com/storage/storage-box/ which is only ~$3.50/mo USD for 1TB of data.
I have three "tiers" of data for myself:
1. Important things I care about: these go into a folder that gets backed up to Hetzner (examples include photos in Immich, code pushed to Forgejo, etc)
2. Smaller files I sort-of care about: these usually just go into Proton Drive (examples include design documents)
3. Large files I don't care about losing: these go into a folder on a hard drive that is not backed up (examples include media)
This took me about a day to set up and I just check in on it every once in a while. Once a year I will do a practice disaster recovery run.
What's simple because it's "hard" is replacing parts 2 & 3 with a network appliance like TrueNAS running a zfs pool that syncs to backblaze every night. Yeah you have to learn a bit but it won't fall in weird ways like the hard drive part here will just fail to mount one night and not back things up for 3 months until you notice. My 2¢
I think the author is having trouble because he is conflating concepts and roles that should be distinct. Sync, rotating snapshots, and deduplicated backups need to be kept entirely separate if you want any hope of maintaining your sanity.
So he's got sync but he's missing some sort of rotating snapshot system which would solve the stated concern of guarding against syncthing replicating corrupted data. Such automated snapshots can then be used as the source to feed the backup pipeline.
That hard drive doesn't make a good backup because it seems that it is always online. You need an offline backup that you manually plug in to run the job once every so often.
He's also making this more difficult than it needs to be by insisting that the backup drive be compatible with windows. Plug the drives for both snapshots and backups into a linux box, format them with a modern filesystem, and get on with life.
Sync is its own clusterfuck and I have yet to arrive at a satisfactory solution myself despite wasting inordinate amounts of time on it. IMO you either go with a network share or you make due with the "least bad" option of syncthing. Personally I've more or less settled on sshfs at this point not because it's particularly good but because it works well enough and doesn't add any additional complexity.
Personally I use btrfs snapshots on all my devices, those get streamed across the network to a NAS, and the contents of the NAS are periodically (every few months) stuffed into borgbackup on redundant offline devices. Aside from sync the other problem you'll run into if you're a data hoarder is how to split backups across multiple drives once you exceed a few TB. Because external drives only get so large but the NAS will inevitably keep ballooning.
Not if you use git-annex.
https://git-annex.branchable.com/scalability/
> Scaling to hundreds of thousands of files is not a problem, scaling beyond that and git will start to get slow.
So that's probably insufficient for me by at least a couple orders of magnitude.
There is a straight up data loss bug in 2.4.3 (zeroed out files, totally silent).
https://github.com/openzfs/zfs/issues/18366
This caused all sorts of grief at day job where dedupe was useful for a big cache. Conversely I've run ZFS at home for like 15 years at this point without trouble. But this one is an absolute nightmare.
Then you only need to backup the NAS.
That gets synced and it's been trouble free so far.
https://support.apple.com/en-us/102154
That's a core Dropbox-level mistake of putting the backup cart before the main use horse - your files should be sorted according to their primary, not backup use
> Syncthing works very well, its only constraint is that it does need the devices to be often on if you want consistent syncing.
So it's working very poorly, this is a very inconvenient constraint
> phone is a whole other beast. It does not have enough storage to
Indeed
Overall, that fits the title perfectly, and is a bad way to setup your backups. Special needs need special apps.
Ntfs equivalent of Chmod/chown is a "go have a long lunch" type of operation.