Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

I have no clue why people still use it and I'd cut my losses if I were you, either backup to the cloud or pull from it, not both at the same time like an absolute tictac.

Now I need a new solution that will work for my parents

You can't connect to their Computer Backup service through third-party software.

Hiding the network always ends in pain. But never goes out of style.

After a backup, you’d go out to a coffee shop or on a plane only to find that the files in the synced folder you used yesterday, and expected to still be there, were not - but photos from ten years ago were available!

When you have a couple terabytes of data in that drive, is it acceptable to cycle all that data and use all that bandwidth and wear down your SSD at the same time?

Also, high number of small files is a problem for these services. I have a large font collection in my cloud account and oh boy, if I want to sync that thing, the whole thing proverbially overheats from all the queries it's sending.

The moment you call read() (or fopen() or your favorite function), the download will be triggered. It's a hook sitting between you and the file. You can't ignore it.

The only way to bypass it is to remount it over rclone or something and use "ls" and "lsd" functions to query filenames. Otherwise it'll download, and it's how it's expected to work.

Or that they're targeting the mass retail market, where people are technically ignorant, and "unlimited" is required to compete.

And statistically-speaking, is viable as long as a company keeps its users to a normal distribution.

But we'd always have a few people at the end of the semester print 493 blank pages using up all of their print quota they'd "paid for". No sir, you didn't pay for 500 pages of printing a semester, we'd let you print as much as you needed, we just had to put a quota in place to prevent some joker from wallpapering the lecture hall.

It was hard to express what we meant and "unlimited" didn't cut it.

Oh well, I guess this is why we're given two kidneys.

Unlimited however, they can offer. I don’t see how people get into mental block of thinking something is nefarious when a company offers you unlimited hosting or data. Yes, they know it’s impossible if everyone took full advantage of that. They also know most people won’t and so they don’t have to spend time worrying about it. It’s a simple actuarial exercise to work out the pricing that covers the use of your users.

Back in the early 2000s I ran a web hosting service that was predominantly a LAMP stack shared hosting environment. It had several unlimited plans and they were easy to estimate/price. The only times I had an issue of supporting a heavy user, it would turn out they were doing something unrestricted. Back then, it was usually something pron or mp3 related. So the user would get kicked off for that. I didn’t have any issues with supporting the usage load if it was within TOS. The margins were so high it was almost impossible to find a user that could give me any trouble from an economic standpoint.

It's that to back up a folder on a filesystem, you need to traverse that folder and check every file in that folder to see if it's changed. Most filesystem tools usually assume a fairly low file count for these operations.

Git, rather unusually, tends to produce a lot of files in regular use; before packing, every commit/object/branch is simply stored as a file on the filesystem (branches only as pointers). Packing fixes that by compressing commit and object files together, but it's not done by default (only after an initial clone or when the garbage collector runs). Iterating over a .git folder can take a lot of time in a place that's typically not very well optimized (since most "normal" people don't have thousands of tiny files in their folders that contain sprawled out application state.)

The correct solution here is either for git to change, or for Backblaze to implement better iteration logic (which will probably require special handling for git..., so it'd be more "correct" to fix up git, since Backblaze's tools aren't the only ones with this problem.)

This is a joke, but honestly anyone here shouldn't be directly backing up their filesystems and should instead be using the right tool for the job. You'll make the world a more efficient place, have more robust and quicker to recover backups, and save some money along the way.

It's the same reason why the postgres autovacuum daemon tends to be borderline useless unless you retune it[0]: the defaults are barmy. git gc only runs if there's 6700 loose unpacked objects[1]. Most typical filesystem tools tend to start balking at traversing ~1000 files in a structure (depends a bit on the filesystem/OS as well, Windows tends to get slower a good bit earlier than Linux).

To fix it, running

> git config --global gc.auto 1000

should retune it and any subsequent commit to your repo's will trigger garbage collection properly when there's around 1000 loose files. Pack file management seems to be properly tuned by default; at more than 50 packs, gc will repack into a larger pack.

[0]: For anyone curious, the default postgres autovacuum setting runs only when 10% of the table consists of dead tuples (roughly: deleted+every revision of an updated row). If you're working with a beefy table, you're never hitting 10%. Either tune it down or create an external cronjob to run vacuum analyze more frequently on the tables you need to keep speedy. I'm pretty sure the defaults are tuned solely to ensure that Postgres' internal tables are fast, since those seem to only have active rows to a point where it'd warrant autovacuum.

[1]: https://git-scm.com/docs/git-gc

- - -

Hey, I tried restoring a file from my backup — downloading it directly didn't work, and creating a restore with it also failed – I got an email telling me contract y'all about it.

Can you explain to me what happened here, and what can I do to get my file(s?) back?

- - -

Hi Jan,

Thanks for writing in!

I've reached out to our engineers regarding your restore, and I will get back to you as soon as I have an update. For now, I will keep the ticket open.

- - -

Hi Jan,

Regarding the file itself - it was deleted back in 2022, but unfortunately, the deletion never got recorded properly, which made it seem like the file still existed.

Thus, when you tried to restore it, the restoration failed, as the file doesn't actually exist anymore. In this case, it shouldn't have been shown in the first place.

For that, I do apologize. As compensation, we've granted you 3 monthly backup credits which will apply on your next renewal. Please let me know if you have any further questions.

- - -

That makes me even more confused to be honest - I’ve been paying for forever history since January 2022 according to my invoices?

Do you know how/when exactly it got deleted?

- - -

Hi Jan,

Unfortunately, we don't have that information available to us. Again, I do apologize.

- - -

I really don’t want to be rude, but that seems like a very serious issue to me and I’m not satisfied with that response.

If I’m paying for a forever backup, I expect it to be forever - and if some file got deleted even despite me paying for the “keep my file history forever” option, “oh whoops sorry our bad but we don’t have any more info” is really not a satisfactory answer.

I don’t hold it against _you_ personally, but I really need to know more about what happened here - if this file got randomly disappeared, how am I supposed to trust the reliability of anything else that’s supposed to be safely backed up?

- - -

Hi Jan,

I'll inquire with our engineers tomorrow when they're back in, and I'll update you as soon as I can. For now, I will keep the ticket open.

- - -

Appreciate that, thank you! It’s fine if the investigation takes longer, but I just want to get to the bottom of what happened here :)

- - -

Hi Jan,

Thanks for your patience.

According to our engineers and my management team:

With the way our program logs information, we don't have the specific information that explains exactly why the file was removed from the backup. Our more recent versions of the client, however, have vastly improved our consistency checks and introduced additional protections and audits to ensure complete reliability from an active backup.

Looking at your account, I do see that your backup is currently not active, so I recommend running the Backblaze installer over your current installation to repair it, and inherit your original backup state so that our updates can check your backup.

I do apologize, and I know it's not an ideal answer, but unfortunately, that is the extent of what we can tell you about what has happened.

- - -

I gave up escalating at this point and just decided these aren’t trusted anymore.

The files in question are four year old at this point so it’s hard for me conclusively state, so I guess there might be a perfect storm of that specific file being deleted because it was due to expire before upgraded to “keep history forever”, but I don’t think it’s super likely, and I absolutely would expect them to have telemetry about that in any case.

If anyone from Backblaze stumbles upon it and wants to escalate/reinvestigate, the support ID is #1181161.

Borg backup is a good tool in my opinion and has everything that I need (deduplication, compression, mountable snapshot.

Hetzner Storage Box is nothing fancy but good enough for a backup and is sensibly cheaper for the alternatives (I pay about 10 eur/month for 5TB of storage)

Before that I was using s3cmd [3] to backup on a S3 bucket.

[1] https://www.borgbackup.org/

[2] https://s3tools.org/s3cmd

[3] https://s3tools.org/s3cmd

~ 5 years ago, I had a development flow that involved a large source tree (1-10K files, including build output) that was syncthing-ed over a residential network connection to some k8s stuff.

Desyncs/corruptions happened constantly, even though it was a one-way send.

I've never had similar issues with rsync or unison (well, I have in unison, but that's two-way sync, and it always prompted to ask for help by design).

Anyway, my decade-old synology is dying, so I'm setting up a replacement. For other reasons (mostly a decade of systemd / pulse audio finding novel ways to ruin my day, and not really understanding how to restore my synology backups), I've jumped ship over to FreeBSD. I've heard good things about using zfs to get:

saniod + syncoid -> zfs send -> zfs recv -> restic

In the absence of ZFS, I'd do:

rsync -> restic

Or:

unison <-> unison -> restic.

So, similar to what you've landed on, but with one size tier. I have docker containers that the phone talks to for stuff like calendars, and just have the source of the backup flow host my git repos.

One thing to do no matter what:

Write at least 100,000 files to the source then restore from backup (/ on a linux VM is great for this). Run rsync in dry run / checksum mode on the two trees. Confirm the metadata + contents match on both sides. I haven't gotten around to this yet with the flow I just proposed. Almost all consumer backup tools fail this test. Comments here suggest backblaze's consumer offering fails it badly. I'm using B2, but I haven't scrubbed my backup sets in a while. I get the impression it has much higher consistency / durability.

both services have internal backups to reduce the chance they lose data

both services allow some limited form of "going back to older version" (like the article states itself).

Just because the article says "sync is not backup" doesn't mean that is true, I mean it literally is backup by definition as it: makes a copy in another location and even has versioning.

It's just not _good enough_ backup for their standards. Maybe even standards of most people on HN, but out there many people are happy with way worse backups, especially wrt. versioning for a lot of (mostly static) media the only reason you need version rollback is in case of a corrupted version being backed up. And a lot of people mostly backup personal photos/videos and important documents, all static by nature.

Through

1. it doesn't really fulfill the 3-2-1 rules it's only 2-1-1 places (local, one backup on ms/drop box cloud, one offsite). Before when it was also backed up to backblaze it was 3-2-1 (kinda). So them silently stopping still is a huge issue.

2. newer versions of the 3-2-1 rule also say treat 2 not just as 2 backups, but also 2 "vendors/access accounts" with the one-drive folder pretty much being onedrive controlled this is 1 vendor across local and all backups. Which is risky.

You are using it to mean "maintaining full version history", I believe? Another important consideration.

So my idea is that it's a competency problem (lack of communication), not malice. But it's just a theory, based on my own experience.

In any case, this is a bad situation, however you look at it.

For stuff I care about (mostly photos), I back them up on two different services. I don't have TBs of those, so it's not very expensive. My personal code I store on git repositories anyway (like SourceHut or Codeberg or sometimes GitHub).

As for GUIs in general... Well, like I said, I just finished several years of bad experiences with some proprietary ones, and I wanted to see and choose what was really going on.

At this point, I don't think I'd ever want a GUI beyond a basic status-reporting widget. It's not like I need to regularly micromanage the folder-set, especially when nobody else is going to tweak it by surprise.

_____

[1] The downside to the dumb-store is a ransomware scenario, where the malware is smart enough to go delete my old snapshots using the same connection/credentials. Enforcing retention policies on the server side necessarily needs a smarter server. B2 might actually have something useful there, but I haven't dug into it.

I like how you can set multiple keys (much like LUKS) so that the key used by scheduled backups can be changed without messing with the key that I have memorized to restore with when disaster strikes.

It also means you can have multiple computers backing up (sequentially, not simultaneously) to the same repository, each with their own key.

Use this command in the developer tools console to change the color.

  .default { font-family:Verdana, Geneva, sans-serif; font-size: 10pt; color:#828282; }
  .admin   { font-family:Verdana, Geneva, sans-serif; font-size:8.5pt; color:#000000; }
  .title   { font-family:Verdana, Geneva, sans-serif; font-size: 10pt; color:#828282; overflow:hidden; }
  .subtext { font-family:Verdana, Geneva, sans-serif; font-size:  7pt; color:#828282; }
  .yclinks { font-family:Verdana, Geneva, sans-serif; font-size:  8pt; color:#828282; }
  .pagetop { font-family:Verdana, Geneva, sans-serif; font-size: 10pt; color:#222222; line-height:12px; }
  .comhead { font-family:Verdana, Geneva, sans-serif; font-size:  8pt; color:#828282; }
  .comment { font-family:Verdana, Geneva, sans-serif; font-size:  9pt; }

    {elem.style.color = '#000 !important'}

Whatever causes it, I do wear glasses (and on a recent prescription too) and the text is still very hard to read.

What is it supposed to do?

There is no mention of F9 on this support page either:

https://support.mozilla.org/en-US/kb/keyboard-shortcuts-perf...

Am I missing something?

Also consider e.g. ~/.cache/thumbnails. It's easy to understand as a cache, but if the thumbnails were of photos on an SD card that gets lost or immediately dies, is it still a cache? It might be the only copy of some once-in-a-lifetime event or holiday where the card didn't make it back with you. Something like this actually happened to me, but in that case, the "cache" was a tarball of an old photo gallery generated from the originals that ought to have been deleted.

It's just really hard to know upfront whether something is actually important or not. Same for the Downloads folder. Vendor goes bankrupt, removes old software versions, etc. The only safe thing you can really do is hold your nose and save the whole lot.