Hacker News new | past | comments | ask | show | jobs | submit login
Arq 5: Massively faster backup and restore for Mac and Windows (arqbackup.com)
258 points by hemancuso on April 12, 2016 | hide | past | favorite | 188 comments



Arq, (For backups to AWS - though obviously supports every cloud back end under the sun) and "Data Backup" by ProSoft engineering (For backups to USB) are my goto backup tools for day-day ensuring all my work documents are kept up to date.

Yes, I have Crashplan (for the last couple years, backblaze for the three years before that) - but the constant chewing up of CPU cycles gets annoying after a while. And both crashplan/backblaze "everything for $5" come with massive caveats (like deleting backups of Hard Drives that haven't been plugged in for 6 months - I've got Arq Backups of Hard Drives that I haven't plugged in for a couple years, safe and sound) - and I've never had an AWS backup bill in excess of $3.00, ARQ does a wicked good job of keeping your backups on a tight budget.

Also - awesome win for ARQ - when I moved to Singapore, I simply added a AWS Singapore S3 Bucket and wowza - fast backups on my gigabit ($49/month) link from MyRepublic. Really feel like I'm living in the future.

I think once I switch away from Aperture over to Photos, which presumably has a rock solid backup to iCloud photos, then simply doing a quarterly backup or so with CarbonCopyCloner + Arq to AWS + DataBackup to USB key will have my OS X backups covered.


like deleting backups of Hard Drives that haven't been plugged in for 6 months

Wait - what?

If your hard disk gets lost or destroyed, how do you protect those backups?


You need to keep them connected periodically otherwise if you lose it without reconnecting... lost.

For backblaze it's even less time.


"You need to keep them connected periodically otherwise if you lose it without reconnecting... lost. For backblaze it's even less time."

No way. What ?

You're saying that if I dutifully pay my $X/mo for unlimited backup space, but fail to connect, or perform an update, the remote data is removed ?

What ?


Yeap, most if not the overwhelming majority of backup services work like that. Some like Mozy have as low as a 30 day counter. [1]

--

[1] At least they had that 30 days limit a few years ago, I haven't used them in a while.


Well, I know one that doesn't work like that :)


I believe for CrashPlan, this ONLY applies to CLOUD backups. If you backup to local media or across computers, there is no "auto-delete" timeframe.


Hello Everyone,

Wanted to jump in here to confirm.

This policy only affects devices that have not connected to CrashPlan Central in 6 months or longer. This does not affect volumes that have not connected to the device in that period of time. (i.e. an external hard drive that has not connected in 6 months.) Additionally, there is no minimum connection time for local CrashPlan backups.

It’s important for CrashPlan users to consistently connect their device(s). Part of CrashPlan’s ability to maintain the archive health and integrity relies upon regular connection from the device. CrashPlan is able to routinely perform maintenance on the archive by comparing checksums between both device and CrashPlan Central.

https://support.code42.com/Administrator/3/Monitoring_And_Ma...

Please let me know if I can provide additional clarity.

Best regards,

Jarrod


Is this also the case for crashplan business or PROe? (ceejay in europe).


Since the infrastructure for CrashPlan's backup engine is the same between our Business/Consumer clients, we recommend that all users routinely connect their devices to the backup destinations. That being said, this policy only affects CrashPlan for Home subscribers at this time.


Yea I think so, it's only for them to keep you from storing everything up there forever when it could just be stale data.

I think it's fair if these are supposed to be daily backups to do this esp with cloud data and having to provide "unlimited" space to everyone using the product.


Don't know whether I'm grand-fathered into a different terms but I've just pulled some files from a back-up on a machine that was last active 5.4 years ago.


It took 3 years of disconnection before I got a notice the other day that one of my old backed up PCs was going to be deleted.

It makes sense; they're not an open-ended data archival company. If you don't have a copy of the data locally, you can't expect CrashPlan to keep the one true copy indefinitely. Connecting the data to the internet twice a year doesn't seem too onerous to show evidence of that, particularly for a cloud backup, the entire premise of which is that you're connected.


Absolutely agree with you - I think CrashPlan/BackBlaze are acting entirely reasonably when they delete old hard drives, particularly if they give a bit of grace after sending the email that they are about to nuke them.

It's that just for some of us, who like to archive something like a 100 GB Hard Drive onto Amazon Glacier, for $0.007/GB/Month. (Roughly $0.70/month + $5 upload fees for a 100 GB Hard Drive Archive) - and just leave it there, presumably for decades, are better served by Arq + Glacier than we are by CrashPlan/BackBlaze - they are entirely different tools for different purposes.

On the Flip Side, backing up users two 5+ Terabyte Hard Drives on S3 with Arq gets a little pricey... How crashplan/backblaze manage to do it for $5 is beyond me. Presumably it's because most its users are sending in < 100 Gigabytes (after deduping)


Yev from Backblaze here -> Absolutely. They key difference is backup vs. archive. Backblaze was designed as a backup solution, it's intended to be a 1:1 copy of your user data, and if your data set changes we change it on our end as well, with a 30-day history for accidental deletes. We need to reclaim that space to keep costs down, and we're not intended nor designed to be an archive (keeping data forever).

Backblaze B2 is designed differently and can be used as an archival system. The philosophies are different, but one of the reasons that we created it was to give folks the option of making actual archives they could keep in the cloud.

We hit the $5/month price-point by having our own server design, and by reclaiming space on occasion when data sets are removed. On the B2 side, since you're paying per GB, we can afford to keep that data for longer stretches. Hopefully Arq will integrate with B2 in the future and you'd be able to use their system to pick and choose what you want archived and have B2 as a possible repository.


Thanks for the clarification.

These pages make it clear how it works and it seems reasonable:

https://help.backblaze.com/hc/en-us/articles/217665398-Backi...

https://help.backblaze.com/hc/en-us/articles/217664898-What-...


I think your comparison is out of date [1], AWS S3 infrequent access has $0.0125 per GB now not $0.022+ that you have listed. Of course yours is still much better with $0.005 per GB.

Also, it does not look like you have any kind of consumer offering for it, as I have to contact sales to get anything at all. So there's really no point for Arq to even try adding support. I like your prices though.

[1] https://www.backblaze.com/b2/cloud-storage-providers.html


We don't write out the infrequent access pricing as it's not what we're competing with, also why we don't price out Nearline or Glacier, it's not at 1:1 for what we're trying to offer. The consumer offering is available from the docs: https://www.backblaze.com/b2/docs/. You can use the web interface if you want, or tie in with CLIs/APIs. The sales channel is primarily for people that want to integrate it in to the apps that they are building. Hope that clears some things up!


”On the Flip Side, backing up users two 5+ Terabyte Hard Drives on S3 with Arq gets a little pricey”

I send my Arq backups to Amazon Cloud Drive using the ”unlimited” plan. It costs $59.99/year. So far I have only backed up < 100 GiB however, so I don’t know how well it handles backups that are multiple TiB.

[Edited to correct confusing typo.]


Well, I am fairly certain that Amazon S3 will handle multi-terabyte backups, mostly because their pricing tiers are for 1 TB, 50TB, 500 TB, 1 Petabyte, 4 Petabytes, and 10 Petabytes - and they make more money the more you store. Performance (at least in Singapore) is also pretty awesome if you have a gigabit connection.

I'll be interested in hearing of any experiences (particularly around performance) of someone attempting to backup on the order of 10 TBytes on the Amazon Cloud Drive. My guess is that if more than very few people do this, then either (A) Amazon puts an end to "unlimited" (and yes, I appreciated the scare quotes), or, (B) They rate limit uploads after a certain size to the point at which it just frustrates people.

For some of us, dealing with a vendor who finds greater usage on your part to be a desirable behavior, such that they actually give you price breaks the more you use, creates a business relationship that is worth more than the several hundred dollars/year you'll end up saving. (Of course, this is coming from the guy who has a $36/year AWS bill, $24 of which is S3 storage)

(Side note - When talking about Storage, it's very rare to use GiB/TiB - Data rates and Storage are almost always GB or TB).


How much are you storing for those $3.00/month? Sounds too good to be true :-)

We're currently just rsyncing our pictures and stuff at home to two 3TB USB-drives (one active and one at my parents' place; using LUKS and btrfs with compression, snapshots). But even after running deduplication, they're filling up (raw files …), so I'm always on the lookout for other options. Upgrading to 2x4TB is a bit expensive, but I haven't yet found anything that'll cost me less than that while still having client-side encryption and Linux support. E.g. tarsnap seems to be about $250/TB-month, http://jotta.no/ is unlimited but has no Linux/encryption support, and I never understood Glacier pricing :-)

(and it's really convenient to be able to just restore from a local USB drive instead of having to wait for the network, though of course it's less convenient not having backups when travelling …)


AWS storage with S3 is $0.03/GB/Month. I have probably around 20 Gigabytes, stored in Singapore and US-East, versioned back 2+ years + another 58 Gigabytes in Personal Photos Backed up on Amazon Singapore (which I really should move to Glacier). It's compressed when stored on S3, so total storage is only 54 Gigabytes on Amazon Singapore, and 12 Gigabytes on Amazon-East.

I shoot a ton of pictures with my SLR, but, at the same time, I don't shoot raw, my camera is an EOS 10D (6.3 megapixel resolution) and I'm hyper aggressive about deleting all but the top 5% of my shots each day. I may shoot 300 pictures and keep 10-15.

So, I guess the major difference is I'm backing up about 78 Gigabytes of Data (though 20 of which is in two locations).


Arq doesn't get the publicity it deserves. It's a reliable, provider-independent backup solution that YOU control. Data gets encrypted locally, then sent over to storage providers. When new storage providers appear, Arq implements APIs and lets you use them.

Most importantly, when restoring, you don't enter your decryption password/key into a browser window. I don't understand how online-backup companies can talk about security while requiring users to give them their passwords in order to restore data.

I've been using Arq for about two years now and I'm very happy with it. For the reference, I have previous experience with CrashPlan and Backblaze.


Can you compare CrashPlan and Arq performance please?

I use self-hosted CrashPlan for several years, it's great, but restores and thinning archives feel slow (like half a day for cca 1 TB of backups).


I had nothing but performance problems with CrashPlan, so Arq was a significant improvement. But that's setting the bar low.


Well arq 4 was slow, but now arq 5 is much faster, uploading at 10MB/s from my macbook over wifi to amazon cloud drive.


I second this! Have used Arq for several years, it works great. Happy to see performance improvements!


Been an Arq user for years now, and recommend it to anyone. Love being able to choose my own storage solution, control encryption keys, etc.

For Arq team (saw one or two here), is B2 on the roadmap?


Arq's author has said on Twitter that the B2 API doesn't support all commands Arq needs, so even if B2 support was planned (which it isn't) it wouldn't be possible yet: https://twitter.com/arqbackup/status/717756616578301952


Yev from Backblaze -> We're working on a few APIs and some should be aligned with what ARQ is looking for. We're hoping that they do integrate with us, giving folks an inexpensive alternative!


Just found out about it... Why not just Google Drive, or Dropbox?


How is this, from a technical point of view, more advanced than running rsync in a regular fashion (with hardlinks to keep cheap snapshots)?

Just wondering about the technical aspects.


One technical thing to note:

Although the application itself is not open source, the developer has open-sourced a restore tool, to help ease the "what if they go out of business" concerns:

https://github.com/sreitshamer/arq_restore

Also, somebody else has created a Go utility that can read the backup format:

https://github.com/asimihsan/arqinator

Finally, if you really wanted to do it all yourself, the data format is documented here:

http://www.haystacksoftware.com/arq/s3_data_format.txt


It needs updating for Arq 5. I'm working on right now.


Client-side encryption is a big one. It also has a passable gui which can be helpful when restoring files, and does a pretty good job of scheduling and throttling upload speeds.

I had an older version that I upgraded to 4.x because they added support for uploading backups to Dropbox and I was already paying for a Dropbox pro account that had plenty of unused space. The ability to have versioned backups on various cloud services that don't support rsync is a big plus.


One differentiating factor is that rsync doesn't work with storage services like Amazon S3, Glacier, Google Drive, and the like.


You could use rsync and s3cmd in tandem to create a decent backup solution. Of course, it does require the command line and some scripting chops, but I've done it in the past to keep daily, weekly, and monthly logical MySQL backups on S3.


which is a good thing.

if a few years ago somebody said "just copy all your private data to some place on the internet... but encrypt it first" you would take it for the crazy argument it is.


What do you think is crazy about it? If it's encrypted, it can't be read. If it's on "some place on the internet", it can only be as redundant or more redundant as any other offsite solution.


> "If it's encrypted, it can't be read. "

Yet. It can't be read yet.


Any agent capable of ever reading AES encrypted files with 256 bit keys is also an agent capable of opening your safe deposit box, requesting your tapes from a data vault provider, or coming over to your house with a rubber hose.

Short of a vulnerability in AES (in which case we have more problems than a few copies of the Anarchists Handbook in our backup files), cracking proper encryption is simply not feasible.


What about in 10 years, 20, 50?


With arq there are multiple layers that would have to be broken to get to the data. Still a risk, obviously - and one worth considering. I bet in the scheme of things though there are much easier ways to access this data available now.


It's also versioned backup, like Time Machine. Not just a clone of the latest.


You can do this with rsync as well, including the hardlinked-to-older file method.

Have a look at the `--link-dest` option to rsync.


Arq is like Tarsnap/Attic/Borg in that it uses shift-resistant block deduplication - it can efficiently cope with small modifications to existing files, where rsync would have to clone the entire file for each new version.


In the context of Time Machine, it does not use that form of deduplication, so rsync is capable of fulfilling the parent's requirements.

In the larger scope of things, that level of deduplication has never been something I've found all that valuable; but of course most binary files I backup are images and music files; not ones which change frequently (if ever).


rsnapshot: "rsnapshot is a filesystem snapshot utility based on rsync. rsnapshot makes it easy to make periodic snapshots of local machines, and remote machines over ssh. The code makes extensive use of hard links whenever possible, to greatly reduce the disk space required."

http://rsnapshot.org


If you are interested in something similar to what you describe, there is always Dirvish. It's basically a program that manages multiple snapshots and backups using rsync and hardlinks.

http://www.dirvish.org


Have you tried Dirvish yourself? No stable release for over ten years doesn’t really inspire confidence IMHO.



thanks


Given the rise of ransomware in the recent months, what protection does Arq offer against that? That thing alone would easily tip the scales for me versus the Time Machine.

I see that the AWS S3 IAM user has both read and write access, so if the ransomware authors ever bother with it, they can kill the backups.

Would that help if I setup versioning on the bucket? Will Arq be able to restore backups from the older version of data, before the attack takes place?

Any other ideas?


Bucket versioning should help with this. I don't use Arq and don't know if it supports s3 bucket versioning, so it might not be convenient but the data would still exist.

Although if an attacker has control writing to your s3 bucket, they could rack up a big bill.


There is no monetary incentive to rack up the bill, but there is an incentive to kill backups - ransom demands are a lot more persuasive when the victim has no backup.


Why would that tip the scales versus Time Machine? Time Machine should provide the same protection as Arq, i.e. versioned backups from before the ransomware attack should be safe.

I know there was some hue and cry a little while ago about Mac ransomware that can encrypt network drives and external hard drives, but there's a reason why the _encrypt_timemachine routine was an unused stub. From what I understand, Time Machine has protections built into the kernel that prevents existing backups from being modified. New backups after the ransomware attack would obviously end up backing up encrypted data, but the existing backups should remain untouched.


Well, it should, but does it?

Time Capsule's drive is just another network drive. The data could be easily erased. There's also a button in the Airport Utility that nukes all data on the drive. There is no reason for me to believe that this button could not be triggered by rouge software.

It would be nice if you could provide citations to the opposite.


It's not "just another network drive". It's mounted specially by the OS. Sure, if you mount the drive like a normal network drive then the protections might be lost (but maybe not; it's plausible that the protection takes the form of an xattr that prevents modification, so mounting it using any mechanism that respects xattrs might preserve the same protection. I'm not at home right now or I'd check up on that). But you don't normally mount your Time Machine backup volume as a normal network volume, and the malware shouldn't be able to do it either (since it doesn't know the password).

I'm not familiar with the button in AirPort Utility that you mentioned. I assume you're talking about a Time Capsule? I don't have one of those, I use a Synology NAS as my Time Machine destination, so I'm not familiar with the button in question. That said, presumably triggering that functionality requires having the base station password, and if you want to speculate about the software actually causing AirPort Utility to launch and manipulating its UI in order to try and literally press the button, that kind of functionality would require the user to grant Universal Access permission to the rogue software (the Accessibility permission in the Privacy tab of the Security & Privacy preference pane).

In any case, if you're talking about theoretical attacks where the software figures out how to actively mount a network drive that isn't already mounted in order to wreck it, then you may as well speculate about it figuring out how to delete data from your Amazon S3 bucket (or whatever other cloud provider you use as an Arq destination).


>if you're talking about theoretical attacks where the software figures out how to actively mount a network drive that isn't already mounted in order to wreck it, then you may as well speculate about it figuring out how to delete data from your Amazon S3 bucket

Yeah, and that is precisely where I started my question. To quote (from the post you have replied to):

[...] I see that the AWS S3 IAM user has both read and write access, so if the ransomware authors ever bother with it, they can kill the backups. [...]


Oh so you did. I had forgotten.


To answer your question directly:

Both Arq and Time Machine create differential backups. Thus, any particular backup can be restored back in time. However, Arq targets non-file-based media (although you could trick it by a little SSH magic). Time Machine requires file-based access.

If ransomeware finds your file-based backup, it will encrypt it and render your backup useless.

The term backup gets bandied about, so it can mean one or more of the following: high-availability, synchronization, and/or disaster recovery. You'll want to look into these and the concept of the 3-2-1 method.


That doesn't answer my question at all.


what part of that didn't you understand?

Yes arq will protect you from ransomware. Time Machine will not.

both backup differences only. So with arq you just pick a backup before everything was encrypted. With Time Machine the problem is your hard drive is on the same machine that's been infected so that hard drive will be encrypted as well.

Arq doesn't have that problem since the data is in the cloud. The ransomeware doesn't have write access to that data, at most it has indirect append access since arq will start backing up the encrypted files. Which is why you'll be able to just pick a version of the backup before anything was encrypted.

---

that is until there is ransomware that checks for arq and tells it to delete all your cloud data :(


> that is until there is ransomware that checks for arq and tells it to delete all your cloud data :(

Well, yes, and that is exactly what I wrote in my original post:

[...] I see that the AWS S3 IAM user has both read and write access, so if the ransomware authors ever bother with it, they can kill the backups. [...]


I would just like to point out that this is major release 5 and they're just now adding threading and consumption of filesystem events.

That's _great_ from the standpoint of launching a product. Putting off adding this complexity probably let them get to market sooner.

If I were releasing something like Arq, I would have to fight myself very, very hard to not add these to the 0.1 release. I don't know this space very well, but maybe there are several Arq-alikes who started earlier, but didn't release until later because it wasn't "done" yet, and they missed their chance.


But ARQ has always been fast, and figured out what it needed to backup pretty much instantly, and just did it's job and got out of the way - unlike things like spotlight/mdworker, or crashplan/backblaze, that are constantly thrashing my CPU and causing my fans to spin up.

From my uneducated perspective, having simple software that just worked was a bonus - who knows, maybe with the addition of threading, and consumption of filesystem events, ARQ is going to become crummy, and a door will be opened for someone else to write simple backup software without those features that gets the job done and doesn't bog up your computer. I guess we'll have to try Arq 5 for a few weeks to find out. Fingers Crossed.


For me personally having a history in this space these are two 1.0 features if releasing today. Back when they released 1.0 it might not have been obvious that these features were needed to differentiate.

But getting the single threaded path solid before adding parallelism is a good choice.


Well, seeing the success of Arq will certainly help you in fighting that fight. :)


I have nothing to add except that I've been a happy customer of Arq for years, and Stefan and team have provided us very helpful and personal support by email on the rare occasions when we needed it.


I've been using Arq for years and absolutely love it. Worth every penny. It is extremely well-built software — it's FAST, doesn't hog resources, and feels very polished & reliable.

I like that I can backup to multiple destinations (AWS S3/Glacier, Dropbox, Google Drive, even my own server via SFTP). IMO you can never have too many backups.

I use it along with Backblaze (and will be setting up Time Machine & Super Duper or Carbon Copy Cloner this week, after putting it off forever).

Congrats to the Haystack team!


Bah, I guess hosting a site at digitalocean isn't suitable for HN front page? Even WP super cache isn't saving me.


Just throw it behind CloudFlare quick, they'll cache it and you'll be set.

They're pretty reputable for free speech and fighting for their customers too, so I trust them to a decent degree.


Agreed. CloudFlare is a simple way to add some caching power to a site for free. Or for a few bucks a month enable a bunch of other features.


If you don't want to change your nameservers to Cloudflare (Takes up to 24 hours), try Kloudsec, and enable Offline Protection. Caches (like Cloudflare), and better, it archives your site even if your origin server is down.

For Kloudsec, all you have to do is to update your A records to point to the CDN IP.


A few notes on why I'd stick with CloudFlare personally: In my experience the NS transfer takes 20 minutes tops. Part of the idea with CloudFlare being nameserver on down is that DNS is a great thing to hit if you're performing a DDoS attack.

Also, CloudFlare does offer the offline mode feature, it's called "Always Online". It's also free for more than 1 page, unlike the Kloudsec one.

It also seems weird to me that they bill for their "webshield" on a per-attack basis... someone firing an automated scanner against you could be costing you money big time.

These guys also don't have the same reputation as CF for being bulletproof (though I haven't heard bad things about them yet either) and they have some limits on their free plan which CF does not have.

Interesting though - always glad to see some competition brewing. I know where I'm taking my business if CF steers me wrong somehow. This looks a lot cheaper than other alternatives.


Add CloudFlare? ;-)


Stop using wordpress.


At a minimum, add some caching in front of WordPress. Even a 1 minute cache of pages will result in being able to host your WordPress site just about anywhere. Throw in a CDN, and you could almost host it on a Raspberry Pi and throw Reddit at it.


This is the correct solution. Not sure why it's downvoted.


It should be fine. The problem is more likely that your stack isn't put together very well. Don't throw more hardware/dollars at it, refine your stack.


Awesome, can't wait to upgrade tonight! Any news on the Linux / CLI version?


PLEASE give us a Linux version!


Can I preorder or sponsor a Linux version somewhere?


+1 :)


If I understand well, Arq does not backup the whole computer. I am looking for a tool that allows for recovery of single files, but also that has a backup of the whole machine, so that if the HD crashes it is possible to rebuild it verbatim. What does HN suggest for such a tool? I used Norton Ghost for years (but it has been discontinued).


Either of SuperDuper! http://www.shirt-pocket.com/SuperDuper/SuperDuperDescription... or Carbon Copy Cloner https://bombich.com/ is great for making a complete, bootable copy of a drive.


What OS? Windows, OSX, Linux, or...?

On Windows, my favorite by far is ShadowProtect. It's actually a sector-by-sector backup that pretends to be a file-by-file backup when you restore a file.

What's great about this is when you modify a huge file, it only backs up the actual sectors you changed. (Most Windows backup programs detect changes on a file-by-file basis.) I have it set to run a full incremental backup every 15 minutes; the backup typically takes 15-30 seconds and is unnoticeable when it happens.

Even though it's a sector backup, you can still restore specific files or do a complete system recovery, either to the same/compatible device or with a Hardware Independent Restore to different hardware. I've done each of these many times.

On OSX, I'm using Time Machine, but I wish there were something as good as ShadowProtect. It is a bummer when I touch a few sectors in a huge file like a VM disk image, and the best Time Machine can do is back up the entire file again.


> It is a bummer when I touch a few sectors in a huge file like a VM disk image, and the best Time Machine can do is back up the entire file again.

You could exclude those kinds of files from Time Machine, and use your VM's own backup system.


Do you really need to protect the entire system? Or have you not captured the setup of your system in a provisioning tool like ansible or chef or the like? And then protect only the necessary files to restore the configuration and user data of the system.

Protecting the entire system should be something that is done near line so if you have a catastrophic loss your time to recovery is less than that if everything was stored somewhere in the cloud. Or Just not do it at all and rely on a tool manage the configuration of your system.

Data files and configuration are really the best thing to protect with a tool like this. If you have a total loss your playbook should include something like replacing the failed hardware, Installing and patching the OS, replaying configuration of the system using ansible or chef, restoring data files.

To me this is the fundamental gap that cloud backup solutions need to fill to really capture the consumer market well. The SMB market already has this as you goto IT and get your system reloaded and then restore your user data, its pretty much the standard for larger corps.


In addition to SuperDuper and Carbon Copy Cloner there is ChronoSync.

http://www.econtechnologies.com/chronosync/overview.html

SuperDuper is the easiest to use, and I would recommend it to anyone as their first choice.

CCC and ChronoSync both have more advanced features: including bootable network copies, and backing the recovery partitions.

ChronoSync also offers two-way folder sync's, enterprise features, and more.


What is there for Windows?


Window's folks generally refer to this as "ghosting", after the discontinued Norton Ghost. I have used Acronis in the past, but there are alternatives:

http://alternativeto.net/software/norton-ghost/?platform=win...


I loved Acronis TrueImage for years until it became bloated. Macrium Reflect has since replaced it. There's a Free edition for you to try out.


I suggest Arq. You're incorrect about not backing up the whole computer: You can tell Arq to back up an entire drive.


I could not recommend Carbon Copy Cloner enough. It makes your external hard disk bootable and verbatim has the same files as your mac. This is for Mac only. Think of a dd with a nice interface. Costs $40.

https://bombich.com/


Windows 7 has this included. They discontinued it for 8 and 10, I guess it was harming the backup market.


System Image backup was re-enabled shortly after due to all the complaints. It is still available in 8, 8.1 and 10.

http://www.tenforums.com/tutorials/5495-system-image-create-...


Veeam Endpoint Backup Free [1] is Windows only but can do incremental file level and volume backups.

[1] https://www.veeam.com/endpoint-backup-free.html



for disk deaths I'd just settle with raid. life should be simple.


RAID does nothing in case of `rm -rf /`

the physical death of a drive is not the only reason you may find yourself needing a bootable backup:

* accidentally deletion of system files * theft, fire * system update that goes wrong


Not always feasible (laptops, macs).


my netbook from 2003 accepts two ssd (Asus eeepc 1000). you should stop getting work equipment because of looks or peer pressure.


raid isn't for backup. it's for uptime.


what i suggest was not for back up, it's for disk death. it was a one line comment...


Stefan, can you disclose how many people are working on the Arq now?

For some reason I always had an impression that you were impressively managing the whole thing just by yourself.


Love arq too. Rumor has it arq 5 will also stop eating up free space with its cache.noindex folder. Excited about that!


True! It uses sqlite databases (finally). (I work at Haystack Software :)


This update is great. Very happy I bought Arq :).

I was excluding cache.noindex from my other backup system (carbon copy cloner). Is it still safe to do so?


Yes


Tried to purchase, but it sends me to a bad URL:

  $ host store.arqbackup.com
  Host store.arqbackup.com not found: 2(SERVFAIL)


Sorry about that. Should work now.


Thanks. I now get to the store, but it gets a 500 Internal Error trying to verify my key. Also, the Purchase link does embed my Arq 4 license key, but in the form, the upgrade license key field is empty, and I have to find it from my old .arqLicense file and paste it. I don't even know if I'm copying the right thing.


They just added a "Retrieve license code" link on the web site below the Arq 4 license code field. I was able to upgrade no problem. FYI.


Same here. Let me know if you figure this out. I'll do the same. I just shot Stefan an email but I'm sure he's inundated at the moment...


Having same key verification problems too. Will try again tomorrow!


Still having the same problem today...



Hmm, there's nothing on the announcement about how to upgrade your existing backups?

Say you have a large backup stored on S3 or Google Drive - has anybody used this, and can tell me if it upgrades it seamlessly to take advantage of the new features (e.g. LZ4 compression), or if you need to do a fresh upgrade?


Yeah, I also curious about this one.


I like Arq, use it as a "real, offsite" backup complement to Time Machine. Not quite as fire-and-forget as Time Machine, but darn close!

And the format of the stored data is available, which is a nice safety feature in case of major problems.


What compression was used before Arq 5? lz4 is super fast, but not particularly space-efficient compared to some slower compression algorithms. Since Arq customers are the ones paying the storage bills, this doesn't seem like an entirely costless decision--Arq is now faster, but you should expect your storage bills to go up a bit due to the lower compression.

I use Borg backup with lz4 compression, so I definitely don't think this is the wrong decision, just something to keep in mind (and, it does seem like something that could and maybe should be user-configurable).


Arq's data format description says that the other supported compression type is gzip, which I assume was used before: https://www.arqbackup.com/arq_data_format.txt


I'm also curious about the choice of LZ4. It's fast, but its compression is pretty awful.

For example, on simple, regular text (JSON) files I'm seeing about 45% worse compression than plain gzip (default compression level). I hope at least it's using the highest compression level, but I've found that "lz4 -9" is about as slow as "gzip -6", still with worse compression.

I'd be happier if the choice of compression was dependent on the size of files. The larger a file, the more you gain from compression.

Does Arq skip compression for already-compressed files (.gz, .xz, .bz2 etc.)?


Curious decision, as lz4's terrible compression ratio burns you both in monthly storage fees, as well as Glacier (or other service) restore fees.

Most of my backup set is pictures and compressed files, so maybe the lz4 decision was because they figured the majority of their customers fell in that category where no compression algorithm is going to help?


Still no support for S3's "Infrequent Access" storage class? It would be perfect for backups and is a good compromise between normal S3 and Glacier.


It does support that: http://i.imgur.com/EC1e83h.jpg.


Great! It should have been mentioned in the announcement because I've been waiting for this option a long time.


Has anyone tried both this and Crashplan? My Crashplan 10-computer subscription comes up for renewal soon so I'd be interested in people's experiences of how they compare. Generally I've found Crashplan to be pretty good. I like the way it backs up in frequent small increments and I like the backup destination options (cloud, folder, other PC, etc.). Does Arq compare well?


Arq can’t send backups directly to a hard drive, but it can use an SSH (SFTP) server as a destination.

[Edit: Apparently the release notes for Arq 5 says it can now back up to a local folder.]

I have never tried Crashplan though. The main reason I chose Arq instead is the impression that Arq has a better thought-out encryption scheme.


It appears Arq's being destroyed by HN currently -- I downloaded the trial for Mac, but when I press "Start Trial" I get "Failed to create trial - A server with the specified hostname could not be found." I'll try again tomorrow.


I use Arq to backup all my stuff hourly to my NAS at home through SFTP and the NAS then backups the most important stuff to AWS daily. That works really well and keeps the cost really low because i need the NAS anyway for streaming media an such.


I am a happy user of duplicity [1]. Discussed here [2]. Some comparison with Arq here [3]. It is more oriented towards servers and system administrators, but works pretty well and has a similar feature set.

[1] http://duplicity.nongnu.org/

[2] https://news.ycombinator.com/item?id=6712244

[3] https://news.ycombinator.com/item?id=6712411


The Mac version has gotten a lot of good response, so I tried installing the Windows version. It feels a bit like a beta sadly.

  - Backups to local folder do not work
  - Scheduled backups don't always run until you open the GUI
  - Open log viewer before any backups have been performed crashes the app
  - File-Exit does nothing
  - Wizard adds whole C:\ to first destination
  - Doesn't seem to backup (all) locked files
I've reported all of these, and the author seems responsive, so I hope it gets better.


Awesome, got an email about an hour ago stating I received a free upgrade to 5 from my 4 license. Been using Arq for a couple years now, it's been a perfect solution for offsite backups.


We upgraded all Arq 4 purchases in the prior 90 days to Arq 5 for free. We even converted the per-computer licenses to per-user licenses, so if you bought licenses for 3 computers, you now have Arq 5 licenses for 3 users.


Fantastic!


Off topic: Is anyone else getting an SSL error trying to connect? This page opens fine on my Mac (OS X 10.11.4, Safari 9.1) but my iPhone (iOS 9.3.1) asks me to trust the certificate (ssl376366.cloudflaressl.com)

Edit: Screenshots: https://imgur.com/a/wdfDt

Also, this appears to not happen on Wi-Fi. Willing to send any information I can gleam from this -- I'd be interested to know if this is a MITM attack or a mistake on CloudFlare's side.


Argh, the website is down...lol:

"Error establishing a database connection"


It appears that there may be an issue with just their blog. The main site[0], and most of its links, works for me.

[0] https://www.arqbackup.com/


At the moment, this site takes ~30 seconds for every single request. But at least it works at all.


I was a user of Arq 3, but it was slow and seemed to eat up resources on my computer. I eventually turned it off, deleted the backups and deleted it. Not a great backup plan.

So I'm keen to know how much faster Arq 4 was, and in turn Arq 5. I'd be happy to try again (I think I'll have to pay full price again as I'm not an Arq 4 user) - but might wait until someone can let me know just how much faster it really is.

This whole thread has also reminded me to run my backups to local Time Machine!


This sounds great. To my mind, it seems an approximation of Tarsnap but on Windows.

Is it possible to use Arq to backup to network connected disks via Windows SMB? In other words, is it necessary for me to use one of the supported cloud providers or can I just use one of my own servers as a destination? Similarly, is it possible to setup multiple redundant backup destinations (e.g., S3 and Google Drive, or S3 and my own servers)?


The release notes at https://www.arqbackup.com/download/arq5_release_notes.html mention that you can now backup to a local directory or network drive. It's weird that the most interesting features were not actually mentioned in the main announcement.


Fantastic. I think I'm going to give Arq a trial and see how it works for my use-case.


Didn't JungleDisk (http://jungledisk.com) kind of start this "backup to AWS, control your own data" movement? Does anyone use them anymore? I recall that they started as pay one price, then switched to a subscription model too...

I wound up on Crashplan, though it's not perfect either.


One long standing issue I have with Arq is that there is no easy way to duplicate backup selections. I would love to be able to easily copy the selections between different destinations, or different systems that are being backed up.

Several years ago, I tried to manually edit the plist preference files, but that was painful and unsustainable hack.


Thanks for the tip. I'm checking out Arq now!

FWIW I've been a big fan of Syncovery (formerly SuperFlexible File Synchronizer) for years. It's a Swiss Army Knife of backup/recovery. https://www.syncovery.com/


Syncovery looks pretty awesome for what it does, I just installed it to take a look, - Directory Synchronizaton - but it's achilles heel for file backups (as opposed to directory/disk cloning) - is appears to be it's lack of any type of versioning. I.E. If I make changes to a file each day, and do backups each day - no way to go back to a version of the file a couple weeks ago?


It has a versioning feature IIRC but I can't vouch for how sexy it is.


Been using Arq since version 2 (2010) on multiple machines. Very happy with Arq and with Stefan's support. I use it with dual destinations for my offsite backup - about 9Tb. I also use it to provide family in another country with a viable offsite backup. Restores just work.


Any of you know how to get your license code for an upgrade? I have the original email from when I bought Arq 4 in 2014, and it contains a license file. This is an XML with sections like 'id' and 'licenseKey' but neither of those are working.


I don’t know, but why not just contact the author? He’s very pleasant to deal with in my experience.


To add, they sorted it out by adding an option on the site to retrieve the license code. I've now got an Arq 5 license and lifetime upgrades.

I think that having Arq back up to spare space at a Linode VM is a great way to use it.


I've emailed, but no response yet (I figure the support folks are very overwhelmed with a new release) so I thought I'd ask here as well.


As an Arq user, this makes me happy, as Arq is not exactly known for being fast.

The announcement doesn't say, but I'm hoping they have reduced the amount of space needed for the client-side cache (currently 18GB (!) on my laptop).


Yes. I just added that to the announcement post. It stores stuff in sqlite databases that are a lot smaller.


It went from 18GB to 600MB! That's awesome, thanks!


Trying to buy my upgrade now - but it keeps telling me my Arq 4 license key is invalid.

I've tried both the licenseKey field, and the hash field from the license XML file I got when I bought Arq 4.

Has anybody else had success buying an upgrade license?


Can I use Arq to backup disks connected to my airport extreme router? I don't have a desktop; will be installing it on my laptop. Can I schedule when to backup ?


been a user since arq 2. It's the best.

Question to any knowledgeable folks: Why does the tool cause so much download in normal operation ? (i.e. no restoring of files)

I have about 60gb backed up at rest, and am on a backup-every-2h-cycle. I generate about 6 GB of download traffic each month. So funnily enough, the traffic costs me more than the actual storage :) mind you it's still next to nothing, but I found it just curious.


I guess those people with less complicated needs could use https://cryptomator.org/ to upload the $60/year Amazon Cloud Drive.


$50/year and you have to provide your own storage? Can somebody explain to me how they are competitive, i.e. in comparison with Crashplan or Backblaze?

edit: my bad, it's $50 for one time purchase, not a year.


Its $50 one time purchase for new purchase. The upgrade from Arq 4 is $25. During store checkout, you can also choose to add a "Lifetime Upgrades" at $30 which will give you free upgrades for Arq 6, 7, 8, 9, 10, 11, 12 etc.


I purchased the Lifetime Upgrades in September 2015. Now it tells me there is a $25 upgrade fee to go to version 5. I emailed support a few hours ago to confirm. If this is true, I am extremely disappointed and feel cheated. Notice on the form to buy a new license it doesn't say anything about "Lifetime Upgrades for this major version _only_ which will be obsolete in a few months". I sure hope this is just a bug that they will fix.


There is an apology for the delay on twitter. Looks like your license for Arq 5 should be coming.

https://twitter.com/arqbackup/status/719924424263249920


Thanks! My faith in them has been restored :)


We're sending Arq 5 license codes to all lifetime purchasers. Sorry for the delay. If you don't get one today please email [email protected].


It's a one time fee for really awesome software. Upgrades are every couple years, and only about $25. My backup bills on AWS have never been more than $3/month (I only backup my actual documents) - and you have total control - don't have to worry about the policy engine at BackBlaze/CrashPlan arbitrarily deleting old hard drives that haven't been plugged in.


It's not per year, it's for current major version (if you don't want lifetime upgrade option).

edit: You get a free upgrade if there is a major version released within 90 days of your order (if you don't order lifetime upgrade of course)


Where do you see $50/year? From what I can gather, that is a one-time fee.


Even at $50/yr, what are the options for locally-controlled encrypted backups? Tarsnap, but it has a less friendly UI.


It also has no wireless.


Not sure where you see that?

On the features page, https://www.arqbackup.com/features/, under "Limit Network Impact", the image lists whether to use all wireless or no wireless.

I don't have the software so I can't confirm.


It is probably a reference to the (in)famous launch review of iPod on Slashdot: https://slashdot.org/story/01/10/23/1816257/apple-releases-i...

> No wireless. Less space than a nomad. Lame.


I find it a bit ironic that this particular blog has an inaccessible database at this particular time.

This being put aside I find their pricing a bit excessive in comparison to Backblaze for example.


How much data are you backing up? The compression and deduplication seem pretty good in Arq. My monthly AWS (and now Google Nearline) bill is usually around $2, so I hit the breakeven point vs the $5/month services pretty quickly.


My current backup is around 300 Gigs. I see the big advantage of Arq (over Backblaze) is the fact that you can choose your host. However their own hosting solution for $10/mo with a 250GB data cap is quite expensive. I can see that for smaller amounts of data Arq is more interesting, especially since it also stores history (which Backblaze does not).


It's worth noting that Arq can back up to Amazon Cloud Drive, which offers unlimited storage for $59.99 a year. As soon as they offered that I switched from Glacier storage; Cloud Driver is faster, cheaper and less fiddly.


That is quite good! Unfortunately, in France there is no Unlimited Everything plan. The prices are similar to the UK (comment from mobiuscog).

However there is one last quirk I have with Arq + Hosting Service. If you need to retrieve your backup _fast_ you will have to reach for a paid service such as AWS Snowball which seems quite expensive. I can't to seem to find more recent information about them shipping smaller disks, just some old articles.


the pricing is great for the US... not so good for the rest of the world (for example: https://www.amazon.co.uk/clouddrive/pricing?ref_=cd_home_pla...)


That’s weird. I live in Sweden and I was able to sign up for the $59.99 unlimited plan just by surfing to https://www.amazon.com/clouddrive/home


For Canadians, it's $60 CAD.


I didn't even know they offered that until now. I viewed the prime advantage of Arq was selecting your host. And really that is "hosts", since you can have multiple backup sets and different hosts. Stuff that I often want to retrieve I might have on Google Drive as well as one of the "cold storage" options.


This. Looks. Awesome.


"Error establishing a database connection"


Fixed now. Sorry. The load from HN was too much for a while.


Can't start a trial account.

Failed to activate. The remote name could not be resolved. store.arqbackup.com




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: