A somewhat daunting project that I've been putting off for a long time is organizing and backing up 23 TiB of files spread across 40+ external and internal hard drives that I've collected throughout my life. There is a variety of filesystem types and interface types.

I take a lot of photos, so a lot of these are files that I would actually want to back up, but many of them are old operating system installs and other "useless" files that I don't need archival storage for.

The actual size of the data that I need backed up I would estimate at around 6 TiB.

A few of my requirements:

1. I don't need the files to be accessible online, in fact, I would prefer if they were not.

2. If anything is backed up to the cloud, I want pre-internet-encryption with keys that only I know and control.

3. I want something simple, that could be recovered using a pragmatic approach and open source software in case of a disaster.

4. I'd like a system where I can easily test my recovery strategy.

Open questions:

1. What local filesystem setup should I use? Number of drives? Local backup approach?

2. If you've done this before, is there a strategy that you used for the actual aggregation of the data? Are there any particularly convenient IDE to USB docks? Any good software that you would recommend for locating duplicate files?

3. What remote backup software should I use?

[ edits ]

Answering some questions from the comments:

Cost: Given a quick look at the cost of archival cloud storage, I guess I would be willing to spend up to $60 per month on a remote copy. (Noting the estimate of 6 TiB of "acutal" data)

How often: I would expect to access the remote files rarely (maybe once a month), and need a complete recovery very rarely (with a low requirement for recovery speed). Local backups I would like to occur at least weekly, with a verification or access frequently (daily?).

Risk tolerance: For the local-encryption-for-remote-storage aspect, I would like something with a high level of confidence in the cryptography and the implementation surrounding it. I would also like a high degree of confidence that I can recover my files in case of a natural disaster or similar that wipes out my local copies.

Local security: I live in a relatively secure home in a relatively low crime area. I could store a copy at a relative's house, although I may move far enough away soon that it would not be practical to deliver or access such a backup.

user:	sumoboy
created:	February 8, 2016
karma:	392
about:	web dev, ppc, analytics, photographer, and 15 other hats for what ever needs to be fixed.