How I encrypt my data in the cloud(robertclarke.com) |
How I encrypt my data in the cloud(robertclarke.com) |
- Tars folders I want to backup
- Encrypts using GnuPG
- Uploads encrypted files to S3 (Glacier).
Simple and cheap cloud backup for me + nothing had to be installed on my NAS except for docker to run GnuPG and AWS CLI in containers.
security tools = 10 opsec = 0
For my personal backups I use a combination of tar, pixz, and GnuPG. There's no fancy deduplication, and it's definitely not efficient, but it's relatively simple and I can restore individual files with ease.
I run a variation of the following command occasionally:
tar -C / \
--exclude='dev/*' \
--exclude='home/*/.cache' \
--exclude='lost+found' \
--exclude='mnt/*' \
--exclude='proc/*' \
--exclude='run/*' \
--exclude='sys/*' \
--exclude='tmp/*' \
--exclude='var/cache/*' \
--exclude='var/lib/docker*' \
-cvf - . | pixz | gpg2 -e -r $PGPID \
| ssh host 'cat > /backup/root.tpxz.gpg'
Wasabi may not be the cheapest storage solution, but they have no egress charges, which makes recovery a non-issue. Good speeds and S3 compatibility are also great. Don't want to run an ad for them, just a happy customer.
The Wasabi offering seems to be equivalent to the "hot" storage options, which, I agree, is crazy expensive in the cloud if you do a lot of egress.
But they're a great fit for my personal use case of well below 1TB of rarely accessed data, while also providing peace of mind that recovery is not an issue, so I'm OK with paying more for that. Plus, it feels good betting on an underdog. :)
The big thing I'm missing from Duplicity are incremental backups, which is not a strong requirement for my use case as bandwidth is cheap and I can delete the oldest N backups to free up space.
But I gain a lot from using a combination of tools: I can easily replace each component, and easily improve my workflow by adding more components, such as deduplication or incremental backups if needed.
I use EncFS for other purposes, but be aware of its security issues[1]. This report was influenced by the founder and CTO of Boxcryptor, so I'd take it with a grain of salt, but I'd still avoid using EncFS for any important data.
Though I'm OK with trading some usability features for security and peace of mind.
On Linux, I use restic, which can also backup to B2 (and via sftp, obviously). restic has this nice feature where you can mount the backups at some destination as a FUSE filesystem. Makes it very easy to go through backups and recovering the bits you need.
You can store files/backups/whatever encrypted with support for many popular endpoints (local filesystem, S3, Backblaze B2).
Then I use a bunch of free cloud providers (including TransIP STACK who gave 1 TB for free at some point) together with Cryptomator [1] which is a cross-platform (Windows, Linux, macOS, Android, and Cyberduck/Mountainduck also support it) Java program. The advantage of it, is it abstracts the filesystem and WebDAV. So you see the decrypted data on a separate filesystem layer, allowing all your normal applications to work. It is also FOSS and gratis.
Is it the best option? I don't know. I like the mentioned advantages. I've never used Arq, for example, but it not working on Linux and Android is a dealbreaker for me.
As for cold wallets, quoting the article:
> Offline wallets are the best way to go for storing a larger amount of cryptocurrency. I use ColdTi wallets to store multi-sig private keys. ColdTi is essentially just a slab of titanium that comes up with a punch set that can be used as a fire-proof seed backup. Very handy :)
These are useless in a case of fire.
[1] Already mentioned multiple times in other posts at the time I wrote this. https://cryptomator.org
On the other hand, I ended up learning about scrypt(written by Collin Percival, who works on FreeBSD a lot and runs Tarsnap), and restic which in layman's view appears to be a better borg.
Also, it's not quite the same functionality, but this also reminds me: For a long time I've used Knox (by AgileBits, the same company that makes 1Password) for encrypted disk images, but they no longer sell or maintain it. It works just fine, but I should probably find a replacement that's still maintained, at least for security updates. Anyone know a good alternative? VeraCrypt (mentioned in the article) seems like one possibility.
Options:
1. Truecrypt container. CON: Upload takes to long
2. ecryptfs CON Always had problems getting it to work. AFAIK it is not under current development anymore.
3. Run a FS in a mounted contaner (Filesystem in a FILE). Slow. Not very stable. Under no circumstance use ext4 or something like it, if you really want to try this, use ZFS to avoid data corruption
4. CryptFS. Great Idea but slow as f.. https://www.cryfs.org/comparison/
In the end I did not the cloud as a second backup for a large system (10TB) since I found no safe, fast and reliable way.
I think they do gain a lot from selling 2TB to people using 30GB and selling additional users of the same <3TB of data to enterprises. (That's gotta be pretty sweet profit if they have takers - $12.50 more a month for zero additional storage and a little more data transfer.)
Also, I use VMs a lot, so LUKS encrypted VDI is my Truecrypt/Veracrypt equivalent. One advantage is the ability to use dynamically allocated DVIs. So VDIs can start small, and grow as you add more data.
Boxcryptor offered a client for macOS, Windows, Android, and iOS that worked really well, and if you needed Linux support, one could install EncFS and use it transparently on that platform. Boxcryptor charged for a creating volumes with more advanced EncFS settings, but if you created the EncFS volume with those advanced settings using EncFS itself (e.g. on a Linux machine), the free version of Boxcryptor could read and write those volumes with those settings.
In 2013, the people who ran Boxcryptor wrote a second version that implemented a proprietary, unpublished encryption and/or file management scheme. They relegated the previous version to an unmaintained Boxcryptor Classic product and eventually removed it.[3] The proprietary version is what is offered today.
IF you want Boxcryptor-like functionality today, the EncFS4win project[4] is a good solution for Windows. EncFS can be installed via Homebrew[5] on macOS and its volumes mounted via a shell script or some FUSE GUI managers. You can install EncFS on Linux and use gencfsm[6] for a GUI manager. The Windows, macOS, and Linux implementations all use FUSE for exposing the encrypted files via a native filesystem interface. For Android, Encdroid provides an application browser for volumes. I am unaware of an iOS solution. I use the FUSE systems to keep certain sensitive cloud documents synced between my Windows, macOS and Linux machines while still being able to edit and use them like normal files on those systems.
EncFS does have a few attack vectors they have been slowly addressing. It also suffers from the same problem that all cloud-synced file-based encryption systems suffer; someone could restore your cloud files to a previous known version without your knowledge. The file-based encryption does not prevent what is in effect a replay attack. A research paper proposed a solution -- CryFS[7] -- with some solutions for this problem, but the implementation is immature.
(edited for formatting)
1. https://www.boxcryptor.com/en/
2. https://vgough.github.io/encfs/
3. https://www.boxcryptor.com/en/blog/post/6-years-of-boxcrypto...
5. https://formulae.brew.sh/formula/encfs
I am happy to report that this has been (recently) solved:
https://www.stavros.io/posts/holy-grail-backups/
"(the) holy grail of backups"
Of course, unless you stand up your own VM with a ZFS partition, there are few cloud options for ZFS.
There is exactly one. You can ZFS send/recv to and from an rsync.net account that is enabled to do that:
https://arstechnica.com/information-technology/2015/12/rsync...
https://www.rsync.net/products/zfsintro.html
OR you can get a plain old rsync.net account and do a "dumb" sync to it and just configure ZFS snapshots on any schedule you like.
Ask about the "HN Discount".
"CryFS solves all of these issues, but because of the increased security it is a bit slower. It is also a very new project and currently only available for Linux and Mac, but has experimental Windows support in the newest version. So if you don't need Windows support today, you can give it a try." https://www.cryfs.org/comparison/
I also remember that my internet provider blocks many ports and I had to use my VPN to get required ports to working. This again scaled things down a little. I found CryFS not usable on Linux
Haven’t tried a full restore, but I occasionally browse old backups and download something as a spot check.
I would much prefer a turn key open source solution that’s commercially developed/supported by one firm (but not tied to that company’s hosting).
Maybe someone will release something under the BSL, and build a healthy company out of it.
This depends on how secret do you want your data to be. You could use block-based encryption/compression and backup. That way you can still dedup encrypted result.
If anyone can inject data into your system and monitor the backup, they could learn when they hit collisions, but for most personal backup cases that's irrelevant.
I think a better approach, if you want to have versionable files, but encrypted outside of the client, would be to do something with diffs, similar to Git, or perhaps staged dockerfile builds, depending on whether it is binary or text data.
What sorts of applications have started to adopt this?
I also thought a different layer to start at would be sqlite databases since I understand that many mobile application use that.
How do applications handle conflicts? It looks like there is a version on files but when is a new version created? On close?
Do you see GDPR or any other compelling event that will cause applications to consider this sort of cloud storage?
As there is transaction control, the conflict handling should be straightforward. That is, the thread got write lock can write the file exclusively and each write is a transaction and commit will form a new permanent version.
GDPR might be a good reason, but I think it can be more general. Any apps need store confidential data can use this, no matter the data is on local or cloud.
This was years ago. It might be fixed.