Protect your data from ransomware with S3 Object Lock

Protect your data from ransomware with S3 Object Lock(blog.symops.com)

58 points by abuggia 4 years ago | 19 comments

jis 4 years ago |

A less permanent solution is to use versioned buckets with MFA Delete turned on. You can then cleanup versions if you need to by disabling MFA delete, which requires the MFA to do. So as long as your MFA device is not on-line, then if someone compromises your servers, they cannot disable MFA delete and cannot remove versioned objects.

prirun 4 years ago |

Object Lock may be useful to protect backups from deletion, but ransomware is now relying on the threat of data exposure more, where Object Lock makes no difference:

https://www.theregister.com/2022/06/25/ransomware_gangs_exto...

"Increasingly, however, cybercrime rings still tracked as ransomware operators are turning toward primarily data theft and extortion – and skipping the encryption step altogether. Rather than scramble files and demand payment for the decryption keys, and all the faff in between in facilitating that, simply exfiltrating the data and demanding a fee to not leak it all is just as effective. This shift has been ongoing for many months, and is now virtually unavoidable."

https://www.theregister.com/2022/06/03/fbi_cisa_warn_karakur...

gregwebs 4 years ago | |

There is a difference for the business though – although the data may be exposed, the business may still be able to maintain continuity. I believe it is the lack of business continuity that has made ransomware so powerful – when a business can no longer function it will do anything to get that back. If the business could continue to operate they would be in a much better position to refuse to pay the ransom. If all businesses refused to pay the ransoms, ransomware would stop.

Obviously it’s still really bad if sensitive information is exposed. But also consider that some of the information essential for business continuity would be less sensitive in a public exposure scenario.

So in some cases it is just as effective, but in many cases it is not. As I understand it, most ransomware providers still attempt both encryption and exfiltration. Exfiltration is now standard not because it is easier but because more companies are able to restore operations from backup.

josephcsible 4 years ago |

> No one, including AWS can shorten the retention period

I don't think this is true. Isn't it more like you can't, and AWS can but promises that they won't?

HyperSane 4 years ago | |

Yes, AWS must still be able to delete the data if you stop paying but there must be a LOT of safeguards in place so Object Lock still makes it impossible for anyone outside of AWS to delete the data.

neilv 4 years ago |

I made this a part of the backup schemes at a startup. Everything (GitLab, PostgreSQL, the biz people's document&storage SaaSes, etc.) got backed up multiple ways, including to S3 objects with long retention periods.

So long as the data wasn't corrupted before it hit one of the designated S3 buckets, the risks of the retention periods seemed minor for that startup at the time: basically, having data there that we wanted to delete, but couldn't.

(For an example risk of not being able to delete, the most likely scenario might be accidentally copying a gazillabytes of objects to the magic bucket that can't be emptied for X months, so having to pay for that storage. For a different scenario, the nature of our business, together with our security assurances and practices, meant that we never handled data that would be improper to store there, such that we urgently needed to delete it. In a highly unlikely scenario of an attack putting very evil data there, we'd have to report the incident to government authorities in any case, and we could effectively cut off our own access to it, while AWS preserves gov't access to it.)

ineedasername 4 years ago |

I use backblaze which will backup any modified file and has copies back 30 days, longer if desired. It would be great if there was some way (is there?) To flag if a file was recently encrypted to detect potential ransomware early before backups are corrupted also. Seems like a changed file being backed up could be compared to the previous unencrypted file for this purpose.

Of course this would be more complicated if the files were user-encrypted in the first place, but good tools to detect these things asap would have a good market above & beyond antivirus that is more focused on detection of the virus and not a complete packaged of file recovery as well.

tgsovlerkhgsel 4 years ago |

From my understanding, you can't delete a bucket that has objects and I assume you can't empty a bucket that has locked objects, but what happens if you close the entire account?

AWS recommends to use accounts for segmentation, and has APIs to manage accounts (https://docs.aws.amazon.com/wellarchitected/latest/security-..., https://docs.aws.amazon.com/cli/latest/reference/organizatio...).

There is a 90 day recovery period, but I can think edge cases where this isn't enough. There is some data that companies need exactly once a year for financial reporting/audits etc. but losing it is seriously expensive. Imagine the person managing that data being disgruntled and closing the account holding the data (and just that data, so the deletion isn't noticed) 6 months before the next audit.

Can you close an account with locked data in it? If yes, is the data actually deleted, or could the org recover it?

Another interesting scenario that could test how reliable the lock is would be a griefer (i.e. an actor that wants to cause damage without profiting from it) who gets access to an organization's account, uploads a large amount of data, and locks it. Will Amazon simply keep the data and waive the cost, or will support unlock the data, or will the organization be forced to pay up? The latter two both have interesting implications (compromised support agents and social engineering in the first case, extortion in the second case - think "we have taken over your AWS account and created several paths of access, send XX bitcoin or we'll lock this exabyte of data and you'll pay $XXXXXXXX in AWS fees, if you start taking away our access or deleting the data we'll see before you finish and use one of the remaining accounts to lock the data").

jeffalyanak 4 years ago | |

It's much harder to hide the fact that you have closed an account completely and much easier to put in a periodic check that the account still exists.

The difficulty with ransomware is that—without object lock—you would need to check that all data is still valid. That is usually going to be very difficult to do and any heuristic checks are liable to miss some cases.

On the other hand, checking that an account is still extant is easy and, since that's an operation that should not be undertaken without a whole oversight process, you can significantly limit who has the permission to do so.

tgsovlerkhgsel 4 years ago | | |

Oh, I totally agree that this solves at least 99% of the problem. I'm just wondering if this is an edge case that remains.

gregwebs 4 years ago |

One thing missing from this great introduction is how to enforce object lock for new objects. This can be done by setting a default retention mode and period on the bucket and having a bucket policy for s3:PutObject(Retention). It looks like I should do a writeup on this. I am working on tooling to make using Object Lock (to prevent ransomware) more convenient.

m1keil 4 years ago |

Just FYI, you can only enable object lock at bucket's creation time. You cannot enable it later on.

drodgers 4 years ago | |

It can be enabled later, but you need to contact support and have them do it

mastax 4 years ago |

What happens if you lock objects then stop paying your bill?

HyperSane 4 years ago | |

AWS must be able to delete the data but since you stopped paying it must not be important to you anymore. No one outside of AWS can delete the data and presumably no one inside AWS can delete Object Locked data IF your bill is payed.

mathgladiator 4 years ago | | |

I'm almost sure S3 keeps data and eats the cost. Although, my understanding is from 7 years ago when I worked on the team that would have implemented it.

If they do, then it probably would require a human to run a tool. It would have to be big bill to warrant that.

djbusby 4 years ago |

Can't ZFS do these kinds of things with snapshot?

implements 4 years ago | |

Yes, but make sure any machine vulnerable to ransomware infection doesn’t have the ZFS server root account credential, directly or by reusing it as a share password, because even though snapshots are immutable root can still delete them.