My assumption is that the cryptographic keys will not be able to leave the HSM. This means that the HSM itself will need to perform cryptographic operations. You send it your data (e.g. a blob to decrypt), and the HSM will decrypt it for you using its embedded keys. Similarly it would implement other cryptographic primitives. To unlock the device to perform cryptographic operations, presumably some kind of pass phrase is required. That pass phrase would need to be online for applications to work in an unattended way. Either in memory or on disk.

So it would seem this would protect against someone stealing your data surreptitiously. The only way to decrypt that data is via the HSM, and presumably the HSM has untamperable access logs. Suppose data is stolen. Then after fixing the attack vector and making sure your systems are trustable again, you'd change the HSM pass phrase, and make sure there are no unexpected entries in the HSM audit log. At that point you can consider your private data to be safe again.

However it seems that an active attack is still possible. If an attacker gets into an EC2 instance with HSM encrypted data and an online pass phrase, he can just use the HSM to decrypt that data. Unless you watch your HSM audit logs really closely, the attacker could have stolen your data before you even notice.

Also it would still be possible for a govt. agency to seize the HSM and the instance with an online passphrase. So this would do little for EU companies that cannot legally have their data become in scope for the PATRIOT act.

[Edit: spelling]