Let's Encrypt obeys CAA, if you try to validate for a domain which has a CAA record saying e.g. "Only Symantec may issue for this domain", Let's Encrypt's software will reject the validation. Current Baseline Requirements don't require this of a CA (they're required to document what they do with CAA but most picked "soft fail" ie they will issue but maybe do extra scrutiny) because they feared it would be used somehow in an anti-competitive way.

Client software like Firefox, or Internet Explorer, ignores CAA altogether, as described in the design of CAA.

Could you describe what kind of concerns you're thinking about that would be Let's Encrypt-specific and that would only affect you if you actively use a Let's Encrypt-issued certificate? Generally speaking, a CA key being compromised would affect everyone (unless they use key pinning mechanisms like HPKP, which doesn't get a whole lot of real-world use at the moment, and probably close to zero usage from customers using a shared web hosting plan unless the provider takes care of it). You don't need to actively use a CA to be affected.

> It could also complicate matters when renewing certs. If I forget to renew my SSL, it gets replaced with Let's Encrypt, and then I go to renew it and the system gets confused because it looks like I already have a cert from another provider.

CAs typically don't care about whether you already have a certificate from a different provider (I mean, why would they?), or were you thinking about possible limitations in OVH's system? In that case, this only seems like an issue if they indeed have no way to replace the certificate once it's been enrolled with Let's Encrypt.

There is no way to induce a vulnerability by using an incompetent or malicious CA, provided you generate your own, strong private key. Even issuing an MD5 or SHA-1 certificate cannot actively harm your visitors unless a second preimage attack is developed against the algorithm (in which case, again, everyone is screwed, not just you).