In theory, yeah.

Power supply failures happen a lot, also spinning magnetic disks fail a lot. So do moving parts like fans. So if you simplify a machine that much it probably gets more durable. There is also the surface area vs. volume scaling that means small lightweight things survive getting dropped on the floor (it's why a spider is so proportionally stronger than a person)

Also you could probably figure out (and afford) some backup solution that copies the whole stick, whereas there has never been a truly practical system for backing up a conventional PC.