Tl;dr. They go faster than the defence response cycle, either the wetware or actual hardware. There is an open question about the ability to make course corrections at such speeds, but that can be fixed through warhead size or choice.

Ukraine is showing that unmanned weaponry is putting manned weaponry at greater risk than anticipated or accepted. When you have a few hundred top line fighters in the entire arsenal, manpads look like a real problem. And stealth has at least two problems: vulnerability to changes in detection technology and weapons that fill volumes of space with particulates.

And dinky little drones do wonders for extending the range and performance of the mk I eyeball.