And you'd have to balance all this against the wealth of severe X security bugs that have been coming out even just this week, and the maintenance burden of supporting it in a volunteer project with limited backing.
There's an argument that they are being too conservative. Other DEs like GNOME have already dropped X.
Then again, it's FOSS, so expectations need to match that.