It's already hard enough to train competent pilots. Who's going to make sure Joe Average is safe to fly over top of houses and power plants and city centers? There are about 6 million car crashes per year in the USA. Do you seriously want those happening over top of you?

Air traffic control is already hard enough with only 5000 airplanes flying in American airspace at any given moment. Who's going to manage it with 5 million airplanes?

Where are these flying cars going to land and take off? You need a lot of runway to get airborne and land unless you have a VTOL vehicle, which, no, you won't.

How is the military going to detect a missile in that mass of 5 million airborne flying cars? And what's to stop a terrorist cell from sending a swarm of autopilot cars loaded with fertilizer into the pentagon? Or the White House? All vehicular safety measures are 2-dimensional (barriers, walls, gates, moats, roads). That all goes out the window once 3 dimensions are in play.

Flying cars aren't a technical problem. Making them fly is the easy part by orders of magnitude (in fact, many flying cars have already been built over the years).