> Even if our ML systems were meaningfully intelligent, there's still the issue of proper training. You can't teach humans to be a safe drivers by showing them a huge slideshow of dash-cam images.

That's an interesting concept to explore:

I would guess that videos could improve people's driving: Imagine new drivers; showing them videos of different situations, actions, and their outcomes, may help. The same videos might not help an experienced driver, but they might be helped by videos of more complex situations or by videos tailored to a specific driving skill.

But I'd be interested in research: When does such training help people and when does it not? What aspects of the training are effective or not?

And can that be applied to ML? It may be the old fallacy of conceiving of computers as 'thinking' like people, which Dijkstra compared to conceiving of submarines swimming like us.

I don’t think that’s the goal.

You said car, not image classifier.

If errors in any subsystem surface out to the car, well then okay. But it's not unlikely that the overall system would deal with an error in the image classifier.

Pay wall so I can’t read the article. But can someone comment on what 0.001% error means in real world scenarios?

To keep the maths easy, that’s 18 wrong frames/10min. So roughly 2/3 of a second.

If these images are spread out, maybe it’s fine. If it’s failing on certain types of images, then those images could be grouped and 2/3rds of a second is more than enough time for a serious failure.

Don't forget - the error might be reproduce-able. As in, it will error rarely, but at that one obstacle, with that one light config, it will error repeatedly and constantly.

AI related car accidents and deaths will have special places and dates, were they will repeat annually.

Problem is, you are not alone on the road. And if somebody else does something really stupid, you can die as a consequence, without any chance to avoid it.

We are definitely not there yet, but I think of myself as a decent driver, and some assistant tools on high-end models are way better (just faster, probably) than me predicting stuff. For the first year this summer, I've driven a car that was quicker than I in an emergency break situation: while I started breaking, it depressed the pedal, and I was quite surprised, 'cause the car in front of me didn't begin breaking yet.

There's also the fact though, that if you're elderly you can't really avoid being a worse driver. I know particularly in rural areas where there's no public transportation, a lot of elderly people would accept an error rate higher than what their own driving used to be. It seems pretty tough for the rest of us to deny them access to a technology like that while it's in the interstitial stage.

Only if you’re the only driver being replaced.

> Why is Carlsen so relatively bad at chess with his understanding of the why?

I think you have it backwards. Stockfish probably could tell you which specific 30-depth line changed its evaluation, while a human player is much more likely to play based on feel and intuition

Boeing lobbied the FAA for way too long to avoid grounding of the fleet. It did happen so, after it turned out Boeing's SW fixes didn't work.

Also, just how that part of the 737MAX was certified is the real scandal, if you ask me. This is the literal example proofing the rule, so.

The error rate would certainly go up at first due to the additional complexity, but the cost of each error goes down. Besides, I have to switch the speed at which I'm driving far more granularly than at county lines, and no one's worried about a car's ability to handle that.

I don't disagree that sometimes central decision making is good, but in a complicated situation where any decision will have some negative consequences depending on the specific context seems like a textbook case of not being one of them.

That's an interesting problem. I wonder what social discussions need to take to reach acceptance for "AI assisted driving". Mandatory AI breaking to prevent driving into a bus queue?

I'm also curious how to find the people who do object vs theory-crafting all possible concerns people could have.

Why not? If it saves 30,000 lives per year is that not a meaningful improvement? I suspect my understanding/acceptance of "unknown risk while driving" is flawed in some way, or at least very different from the general populace.

Hey man, if we’re rebuilding roads for the sake of self driving cars, let’s just go back to ubiquitous light rail instead, like we had before cars got popular. This whole self-driving industry is so ridiculous when you consider that this has been a solved problem for over a hundred years..

> And third, the world has almost entirely been rebuilt to compensate for human observation flaws.

I don't entirely agree with what I think your point is. Fundamentally, humans are pretty great at using context to work their way through a variety of unfamiliar situations. The work we do on intersections is about tuning. Even in a bad intersection with horrible flaws, 99.9% or more of all humans navigating it will be successful. The reason we keep tuning them is because our tolerance for death is zero. 1 death for every 100M miles driven is pretty good, but many people still find it completely intolerable. We're going to keep tuning.

But I don't think that means that making roads safely navigable by algorithms is going to be a simple matter of tuning them.

While I agree that AI in current forms can be very useful, I believe that the problem of e.g. driverless taxis requires understanding of other humans and empathizing with their intentions to be truly viable. Driving is a social activity, and the current self-driving tech is about as convincing as trying to carry on a conversation with Alexa at a party. I do believe that we need AGI before self-driving will be more than a better cruise control.

I think the better cruise control is very useful and I love to see it, but Tesla’s marketing of it as “full self-driving” is disingenuous as best, and industry-chilling + deadly (as we’ve seen) at worst.

> Even if this is the case, why does it matter?

Because people are using Deep-learning over single-lens cameras to replace depth-perception... and then wondering why the cars that do this run into stationary objects with flashing lights. https://static.nhtsa.gov/odi/inv/2021/INOA-PE21020-1893.PDF

No one really cares about where deep learning works. People are complaining about all the areas where deep learning is failing, with dramatic and deadly results.

Entirely agree. The problem is that most people don't understand that and easily fall prey to thinking that AI is a magical black box that can solve any problem you throw at it. In no small part because of all the hype by salespeople and the media. The reality is that NNs are great but only for some types of problems AND where there is a high tolerance to false positives and negatives. Clearly, this does not include problems where safety is critical (unless you can really demonstrate that AI does better at safeguarding than humans AND you can convince the public to not be scared).

All true. But due to the hype we often see people trying to use overly complex ML models in problem domains where simpler deterministic statistical techniques such as linear regression analysis would be just as accurate with lower costs and better testability.

The issue is that people think that neural net technology is AI when, in fact, there is 0 Intelligence there. Which is fine, except when you have people like Elon Musk, Google and others thinking that there is I there, just need to throw more processors and data at it, and I will magically emerge. We now face a growing number of non-I cars on the road starting to kill people, so it's a real urgent problem that all who use neural nets need to, at minimum, understand that that tech will never give I and to communicate that fact to their customers and their goals with using it.

the problem is the unboundedness of the failure modes of (the current generation of) AI, not with how well it can approximate hard functions within regions of interest (which is essentially always a small subset of the range of said hard functions).

The main problem is human overconfidence in the resulting functions, which are often measuring things humans didn't realize was embedded in the data (eg with tumor recognition), or are utterly hopeless at representing the infinite edge cases that the real world can present (eg self-driving cars).

It's one thing to have hoped that these methods could solve these problems when the improvements were coming rapidly, but there will always be a limit to how well these systems can perform. And the problem is that they fail in entirely non-intuitive ways, making human oversight to correct for errors very difficult or impossible as well.

I'm more interested in replacing _other_ drivers, more than myself. Really if we could replace the bottom 10% of drivers with AI, even at the level we have today, I imagine that would be a net improvement. But that isn't really a feasible program. As for future, improved AI, I would trade my own driving for the more efficient and safer system.

The severity of the accident is an interesting point. Though it intuitively feels like there are ways for software to mitigate the severity of an accident when it realizes it is about to crash than a human who might be asleep, intoxicated or otherwise have a slower reaction time.

The entire way we approach driver liability in general is insane.

The Attorney General of South Dakota was looking at his phone, swerved into the shoulder, killed man and then left the scene. He claimed he thought he hit a deer, even though the victim's head went through the windshield and the victims glasses were later found inside the car.

What consequences did the Attorney General face? Was his licence revoked or suspended? Did he serve any jail time? Did he resign? The answer to all of these is "No." The only result was two misdemeanors and a 500 dollar fine.

So yes, accepting occaisonal inhuman errors from a system that is 10x safer (hypothetical, no current systems have this record) than human drivers may also be insane but it would still be far more sane than the current approach to human drivers.

There is no code, that is the problem.

You (or the Transport Administration) do not have access to the training data, the training parameters or anything at all.

It is just a black-box which we are all to trust because "it works, mostly".

In my book, that goes against any notion of admissibility by a government agency.

Yes anything or anyone outside of the norm becomes a target.

E.g. wear your hair in a funny way that's never been trained on? You're a target!

Slight quibble: I very much agree with your overall point but human-driven cars slam straight into rescue vehicles all the time. They're often intoxicated but not always. There's just something about flashing red lights that attracts driver attention and all too often the car follows.

This is why we put the BRT (big red truck) behind the rescue scene and park it at a 45 degree angle. It weighs 15x as much as a car so it won't move much when a car hits it, and we want the car to bounce off sideways and away from the rescuers (us) when it happens.

Assuming that in aggregate they do kill fewer humans, then this is a rare instance of an actual trolley problem. Do you want to kill fewer people by changing the norms and reasons?

why doesn't it work? isn't all our society always abstracting away the individual stories behind statistics? (crime is down by 10% ! GDP is up by 0.2%, birthrate is down by xx).

we're used to it and we're happy about it and we use it to make decisions (buy house now, move to a new city ...).

Plus in those rare cases that you mentioned at least the same system will be feedback the incident to make sure that this won't happen again in the future... so in a morbid way there is a way to learn from such fatal incidents ... you can't really say that about such incident if the driver was a human.

It’s not a fallacy, it’s a risk analysis that will save lives.

" It seems like authorities are struggling"

I would rather describe it as sleeping at the wheel, passed out drunk and having pissed their pants after vomiting a bottle of hard liquir.

Okay, maybe i got carried away with the metaphor, but you get the idea

Are you implying that autonomous navigation isn't a stepping stone to powerful weapons?

Solving aging would be a massive win for humanity at the individual level, and a horrific catastrophe at the species level.

We make those choices all the time.

Taking a plane instead of driving is similar. You lose direct control, but it's orders of magnitude safer.

I'll take a 0.001% chance of death over a 0.1% chance every time.

Anytime you get on the road you are exposing yourself to the possibility that a suicidal or drunk or elderly person with dementia will kill you.

Personally, I would like to have very rigorous driving tests for old people and remove licenses for people with DUIs. The obvious question then is, how are old people and drunks supposed to get around. Self driving cars seem like an option eventually. It makes you safer compared to the alternative, which is we generally wait until someone is killed before we permanently remove someone's license.

I wonder how many people say this whose personal driving habits make them much higher risk than the baseline.

> You can’t hire enough safety drivers to scale this process and releasing it is the fastest way to get there.

I am sorry but I sincerely hope you are never made responsible for the release of anything remotely safety-critical.

Please re-read what you wrote. You are saying that because a business does not have enough money/resources to scale a process in real-world conditions that the solution is to release something and verifying in the real-world, on a public road, risking real human lifes?

If you don't have enough money to test it then you don't have an actual product/business.

Waymo appears to disagree with you, they ran tests with safety drivers and in controlled conditions for a very long time, and they eventually got to the point where there are customers riding in unmanned taxis. Tesla is just applying a "move fast and break things" approach to pedestrians' lives in order to satisfy shareholder interests and executive egos. They're not the only ones, as we saw from Uber's absolutely shameful performance that eventually lead to a death (safety driver involved but not solely responsible, of course)

If you save a child from drowning you don't get a 'a kill coupon' you can retrieve next time you are accused of murder.

> Munro is very critical of many aspects of car design; I cannot understand why he's such a fan of FSD.

Maybe due to personal profit?

https://old.reddit.com/r/RealTesla/comments/kxj0or/twitter_s...

>I tried FSD on the $200/month plan and dropped it: It makes the car unsafe. To command a lane change you hold down the turn signal stalk. If you fail to hold it down long enough the car suddenly swerves back to the lane it was in. This is (to say the least) disconcerting at 80 mph.

This isn't true. You just tap the stalk down until it clicks. Sometimes if it detects issues in the other lane it will not complete the lane change and go back into your existing lane. But you do not have to hold it down the whole time.

> Munro is very critical of many aspects of car design; I cannot understand why he's such a fan of FSD.

Because he doesn’t understand it and thinks it’s magic. He has not much clue about modern tech and is just buying into the hype.

> I tried FSD on the $200/month plan and dropped it: It makes the car unsafe. To command a lane change you hold down the turn signal stalk. If you fail to hold it down long enough the car suddenly swerves back to the lane it was in. This is (to say the least) disconcerting at 80 mph.

That's not quite accurate. Unrelated to autopilot / FSD, you can do a small press on the turn signal and it will signal three times and then stop signaling. You can also push all the way down and it will signal until you turn it off. You don't have to hold it down.

FSD will only continue switching lanes while the turn signal is on, so if you do a small press down, you may see the behavior you described.

Had the opposite experience on a testdrive purposely (within reason) trying to see if it would merge into a lane that subsequently had a merging onramp with a car resulting in the blindspot likely needing to move in.

Was super impressed it seemed to over correct and be thoughtful about consequences a few seconds down the road.

Anecdote though.

You think Uber doesn't provide a good service to the world? You are in favor of Taxi/Medallion monopolies?

Maybe you were thinking the autopilot accident rate being 5-10x _less_ than human drivers?

> In the 2nd quarter, we recorded one crash for every 4.41 million miles driven in which drivers were using Autopilot technology (Autosteer and active safety features). For drivers who were not using Autopilot technology (no Autosteer and active safety features), we recorded one crash for every 1.2 million miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 484,000 miles.

Tesla publishes these safety reports; the accident rate has held fairly steady quarter over quarter: https://www.tesla.com/VehicleSafetyReport

If there is clear evidence that these cars would have lower fatality rates than human drivers (I don't think this evidence currently exists), then holding off release to continue development is potentially a moral bad, for the same reason that we end clinical trials early when they demonstrate clear life-saving potential that would save lives in the control group.

I don't think it is morally reprehensible to ask what other people's intuition around this problem is.

One immediately obvious issue is comparing Tesla vehicles, which are all relatively new vehicles typically owned by relatively older drivers to all other passenger vehicle.

The other immediately obvious issue is that the self driving features of a Tesla probably work best in ideal conditions, where human drivers also do.