I'm pretty sure Waymo does this. It's hard to imagine not using this information, it's basically 50% of what I do when driving (e.g. that kid that was running on the sidewalk suddenly disappeared from view in an unexpected way; I wonder if he's about to cut across the street, etc).

Nice in theory but in practice they still need a ton of training data. The bitter lesson is very bitter. Tesla's end to end FSD uses occupancy networks and brake stabbing still hasn't been fully solved.

Ironically, I'm sure most humans have done the eyeball equivalent a handful of times, despite our knowledge of object permeance.

Maybe I misunderstand what you mean but the car itself (chassis, bodywork, drivetrain) is a commodity today. If you can turn any car into an autonomous car then the car isn't worth that much in the self-driving game. Tesla's stock is propped by the promise of becoming the first to achieve autonomy not by its ability to build cars, or else VW and Toyota would be worth multiples of Tesla. They aren't because cars are a commodity. Autonomous car brains are the hot deal.

If manufacturing ever became Google's Achilles heel, they can take a majority stake in something like VinFast and manufacture a Google car.

It's like saying Linux will lose out to Microsoft when it comes to manufacturing. Yes, they do lose out by not manufacturing anything and yet their product is slowly taking over the market. Local loss (no profit from manufacturing), global win (core product is increasing market share).

I still find it insane that the entire established automotive industry saw what Tesla was doing with cameras and incrementally working toward FSD, and still to this day has not even started to collect data (or even install enough instruments to collect the data!) to get onto the same trajectory. Meanwhile the Model Y is the best selling car in the world multiple years in a row.

Those operate in constrained areas of a few cities though - with a control room, that's a long way from telling your car to drive across state unattended.

I'm not sure how you would tell, but that is a critical question for scaling and true autonomy.

As of August Tesla had 80+ unsupervised taxis in operation with more than 380k miles driven total, and they are required to report incidents to the NHTSA. At the rate at which they are adding miles, if they are not now safer than human drivers, we should know soon.

Why would someone with no ego care about looking foolish?

Have you ever tried reading a book while being driven around? You get car sick really fast, because your vestibular system does not like constantly getting jostled around with mismatching visual inputs. Eating dinner while being driven around is a similar story: anyone around you does something unexpected, and your dinner ends up being spilled all over the car.

Self-driving cars will never have the same smoothness as planes or trains, simply because they have to share the roads with the general public. You might be able to sleep, but your commute isn't going to neatly match up with your sleeping schedule, so that's only really useful for longer road trips.

> this halfway autonomy where you have to watch everything is kind of useless

No, it is actively dangerous. Humans are extremely bad at staying fully-focused and being ready to act in the blink of an eye, while at the same time having literally nothing to do 99.99% of the time. If you don't let your attention wander, it is significantly less exhaustive to drive yourself. Halfway-autonomous cars exist primarily so that car manufacturers can demand extra money without having to accept any of the reliability.

What makes you think it doesn't? All those cameras, the expensive chips and cooling for them, and redundancy for all the systems don't come free.

The 5 millimeter-wave radars and 12 ultrasonic sensors are $100-150. LiDAR is $150-200. The 12 cameras, control unit, and 300 TOPs of computing is $150-200. This computing is comparable to what Tesla puts on their consumer cars, and the sensors suite is much more extensive. (The current robotaxis have about the same computing power as the consumer cars but a lot more RAM).

Current trend nowadays is watching tiktok, though thankfully I've only seen it at red lights, but that might just be because I'm not staring at other drivers when driving around at 40mph.

I do want safer vehicles and I do see a path to get there with this technology.

What I don't want is companies training their technologies with legitimate risk to the public, with a clear pathway to profiting off of this risk and public data, with no benefit to the public _unless_ they happen to succeed, and to some degree, unless you can afford their products. Something feels off about that.

Of course, everyone benefits if vehicles are safer, but it does feel like there is an extractive factor regardless. These companies benefit from public resources, are wholly enabled by public resources, but are trying to create a situation in which the technologies they create are more or less required to be used in order to utilize the public resource. Does that make sense? I suppose it's inferred, but, I don't think it's unreasonable to believe any car manufacturer with a working FSD technology would LOVE to have that technology mandated to be on every new vehicle. Imagine leasing that to every manufacturer who didn't develop it themselves so they could continue to sell in the USA.

On one hand, safer vehicles are a MUST because we allow an absurd number of people to die due to unsafe driving every day. On the other hand, I don't feel right about private companies benefiting so much from roads the public built, while training their technology with some degree of risk imposed on the public, and with no clear desire in these companies to give back. They tend to pay paltry sums in taxes, if any. Rivian was at a net loss of over 4 billion for 2024. I understand they're trying to build something that requires massive capital, but, I also understand that these companies are experts at operating at a loss for prolonged periods of time. Amazon is a stellar example.

This definitely cuts both ways, but I'm not sure exactly where I stand. I need more information to decide, but I'm confident that there is some degree of exploitation of public data and resources here. I'm not trying to imply Tesla, Waymo, or Rivian are the only ones guilty of this either. The tragedy of the commons is a well-known phrase because it's such a typical phenomenon. I'm also not suggesting that there's a simple solution. There can't be because the fragmentation of the polities in which these companies operate and train their technology within.

First, our current city driving environments were designed for being used by humans, so take into account human strengths and weaknesses.

Second, that kid has spent 16 years in such an environment learning how humans function there. He's got a very good model of how things work.

By the time I was 16 year old Timmy I'd spent a whole lot of time walking next to roads with cars, biking next to or in those roads with cars, and riding in cars on those roads. What was new when I was the driver was mostly learning the physics of cars and driving.

(I suppose nowadays, with the way parents can get in trouble letting kids out of their sight, we might have to worry about a generation of 16 year old Timmys who have not spent any significant time walking next to or biking next to or in roads with cars).

Third, that kid will have a pretty good model of predicting future behavior of other drivers, pedestrians, and cyclists based on figuring out their intent, which can even let them make some pretty good predictions about them even when they are hidden or masked from the kid's sensors.

Fourth, Timmy has a bunch of knowledge learned from listening to other people, reading, movies, etc., that can be applied while driving. For example suppose they live somewhere where there are many cows, but no deer. If they see a cow near the road at a place with no fence, they will have an appropriate model from experience that tells them cows are generally slow moving. Be cautious when passing, maybe giving it some extra space.

If the see a deer even though they have no experience with them they likely know that deer often bolt rapidly into the road right when a car gets close due to hearing stories from people who live in deer heavy regions or from news, books, or TV/movies, so go slow and be very cautious passing that.

Autonomous cars have a big advantage in reaction time, so if a bad situation starts to unfold they are likely better at saving the day than Timmy (or an experienced human driver) would be, but at least the systems so far deployed so far don't seem to be up to Timmy's understanding of other actors and their intent. They also seem unlikely currently to have that widespread non-driving knowledge that can be called upon to understand novel situation involving things that was not in their driver training.