I think this is a useful and impressive study - I haven't read all 40 pages, neither have you :) I did do some good-faith skimming. Assuming Waymo didn't falsify their data (they didn't), this makes me feel comfortable having Waymo in SF and Phoenix. I think it's clearly safer than an Uber. But some caveats:
- The major caveat is that Waymo is not being directly compared against sober humans driving lawfully.[1] The reason why this caveat is so important is that technology which makes it impossible for humans to exceed a posted speed limit might be overall much safer than replacing human drivers with autonomous drivers. Uber isn't more dangerous than Waymo because humans are incompetent, it's because humans obey orders from impatient drivers and Waymo currently does not. This is a UI choice, not an AI advancement.
- More specifically, lawful driving is an important caveat because Tesla Autopilot had two different settings for driving unlawfully, according to the users' own sense of personal risk. An AV manufacturer who advertises "AI-assisted speeding" will almost certainly find a lot of customers, even if it's under the table. People don't speed and run red lights because they're too stupid to understand why it's dangerous: they do it because they're reckless and selfish. AI won't stop that, only regulation will.
- Another caveat is that Waymo was trained on human-dominated streets. Waymo being safer in a sea of human vehicles does not actually translate to Waymo being safer in a sea of Waymos. I think this is a low-probability risk but it's hardly a simple question: I believe Waymo has had issues where several AVs occupied the same street after an event and blocked traffic because they couldn't decide what to do - they were waiting on each other to behave like a human. But again, the risk seems like gridlock, not property damage or injury.
- And a minor but still important caveat is that SF and Phoenix have modern linear grids which have been mapped to death by AV manufacturers. As a Boston resident I am still holding my breath about their performance here :)
[1] Not because of anything insidious, it's just a granularity that both the analysis and the data struggle to capture.
Once automated cars make up a significant fraction of cars on the road I would expect some communication scheme to emerge to make them coordinate with each other, maybe even sharing observed positions of other road participants.
There is the risk that they will misbehave or deadlock around each other, but also a great opportunity for them to communicate intent to each other at a level human drivers just can't do from a sound-insulated cabin.
My point is that it will take a lot of work to train them to communicate with each other. They are not smart enough to have a scheme simply pasted in, nor are human programmers adept enough to hard-wire a carefully-trained AI to have this scheme implemented. Absent extensive retraining you'd probably just have Waymos getting confused about all sorts of edge cases in the communication scheme itself.
I am not saying that it's doomed to fail - simulations can probably do most of the heavy lifting, and of course the tech itself will advance. I am saying it's a mistake to assume stuff like this is somehow a free lunch, or even "the easy part." It's unknowns about unknowns.
I would be surprised is they don't already share at the detailed map level. Was any published on how much the cars contribute to improving the map day after day (including temporary road closures and road work)? - but probably not yet directly when at the same intersection.
I think the hardest thing about a city like Boston is going to be the areas where the rules of the road go out the window in favor of the social contract. Drop off and pick up at Logan or a sox game can be pretty wild. I don’t think navigating Beacon st or Comm ave will be that big of a deal. I’ve spent about 8 hours in a Waymo and I grew up driving in Massachusetts.
The hardest thing about driving in Boston is that the streets are wildly non-rectilinear. Waymos obviously don't understand "the rules of the road," they are exhaustively trained on the rules of the road until they can imitate them. For rectilinear intersections they can transfer this training to a variety of different streets; not so in Boston, where you might have a weird three-way intersection involving oblique angles. I am also not sure if SF / Phoenix have many roundabouts. Or, near me, this horrific pair of four-way-intersections somehow combining into a five-way(?) intersection: https://www.google.com/maps/place/42%C2%B022'50.0%22N+71%C2%... I suspect Waymo would get badly confused here unless it was directly trained on this specific intersection.
There is plenty of stuff in SF that's brain-freeze level for human drivers. From rectilinear grid "improved" with a million specialized lane markings to plenty of vehicles blocking traffic short of ingenuity.
There are some odd intersections in Phoenix. And don’t even ask about the “suicide lanes” which change from turning lanes to unidirectional during rush hour with the direction depending on time of day. AFAIU Waymo drives everything in their coverage area ahead of rollout so they’re definitely “training” on oddities.
> I believe Waymo has had issues where several AVs occupied the same street after an event and blocked traffic because they couldn't decide what to do - they were waiting on each other to behave like a human.
> And a minor but still important caveat is that SF and Phoenix have modern linear grids
San Francisco has some amusing areas as well. If you look around the streets near Buena Vista park or even Twin Peaks, you'll see plenty of funky curves, six way "intersections" and so on. Lots of two-ways that are only lane wide.
Like the older parts of Boston and Cambridge, it's a holdover from the days of horses and the geography. That's where the paths were, people built houses, and now you've got to deal with it :).
- The major caveat is that Waymo is not being directly compared against sober humans driving lawfully.[1] The reason why this caveat is so important is that technology which makes it impossible for humans to exceed a posted speed limit might be overall much safer than replacing human drivers with autonomous drivers. Uber isn't more dangerous than Waymo because humans are incompetent, it's because humans obey orders from impatient drivers and Waymo currently does not. This is a UI choice, not an AI advancement.
- More specifically, lawful driving is an important caveat because Tesla Autopilot had two different settings for driving unlawfully, according to the users' own sense of personal risk. An AV manufacturer who advertises "AI-assisted speeding" will almost certainly find a lot of customers, even if it's under the table. People don't speed and run red lights because they're too stupid to understand why it's dangerous: they do it because they're reckless and selfish. AI won't stop that, only regulation will.
- Another caveat is that Waymo was trained on human-dominated streets. Waymo being safer in a sea of human vehicles does not actually translate to Waymo being safer in a sea of Waymos. I think this is a low-probability risk but it's hardly a simple question: I believe Waymo has had issues where several AVs occupied the same street after an event and blocked traffic because they couldn't decide what to do - they were waiting on each other to behave like a human. But again, the risk seems like gridlock, not property damage or injury.
- And a minor but still important caveat is that SF and Phoenix have modern linear grids which have been mapped to death by AV manufacturers. As a Boston resident I am still holding my breath about their performance here :)
[1] Not because of anything insidious, it's just a granularity that both the analysis and the data struggle to capture.