It is completely insane that this is happening. I did DD on a company in the automotive space a couple of years ago and flagged that they did not check if the vehicle was stationary, motor disabled before updating. They were all surprised at how I thought that this could possibly ever lead to issues.
I have Java code running on commercial aircraft. You can’t actually run Java code on commercial aircraft because the FAA doesn’t (or at least not at the time) know how to certify it.
The entire box it’s on isn’t powered while the plane is in motion (“wheels on ground”). It’s shut off before preflight and doesn’t turn back on until the plane is on the ground. The service my code is part of is responsible for queuing updates and downlinking telemetry. Updates are manual and obviously you can’t run them while in motion if the box they are on doesn’t even have power.
Cars probably don’t have to go this far, but there’s a continuum and they’re clearly in the wrong part.
Even iPhones and windows let you schedule update times. Just the fact that a freaking MOVING MACHINE doesn’t is egregious on itself. Imagine if stellantis would manufacture industrial equipment or nuclear reactors!
I wonder how many OTA updates for cars could be left as a task for the mechanic, the way airplane updates are.
Airplanes are required by regulation to have a backup of all software to operate the plane, presumably so that a plane can’t get stranded by an emergency landing requiring system resets. What we built replaced a physical folder full of floppies or CDROMs taking up space in the cockpit. Some of my coworkers insisted it was for weight but I’m absolutely sure that pizza box server weighed more than the book.
Given the state of the software industry, it's honestly more surprising that this doesn't happen more often. Our industry is a complete joke, and somehow we've been given responsibility over people's lives.
We are really only about 60 years old as a proper profession, and we seem to be trailing behind doctors for professionalism and standard of care by about 100 years.
I don’t know what will turn out to be our penicillin, or our Joseph Lister, but in 1960 the former is something that didn’t exist when older doctors were in school, and latter had only been dead for fifty years. It may not have happened for us yet.
This is a super important point that I don't think a lot of people fully recognize. Medicine is a super interesting comparison because you honestly don't need to go all that far back to find some pretty egregious examples of doctors making things much worse due to ignorance or incompetence. My favorite example of this is the assassination of President Garfield, who most likely died not to the bullet wound itself but from the doctors rummaging around to try to find the bullet with unsanitized hands, causing infection and damaging organs...on the wrong side of his body[1].
On the topic of professions: Joseph Lister was a surgeon. Modern surgery (which I define as surgery aided by anesthesia) is a relatively recent discipline dating to the early 19th century. The introduction of anesthesia made lengthy and intricate operations possible but also ushered in novel problems and complications. Surgery as a field had to learn tough lessons over time.
He was known more for antiseptics but the biggest surgery moment for me will always be “using soap” and I wonder what the software equivalent is.
Like I said we are still young, so it feels sort of arrogant saying we have figured something out when I know how many things are industry standard now that almost resulted in shouting matches trying to get done even 20 years ago. Maybe our soap moment is coming up ten years from now.
But I suspect automated testing may be the wash your hands, because it represents a sort of hygiene that “we” used to just say fuck it or make a minimal effort.
> Given the state of the software industry, it's honestly more surprising that this doesn't happen more often.
It probably does. We just don't notice.
> Our industry is a complete joke, and somehow we've been given responsibility over people's lives.
Amen to that. kqr made some choice comments the other day in that thread about the airliner that came to within a hair of crashing due to running out of fuel. Thinking about risk is not a skill that we're born with and it is always sobering to read the 'risks digest' for a bit and to see how thin the ice really is.
It does. I have a Ford CMax from 2014. For years, when the SiriusXM radio software update would happen it would get stuck downloading. The geniuses at Ford decided the update should continue trying to complete even if the car was turned off. So once the download got stuck, it would completely drain my battery every single time. I'd rather have a car that moves that the latest SiriusXM update in my radio. The only fix was to pull the fuse if you noticed that it was happening.
I'm willing to see a difference in software standards between (say) Waymo and Jeep. One is a software company, the other is a sheet-metal company. If you just tar the whole industry you lose an ability to learn from those doing it better.
Tesla is very controversial, and they have clearly made some serious software mistakes, but they are so much better at software than any other maker I've encountered, except maybe mazda who eschew touch screens for physical buttons, but that is a ui success, not a software culture success. Tesla wrapped an electric car around a software company. They treated fit and finish and panel mating etc. as the throwaway/buy it cheap aspect (ok that is pretty harsh. It isn't that bad) and focused on the software. Where legacy makers did the opposite.
you are being generous. Tesla's software "mistakes" have killed several people. They needlessly try to reinvent the wheel in the name of innovation and end up ignoring decades of auto industry knowledge.
I do not trust them and never will. This is the #1 reason why every car is buy is just a car. I do not trust techbros with devices that can kill you, especially cars.
The software mistakes that killed people were software doing things no other automaker even tries to do. Very possibly with good reason. The software that does bog standard normal things like coolant control and battery preconditioning works well and seems to be tested and deployed in a reliable way. That is still so much better than what we get out of others. I would love an electric car with no can bus or microcontrollers, so I'm right with you. If anything the point to be made is that Tesla, who has killed people with its software, is still way better than average... So yeah, we are bad at what we do.
> doing things no other automaker even tries to do
"Move fast and break things" is not really a virtue when the thing moving fast is a two-ton hunk of steel and the things breaking are people's bodies. Getting the easier stuff right but then then also killing people isn't "doing better" in my opinion; sometimes it's better to have a lot of lower magnitude failures than infrequent but catastrophic ones.
I presume you're referring to Autopilot/FSD. I don't trust it at all, don't use it on my Tesla, and will not get into a "robotaxi" using it, but it's an optional feature.
Autopilot aside, though, the regular boring car software bits are rock-solid, and I've never had an issue with using it or after an update.
I do recall a story a number of years ago where one of the automatic updates changed the UI and hid the defrost behind a menu (or something along those lines). I don't know that anyone died as a consequence, but it was criticised as being quite reckless as it is a feature that when you need it, you need it right away.
Probably because the regular boring car stuff is not even made by car companies anymore LMAO.
The steering racks are made by Bosch or maybe ZF. Brakes come from Brembo. ABS module and its software is Bosch aswell. same goes for brakebooster, EPS pumps, AC compressors, and airbag controllers. I think the only electronics Tesla develops and manufactures are EV power electronics, infotainment, ADAS&Co and the drive motors.
If you take a VW Golf, you'll find the ECU and all of the software running the car is built by Bosch too. Essentially they sell VW a kit which needs to be mounted on a vehicle platform. Tesla is likely one of the only companies for better or worse that don't follow this model.
> you are being generous. Tesla's software "mistakes" have killed several people.
Citation needed.
In the early days of autopilot/FSD most of the fatalities were people doing stupid things like watching a movie or sleeping in the back seat. That's why it now has to monitor your face with a camera to detect whether or not you are watching the road - to stop people from being idiots.
However we must acknowledge that any change in the automotive space is going to lead to problems and some percentage of those are going to cause injuries. That is the nature of cars. They do not have the certification standards of aircraft nor the training of pilots. They can't and they won't.
It is also inevitable that autonomous driving is going to make different mistakes than a person would make. On a miles-driven basis it still produces fewer accidents and injuries than human drivers.
I’m going on a limb here because i’m not directly on the software industry but my first suspect would be metrics and the fact that you have to deliver a product at certain time “no matter what”.
According to the article, that's not what is happening. The update itself completes fine; it's the updated firmware that is buggy, and seems to cause/require a reset of the ECU while in operation. Not that that makes it any less insane, but the update process does not seem to be implicated here.
they did not check if the vehicle was stationary, motor disabled before updating. They were all surprised at how I thought that this could possibly ever lead to issues.
My anecdata is that my car won't update its software without the owner explicitly requesting it. And then, it will only do it if the car has something like 50% charge, hasn't been used for an hour, and nobody is inside.
I once tried to do the update while I was inside, and it refused.
My BYD wants the battery over some percentage, the vehicle in park, and the hood closed. The hood one was surprising, I wonder if it's for the safety of the car or of anyone working on it.
Probably a safeguard to keep sonebody from unplugging something during the update.
I can't speak about other cars, but my EV has nothing you can unplug. It's not like a regular car where stuff is exposed.
All it has under the hood is a storage space for charging adapters, a first aid kit, and a cap for the windshield washer fluid.
Even accessing the regular 12V battery takes a bunch of time and tools. The manual states several times that it should never ever be used to jump start another car, though it doesn't explain why.
If a power failure during the upgrade causes some unrecoverable problem that is a serious design failure. The answer isn't "make power failures less likely" instead it's "make the update process robust to power failure". This kind of disconnected hubris--thinking you can just wish reality away--seems unique to software. Why are they allowed to get away with it?
Sure, but if they aren't checking the super obvious potentially dangerous cases, doesn't that say something about the likelihood of their process detecting something less direct like this?
