I completely agree that we put a lot of value into personal agency but I do think it's a bit unreasonable. Agency when driving is only perceived. You could have a heart attack and smash into pedestrians. Your car could fail mechanically. You could be rear ended by a large truck. You could be killed by a drunk or inattentive driver. Your agency can not entirely prevent that. Also you may be in the top percentile in driving but like everything, everyone thinks they are a great driver so you can't take how you feel and generalize it to the population.

I'm not arguing that self driving cars are solved but maybe they aren't quite as bad as people feel. We need proper statistics.