Ironically, one of the ways that humans are worse than AI, is that any given human learns from an even smaller fraction of that collective experience than AI already does.
That's because it's not trying to do so. The observation is that humans are broadly unable to prepare for the failure modes of other humans, even when those failure modes have been studied and the results of those studies widely published. This means that while the failure modes of humans are indeed different from the failure modes of LLMs (and AI more broadly), these differences are not what I anticipate to be the most important next step in AI research.
AI are better than most humans at dealing with human suckage, for example because unlike humans the LLMs have read all that literature about human suckage, but that's not relevant to what I was saying.
My point is: other failure of AI are more pressing. IMO the inefficiency with regard to examples, e.g. even cancelled/sold off self-driving car projects (Uber's ATG) have more miles of experience than a human professional driver can get in their entire career, and look how bad that model was.
Making a self driving car fail like a human means getting it distracted by something on the phone. Plus a bunch of other failure modes we should ignore like "drunk" and "tired".
Even if you don't fully solve the example inefficiency, merely improving it will make a big difference to performance.
>for example because unlike humans the LLMs have read all that literature about human suckage
No they haven't. If you read the cliff notes of a book, you haven't read that book. An LLM is a generalization over their entire training set, that's not what the word "reading" has ever meant.
The LLM does not "know" anything about human suckage or how to get around it, and will not use those "learnings" in it's "thinking", it will only come up if the right nodes in it's model trigger, and then it just generates tokens that match the "shape" of writing that was written with that knowledge.
A bloom filter can be used to test for presence of something in your DB, with configurable probability even (something that LLMs massively lack), but a bloom filter does not Know what is in your DB
When you fit a linear regression to a plot of free falling speed over time, you will have an equation for acceleration of gravity, but you don't "Know" gravity, and that equation will not allow you to recover actual generalizable models of gravity. That limited model will still get you most of the way to the moon though.
Generally the next claim is "same as human brains" but no, that has not been proven and is not a given. "Neural Networks" are named that way as marketing. They've never been an accurate simulation of actual animal neurons and a single animal neuron has far more robust capabilities than even many "Neurons" interconnected. Consider how every animal neuron in an animal brain intrinsically swims in a bath of hormone gradients that can provide positional 3d information, and how the structure of those real neurons is at least partially structured based on a thousand generations of evolution, and involves highly conserved sub-structures. Brains do not learn like neural nets do.
> AI are better than most humans at dealing with human suckage
That is a valid opinion, but subjective. If I say that they're not better, we're going to be exchanging anecdotes and getting nowhere.
Hence, the need for a less subjective way of evaluating AI's abilities.
> Making a self driving car fail like a human... "drunk" and "tired"
You don't understand.
It's not about making them present the same failure rate or personality defects as a human. Of course we want self-driving cars to make less errors and be better than us.
However, when they fail, we want them to fail like a good sane human would instead of hallucinating jibberish that could catch other humans off guard.
Simplifying, It's better to have something that works 95% of the time, and hallucinates in predictable ways 5% of the time than having something that works 99% of the time but hallucinates catastrophically in that 1%.
Stick to the more objective side of the discussion, not this anecdotal subjective talk that leads nowhere.
