Ha! That’s the hype there. They made predictions, like any other model out there. Their models are better but not close to what we get out of xray crystallography which is a painstaking process.
Protein folding is nowhere near a solved problem. A third of proteins don’t have high enough accuracy.
I can also make a prediction for every single known protein, it's trivial to do. My own predictions would be uniformly wrong. The question is how accurate AlphaFold's predictions are - of course, this question was almost totally avoided by the news reports and DeepMind press releases. It is accurate enough (and accurate often enough) to be a useful tool but by no means accurate enough to say they "literally solved protein folding."
"There is no reason for any individual to have a computer in his home."
- Ken Olsen
I was there. I remember people saying it, but this f*cking site wants to downvote me into negative for remembering it, probably because they are thinking of business computers. Everyone was on board with that. I'm talking specifically of home computers.
I wouldn't worry, I think what you're saying is pretty well known. We've all heard it before from every tech evangelist out there. Everyone knows that sometimes tech naysayers turn out to be wrong; the point is that it's pretty useless information.
The AI naysayers will be similarly remembered was the point of my "useless information" post. I was surprised to be downvoted, probably because those doing it have an axe to grind and have poor reading comprehension skills. Surprising for Hacker News, but I've noticed every site is getting dumber at an alarming rate so why should Hacker News be the exception? If you're going to downvote me for disagreeing with me at least say why. It's in the HN guidelines
I can vouch for this. There was a lot of rhetoric about how a normal family wouldn't really need a computer in their home. This point was usually raised to support an argument that the personal computer market would be quite small.
No doubt! I err on the side of "most things are over-hyped" (see: room temp superconductor hype that lasted a week), but I try to remind myself there's a yin and yang thing going on.
Yin: "this tech will change the future"
Yang: "this is over-hyped and the promises are hot air"
It's easy to default to "yang" because it is true most of the time (especially with tech). But you gotta acknowledge that "yin" is actually right some times too.
I often like to use an analogy involving a local volcano.
The odds are incredibly strong will not explode today, but the granularity/time-periods matter, and there's a fundamental asymmetry in how we value the false-positive rate versus the false-negative rate. :P
___
"Look, I've made daily predictions for 30 years now, and they're all perfectly 100% accurate, go on your hike, it'll be fine."
<Volcano suddenly erupts in background>
"Did I say 100%? OK, fine, after today it's 99.99%, which is still awesome."
Right now, as stated, the "Yang" side as applied to AI is clearly true. Even if the tech will "change the future" it will be no less correct for us to say that current AI products are overhyped/vaporware and that AI salesmen and researchers are passing off sci-fi stories and business strategies as wise prognostication. Even if what they're saying turns out to be true, it's completely correct to say that they're just (sometimes unbeknownst to themselves) wildly guessing.
I don't really intend to bicker, but I'm a little curious about the thinking here..
Maybe it's getting too philosophical, but if you're correct because of "wildly guessing".. you're still correct. Maybe you've only been correct 2% of the time with your predictions, but that doesn't change being right or wrong in any given instance.
If someone says "it will do A" and you say "no it won't, you're passing off sci-fi as prognostication", and then it does end up doing "A", you were wrong? No? If someone's AI tech does end up "changing the future" then how would you not be incorrect if you had previously said it was vaporware?
> If someone's AI tech does end up "changing the future" then how would you not be incorrect if you had previously said it was vaporware?
"This product is vaporware" doesn't mean "This product is impossible and can never come to fruition." Vaporware is "software or hardware that has been advertised but is not yet available to buy, either because it is only a concept or because it is still being written or designed."
It doesn't matter even slightly if Altman and Huffington's app will materialize and change the universe; it's still vaporware. It's just what the word means.
Don't forget the time crystal: It was overhyped in the past, as well. But there are endless details to how things could turn out, few of them expected in advance.
AlphaGeometry is a hyper-specific system trained to add auxiliary geometric objects, like extra lines, to existing Euclidean geometry configurations. These prompts are not even sensible inputs to AlphaGeometry.
> A.I. has helped chatbots carry on conversations almost indistinguishable from human interaction. It has solved problems that have confounded scientists for decades like predicting protein shapes. And it has blurred the lines of creativity: writing music, producing art and generating videos.
Why was this article written now?? This is the only paragraph that substantially marks it as post-2016. For such a key paragraph, I think it's rather too sloppy: eg just one more example of reporters/DeepMind transforming AlphaFold's useful two-thirds accuracy rate into the culminating solution of an age-old problem.
> “The industry is wrestling with this. Technically the companies have the copyrights, but we have to think through how to play it,” said an executive at a large music company. “We don’t want to be seen as a Luddite.”
I'm a mathematician but tbh I have no clue what you mean by saying that arithmetic progressions of primes are "trivial" or analogous to anything here or in machine learning.
Yeah, the messaging got a little muddled, but the relation was purely analogical.
I was trying to point to a situation where you have a clear problem: a generating function for the prime number sequence; and a solution that identifies a small subset of the intended sequence without addressing, or even informing in any substantial way, the full breadth of the original problem.
> At the time of writing the longest known arithmetic progression of primes is of length 23, and was found in 2004 by Markus Frind, Paul Underwood, and Paul Jobling: 56211383760397 + 44546738095860 · k; k = 0, 1, . . ., 22.'.
The triviality was overloaded to both imply that calculating this subset is trivial, it is a simple arithmetic progression, and that subset of the full prime number sequence is now trivial to produce.
In the same way that the Green-Tao theorem has yet to lead to a complete solution to the prime number sequence, I feel, the machine learning techniques will fail to lead to a complete solution to protein folding.
It would be very hard to make a good analogy with this since the problem of "finding" arithmetic progressions is, as far as I know, of negligible interest compared to the structural knowledge of their existence. The situation is perfectly reversed in both computational biology and machine learning. But maybe I misunderstand what you mean by "a complete solution to the prime number sequence."
Great article, covers well both the achievements and the shortcomings. It's crazy how many people write about these kinds of AI developments while completely skipping over anything like the following:
> The “good news is that when AlphaFold thinks that it’s right, it often is very right,” Adams said. “When it thinks it’s not right, it generally isn’t.” However, in about 10% of the instances in which AlphaFold2 was “very confident” about its prediction (a score of at least 90 out of 100 on the confidence scale), it shouldn’t have been, he reported: The predictions didn’t match what was seen experimentally.
> That the AI system seems to have some self-skepticism may inspire an overreliance on its conclusions. Most biologists see AlphaFold2 for what it is: a prediction tool. But others are taking it too far. Some cell biologists and biochemists who used to work with structural biologists have replaced them with AlphaFold2 — and take its predictions as truth. Sometimes scientists publish papers featuring protein structures that, to any structural biologist, are obviously incorrect, Perrakis said. “And they say: ‘Well, that’s the AlphaFold structure.’” ...
> Jones has heard of scientists struggling to get funding to determine structures computationally. “The general perception is that DeepMind did it, you know, and why are you still doing it?” Jones said. But that work is still necessary, he argues, because AlphaFold2 is fallible.
> “There are very large gaps,” Jones said. “There are things that it can’t do quite clearly.”
> However, in about 10% of the instances in which AlphaFold2 was “very confident” about its prediction (a score of at least 90 out of 100 on the confidence scale)
I wonder what that confidence score means... If it is 90% probability, then we'd expect it to be wrong 10% of the time
It really depends on the context; (Some) LLMs look impressive specifically because the error rate is comparable to a high score on an exam… the mistake is that even if it was a straight-A student (it's too alien to be that) then it would still only be a student, and we don't put fresh graduates in charge of everything important.
I don't have the domain knowledge to even guess how good 90% is in molecular biology research.
> we don't put fresh graduates in charge of everything important.
We put dribbling halfwit morons in charge of everything. I'm thinking of Liz Truss as UK Prime Minister, but I'm sure most countries have their own examples.
This tool is designed to be helpful right now. Looking ahead, there's no reason why AI can't eventually match, or even surpass, human intelligence across the board.
Whether it's advancements in LLMs, with features like long-term memory, or breakthroughs in other areas of ml, it's not guaranteed that humans will remain needed in the research process.
> Looking ahead, there's no reason why AI can't eventually match, or even surpass, human
> intelligence across the board.
There is a reason, actually: what is presently called an "AI" has no concern for the truth. It is a bullshit machine that aims to mimic the right answer.
> The author of the post is Terence Tao that is the best live mathematician. If he says it's a "breakthrough", it's a breakthrough.
I think it's pretty silly to say that Tao is the best living mathematician, but even if he were I don't think that this would be a useful way to think about things.
> I think it's pretty silly to say that Tao is the best living mathematician
I agree. I was going to write "one of the best" or "probably the best" or something like that. It's like discussing if Messi or Maradona were the best football players. [1]. Anyway, all three of them are quite good.
> but even if he were I don't think that this would be a useful way to think about things.
I also agree. It's just and argument by authority. For a decition that would change the lives of millons of persons and has a lot of subtle tradeoff and unknown unknowns, I'd ask for a better justification and evidence. But deciding if this is a breakthrough or not, may only change the lives of a few graduate students and a few grown up mathematicians, so I'm happy to take the word of Tao.
No matter what you think, best living mathematician is what other mathematicians say about him.
But I'll humor you. How would you prefer we say things?
- Highest IQ on record.
- One of only 3 people to score over 700 on the Math SAT before he was 8. He had the highest score of the three with 760.
- At ages 10, 11, and 12 he set the record for winning Bronze, Silver, and Gold respectively in the International Math Olympiad. After that he lost interest.
- PHD from Princeton at 21.
- Full professor at UCLA at 24. This is a record.
- He is a respected leader in at least a half-dozen areas of mathematics. He regularly publishes in many more. It is unusual for a mathematician to have significant publications in 2 areas.
- Wikipedia lists 28 major prizes for him. Excepting Australian of the Year in 2022, all are major mathematics prizes. No other mathematician comes close.
- Once you exclude junk journals, Tao publishes papers faster than any other mathematician. And his are all good.
- Tao's papers show up in the popular press more often than the next 3 mathematicians combined.
And so on.
At what point is there a better way to think about this than, "best live mathematician"?
(And yeah, until I began noticing Tao, I would have also thought that a silly way to think...)
The idea that Tao has accomplished more than, say, Serre because the latter, who won the Fields medal at 27, only received his PhD at 25 and his bachelor's at 22 while the former received his PhD at 21 and his bachelor's at 16 is so absurd that it can be refuted merely by alluding to it.
Serre is indeed a top mathematician. (I'm actually surprised to find out that he's still alive!)
At this point Tao only has 3/4 his number of publications, similar numbers of textbooks, a similar number of awards (using https://mathshistory.st-andrews.ac.uk/Biographies/Serre/ to count awards), and so on. I'd count Tao as having more of what I see as major breakthroughs, but that is subjective. But then again, Tao is half of Serre's age.
Yeah. I still think it is fair to put Tao in the same tier as Euler, Gauss and Hilbert.
Sorry, I think this style of hagiography is completely goofy, there's nothing else to say about it. And I'm sure it's not even true that he has the "highest IQ on record."
To make a claim like "greatest living mathematician" it would be more appropriate to talk about his actual research accomplishments and how they compare to canonical figures like Gromov, Milnor, Serre, Smale, Yau, among many younger counterparts.
But, personally, I think defending any such claim for any particular person is kind of a juvenile exercise. I am a mathematician and among the slight minority of mathematicians I know who would take the question seriously, a minority of them would select Tao. Which of course isn't to say that he isn't universally considered a top mathematician.
AlphaFold deserves some hype but this is a tremendous overstatement.