The current generation of LLM's have convinced me that we already have the compute and the data needed for AGI, we just likely need a new architecture. But I really think such an architecture could be right around the corner. It appears to me like the building blocks are there for it, it would just take someone with the right luck and genius to make it happen.
> The current generation of LLM's have convinced me that we already have the compute and the data needed for AGI, we just likely need a new architecture.
I think this is one of the greatest fallacies surrounding LLMs. This one, and the other one - scaling compute!! The models are plenty fine, what they need is not better models, or more compute, they need better data, or better feedback to keep iterating until they reach the solution.
Take AlphaZero for example, it was a simple convolutional network, not great compared to LLMs, small relative recent models, and yet it beat the best of us at our own game. Why? Because it had unlimited environment access to play games against other variants of itself.
Same for the whole Alpha* family, AlphaStar, AlphaTensor, AlphaCode, AlphaGeometry and so on, trained with copious amounts of interactive feedback could reach top human level or surpass humans in specific domains.
What AI needs is feedback, environments, tools, real world interaction that exposes the limitations in the model and provides immediate help to overcome them. Not unlike human engineers and scientists - take their labs and experiments away and they can't discover shit.
It's also called the ideation-validation loop. AI can ideate, it needs validation from outside. That is why I insist the models are not the bottleneck.
For Alpha Zero, the "better data" was trivial. The environment of board games is extremely simplistic. It just can't be compared to language models.
The problem with language is that there is no know correct answer. Everything is vague, ambiguous and open ended. How would we even implement feedback for that?
> The current generation of LLM's have convinced me that we already have the compute and the data needed for AGI, we just likely need a new architecture
This is likely true but not for the reasons you think about. This was arguably true 10 years ago too. A human brain uses 100 watts per day approx and unlike most models out there, the brain is ALWAYS in training mode. It has about 2 petabytes of storage.
In terms of raw capabilities, we have been there for a very long time.
The real challenge is finding the point where we can build something that is AGI level with the stuff we have. Because right now, we might have the compute and data needed for AGI but we might lack the tools needed to build a system that efficient. It's like a little dog trying to enter a fenced house, the closest path topologically between the dog and the house might not be accessible for that dog at that point because its current capabilities (short legs, inability to jump high or push through the fence standing in between) so while it is further topologically, a longer path topologically might be the closest path to reach the house.
In case it's not obvious, AGI is the house, we are the little dog and the fence represent current challenges to build AGI.
The notion that the brain uses less energy than an incandescent lightbulb and can store less data than YouTube does not mean we have had the compute and data needed to make AGI "for a very long time".
The human brain is not a 20-watt computer ("100 watts per day" is not right) that learns from scratch on 2 petabytes of data. State manipulations performed in the brain can be more efficient than what we do in silicon. More importantly, its internal workings are the result of billions of years of evolution, and continue to change over the course of our lives. The learning a human does over its lifetime is assisted greatly by the reality of the physical body and the ability to interact with the real world to the extent that our body allows. Even then, we do not learn from scratch. We go through a curriculum that has been refined over millennia, building on knowledge and skills that were cultivated by our ancestors.
An upper bound of compute needed to develop AGI that we can take from the human brain is not 20 watts and 2 petabytes of data, it is 4 billion years of evolution in a big and complex environment at molecular-level fidelity. Finding a tighter upper bound is left as an exercise for the reader.
> it is 4 billion years of evolution in a big and complex environment at molecular-level fidelity. Finding a tighter upper bound is left as an exercise for the reader.
You have great points there and I agree. Only issue I take with your remark above. Surely, by your own definition, this is not true. Evolution by natural selection is not a deterministic process so 4 billion years is just one of many possible periods of time needed but not necessarily the longest or the shortest.
Also, re "The human brain is not a 20-watt computer ("100 watts per day" is not right)", I was merely saying that there exist an intelligence that consumes 20 watts per day. So it is possible to run an intelligence on that much energy per day. This and the compute bit do not refer to the training costs but to the running costs after all, it will be useless to hit AGI if we do not have enough energy or compute to run it for longer than half a millisecond or the means to increase the running time.
Obviously, the path to design and train AGI is going to take much more than that just like the human brain did but given that the path to the emergence of the human brain wasn't the most efficient given the inherent randomness in evolution natural selection there is no need to pretend that all the circumstances around the development of the human brain apply to us as our process isn't random at all nor is it parallel at a global scale.
> Evolution by natural selection is not a deterministic process so 4 billion years is just one of many possible periods of time needed but not necessarily the longest or the shortest.
That's why I say that is an upper bound - we know that it _has_ happened under those circumstances, so the minimum time needed is not more than that. If we reran the simulation it could indeed very well be much faster.
I agree that 20 watts can be enough to support intelligence and if we can figure out how to get there, it will take us much less time than a billion years. I also think that on the compute side for developing the AGI we should count all the PhD brains churning away at it right now :)
From Wikipedia on Axial Flux Motors:
>"Mercedes-Benz subsidiary YASA (Yokeless and Segmented Armature) makes AFMs that have powered various concept (Jaguar C-X75), prototype, and racing vehicles. It was also used in the Koenigsegg Regera, the Ferrari SF90 Stradale and S96GTB, Lamborghini Revuelto hybrid and the Lola-Drayson.[9] The company is investigating the potential for placing motors inside wheels, given that AFM's low mass does not excessively increase a vehicle's unsprung mass.[10] "
That's unfortunate. My personal sense is that while agentic LLM's are not going to get us close to AGI, a few relatively modest architectural changes to the underlying models might actually do that, and I do think mimicry of our own self-referential attention is a very important component of that.
While the current AI boom is a bubble, I actually think that AGI nut could get cracked quietly by a company with even modest resources if they get lucky on the right fundamental architectural changes.
I agree - and I think having interdisciplinary approach here is going to increase the odds here. There is a ton of useful knowledge in related disciplines - often just named differently - but turns out investigating the same problem from a different angle.
I mean historically, with essentially every other incrrease in technology and GDP this has been true, at least broadly speaking.
The quality of life of the poorest in the world has been improving over time.
It is absolutely worth considering "would things really be different this time?" But it's also a mistake to think automatically that it WILL be different this time.
Past performance is not indicative of future results.
The problem is we have to think about the actions we are taking and what those results will be. Just saying "oh, all of history before now needed human labor, but don't worry we're replacing that and nothing is going to change" is a nonsensical statement. In previous times when massive technology changes occurred we typically see very large wars break out. This was terrible and all, but war was a regional thing with regional effects. During the 20th century war became a global thing with global consequences all the way up to biosphere altering weapons of mass destruction.
The threat of MAD set up a time of relative global peace coupled with fast transportation and an explosion in the human population has set up a very fragile state of humanity. Products must keep moving and energy must keep being produces or billions and billions of people will die because they need things from the other side of the world to show up.
Depending on data quality in the area you need, one of the many apps that use OpenStreetMap data can be a good backup option. Where I am in the US it is plenty reliable enough for highway driving.
Yes, I'm aware. And that's my default minimum starting place which works until you realize that google maps routed you through a private gated road and you need to readjust and reroute while out of cell-range. Or a large section of the road is passable only on foot or perhaps with a quad, but certainly not even a 4wd truck. Or it's convinced that you should simply "turn left!" off a cliff numerous times.
All of which happened to me just last week here in rural CA.
May I suggest caching/downloading your map data? Google Maps for example will allow you to cache areas. I used it when traveling cross country. Super duper valuable.
Are we just ignoring the potential for totally novel forms of prion-like proteins? Like there are many other proteins. If another of them is prone to catalyzed misfoldings like prions are, that could be a seriously humanity-threatening event.
Not sure what you mean. One can easily just make the existing prions if they wanted; they are good enough to destroy humanity if one finds a potent delivery mechanism. Again, the problem is not the protein printing tech or the protein design tech; you need to figure out a contagious, multiplying delivery machanic, ie a virus.
It may surprize people who dont work in these areas, but finding ways to kill humans by injecting something in them is completely trivial. The hard part is finding something that does not kill them and has a therapeutic effect.
If that was a genuine question, yes I have a very decent idea what prions are. We were discussing if the computational design tech should be giving people pause and the answer is no. If anyone really wants to create nasty stuff they dont need computers much and certainly no fancy ML tools. After the design, getting nasty stuff into humans typically happens either via viruses, pathogens, foods, or illicit drugs. Illicit drugs and foods are monitored in many parts of the world, and in Europe it is hard to mess with the food supply at scale. But even if you did, there is really no need for these designer tools as there are enough nasty sequences, including prions, that are known. On the other hand, these computational tools can help create safe biomolecules — it is just too easy to create nasty stuff with chemistry and biochemistry, but it is still very very hard to create safe and effective therapeutic molecules.
> Viruses are unnecessary for prions (near as we can tell), and a nasty virus can do plenty of direct damage.
Agreed
> Someone could (if they discovered one) construct a nasty prion directly eh? And we’d probably be pretty fucked.
It would be easy to construct a potent prion-like sequence that works in isolation (in vitro) with such a tool, but not clear what the rationale would be even if one has nefarious purposes. We know of enough of them (eg pick Creutzfeldt–Jakob disease for human hosts; if you have a hydrogen bomb, the variations on the engineering could be intersting but most likely not needed.) I dont think we would be fucked if someone constructed new prion-like sequences with this method, though we might be at risk if someone makes genuine efforts to find ways to deliver the existing (or new ones) to humans at scale. The main risk factors in prions diseases involve exposure and host compatibility. You need enough time and thus enough of a starting dose for harm to happen. You need to enter the host and go to the right place, say nervous system, without being chopped up along the way by random proteases and without self aggregating in the wrong place and becoming useless. There may be other computational tools that in combination would help realise more potent/nasty/risky designs, but it is probably much easier to use human cell lines and benchwork (design a funky evolution experiment) to fish out nasty sequences that actually work in human cells.
Overall in biology and chemistry killing is easy, therapy is hard. Prions are somewhat interesting from a biology or engineering perspective, but they dont have the huge added risks of rapid exponential growth and easy transmission to hosts that many viruses have. They seem more like a curiosity and primitive danger form, and not a huge public health risk factor. Maybe there even exist some semi benign (even useful?) forms related to prions that we dont study much yet and they may end up influencing things like our understanding of aging or immune responses.
Prions relate to nucleation and first order phase transitions, so they have some technical appeal to people with past background in physics or chemistry, including myself, but they are not making it in my top 20 threats for humanity, not even if there were concerted efforts by rogue actors to create new nasty ones.
prions are transmitted from animal to animal in the wild, for example chronic wasting disease or kuru. Isn't it possible for them to spread via an STD or something? Or can they only be grown in the brain?
If your business model relies on fairly draconian control over the liberties of individuals in ways that defy normal human social norms (like propagating and giving away plants), then its probably not the right business model. Not even just from a moral standpoint; its a losing business model.
Also, exerting control over a commercial cloning effort and trying to control personal use and propagation are totally different scenarios.
The reality is their business is just not going to be substantially hurt by personal propagation and use.
Again, the business recognizes this difference. They openly permit personal propagation. They just don't want you starting a for-profit glowing plants nursery with your clones.
Practically? Unless you gift it to a thousand "friends", they're unlikely to a) hear of it, b) care about it, or c) break even on the legal costs of going after it.
reply