> The point is, why predict that the growth rate is going to slow exactly now? What evidence are you going to look at?
Why predict that the (absolute) growth rate is going to keep accelerating past exactly now?
Exponential growth always assumes a constant relative growth rate, which works in the fiction of economics, but is otherwise far from an inevitability. People like to point to Moore's law ad nauseam, but other things like "the human population" or "single-core performance" keep accelerating until they start cooling off.
> And note, there are good reasons to predict a speedup, too; as models get more intelligent, they will be able to accelerate the R&D process.
And if heaven forbid, R&D ever turns out to start taking more work for the same marginal returns on "ability to accelerate the process", then you no longer have an exponential curve. Or for that matter, even if some parts can be accelerated to an amazing extent, other parts may get strung up on Amdahl's law.
It's fine to predict continued growth, and it's even fine to predict that a true inflection point won't come any time soon, but exponential growth is something else entirely.
> Why predict that the (absolute) growth rate is going to keep accelerating past exactly now?
By following this logic you should have predicted Moore’s law would halt every year for the last five decades. I hope you see why this is a flawed argument. You prove too much.
But I will answer your “why”: plenty of exponential curves exist in reality, and empirically, they can last for a long time. This is just how technology works; some exponential process kicks off, then eventually is rate-limited, then if we are lucky another S-curve stacks on top of it, and the process repeats for a while.
Reality has inertia. My hunch is you should apply some heuristic like “the longer a curve has existed, the longer you should bet it will persist”. So I wouldn’t bet on exponential growth in AI capabilities for the next 10 years, but I would consider it very foolish to use pure induction to bet on growth stopping within 1 year.
And to be clear, I think these heuristics are weak and should be trumped by actual physical models of rate-limiters where available.
> By following this logic you should have predicted Moore’s law would halt every year for the last five decades. I hope you see why this is a flawed argument. You prove too much.
I do think it's continually amazing that Moore's law has continued in some capacity for decades. But before trumpeting the age of exponential growth, I'd love to see plenty of examples that aren't named "Moore's law": as it stands, one easy hypothesis is that "ability to cram transistors into mass-produced boards" lends itself particularly well to newly-discovered strategies.
> So I wouldn’t bet on exponential growth in AI capabilities for the next 10 years, but I would consider it very foolish to use pure induction to bet on growth stopping within 1 year.
Great, we both agree that it's foolish to bet on growth stopping within 1 year. What I'm saying that "growth doesn't stop" ≠ "growth is exponential".
A theory of "inertia" could just as well support linear growth: it's only because we stare at relative growth rates that we treat exponential growth as a "constant" that will continue in the absence of explicit barriers.
Solar panel cost per watt has been dropping exponentially for decades as well...
Partly these are matters of economies of scale - reduction in production costs at scale - and partly it's a matter of increasing human attention leading to steady improvements as the technology itself becomes more ubiquitous.
This is where I’d really like to be able to point to our respective Manifold predictions on the subject; we could circle back in a year’s time and review who was in fact correct. I wager internet points it will be me :)
Why predict that the (absolute) growth rate is going to keep accelerating past exactly now?
Exponential growth always assumes a constant relative growth rate, which works in the fiction of economics, but is otherwise far from an inevitability. People like to point to Moore's law ad nauseam, but other things like "the human population" or "single-core performance" keep accelerating until they start cooling off.
> And note, there are good reasons to predict a speedup, too; as models get more intelligent, they will be able to accelerate the R&D process.
And if heaven forbid, R&D ever turns out to start taking more work for the same marginal returns on "ability to accelerate the process", then you no longer have an exponential curve. Or for that matter, even if some parts can be accelerated to an amazing extent, other parts may get strung up on Amdahl's law.
It's fine to predict continued growth, and it's even fine to predict that a true inflection point won't come any time soon, but exponential growth is something else entirely.