I guess from Nvidia’s perspective a lot of those thing don’t really matter as long as the people making/using “AI” need hardware. As long as Nvidia is a semi-monopoly it’s probably one of the safest bets in that way

There are echoes of the cryptomania boom here. Nvidia had the field for itself there too. The question is how soon, how fast, how widespread, in what shape and how lucrative the permanent adoption of AI algorithms.

For example, if the largest models are trained and shared as open source and only inference is done locally, its dominant position will be erode much faster.

On the other hand if only a few supersized entities train large models and serve them through API's they will eventually have their own datacenter chips.

Ultimately the Nvidia booms and busts reflect the lack of serious competition in this space. Once the previous oligopolist made the wrong calls on multi-core CPU designs, the field was left wide open.

The more general challenge for these players is how their offering will land in the post-Moore's law landscape.

These accelerator type chips are hard to program, consume lots of electricity and are good for very specific tasks. The mechanics and economics of the mass market CPU era dont apply.

Grab your pop corn.

> For example, if the largest models are trained and shared as open source and only inference is done locally, its dominant position will be erode much faster.

Open source models would still need to be trained on Nvidia’s HW?

I agree with your other points though, if the market becomes heavily concentrated I just don’t see the larger companies not developing their own chips/funding Nvidia’s competitors. Data center seems to be fundamentally different from the gaming market in the way that price/performance is the only thing that really matters. Which would imply that it should be easier for other competitors to challenge Nvidia.

> Open source models would still need to be trained on Nvidia’s HW?

yes, but how much hardware would really be required for training in this scenario and does it support a stratospheric valuation? Training models versus using models are dramatically different processes. Training is very intensive in cycles and memory but it is one-off and centralized to the model developer. Inferencing is in contrast relatively less intensive but its use is continuous and decentralized to potentially billions of users. Consider also that we are likely in the brute-force era and model size will keep shrinking, potentially quite dramatically.

While models have a lifecycle and will, in general, need to be retrained with new data, be specialized to private data etc, once a model is released it may be used for years without change. In fact you want this stability and longevity to release things in production.

My guess (but it is only a guess in what is a very uncertain landscape) is that just like mass-produced general purpose CPU's eventually won the economics game and became the dominant chip design (disrupting both the RISC workstations and HPC supercomputer clusters of the time), inferencing chips will win the mass produced "AI" economics game and will become the dominant accelerator, co-processor or super-sized CPU design.

The important commonality between inferencing and training is that both are numerical linear algebra. The economic model that will successfully commercialize inferencing is thus likely to drive the entire architecture for this specialized type of compute - the mass produced inferencing architectures will subsidize developing the higher-end training configurations.

Of-course, even if that reading is accurate, Nvidia could still somehow compete, but it could also be disrupted fairly structurally.