NVidia would be dramatically affected, just not overnight.
If Intel or AMD sold a niche product with 48GB RAM even at a loss, but hit high-end consumer pricing, there would be a flood of people doing various AI work to buy it. The end result would be that parts of NVidia's moat would start draining rather quickly, and AMD / Intel would be in a stronger position for AI products.
I use NVidia because when I bought AMD during the GPU shortage, ROCm simply didn't work for AI. This was a few years back, but I was burned badly enough that I'm unlikely to risk AMD again for a long, long time. Unused code sits broken, and no ecosystem gets built up. A few years later, things are gradually improving for AMD for the kinds of things I wanted to do years ago, but all my code is already built around NVidia, and all my computers have NVidia cards. It's a project with users, and all those users are buying NVidia as well (even if just for surface dependencies, like dev-ops scripts which install CUDA). That, times thousands of projects, is part of NVidia's moat.
If I could build a cheap system with around 200GB, that would be incentive for me to move the relatively surface dependencies to work on a different platform. I can buy a motherboard with four PCI slots, and plug in four 48GB cards to get there. I'd build things around Intel or AMD instead.
The alternative is NVidia would start shipping competitive cards. If they did that, their high-end profit margins would dissolve.
The breakpoints for inference functionality are really at around 16GB, 48GB, and 200GB, for various historical reasons.
> If I could build a cheap system with around 200GB,
Even if AMD dropped the price to $2000, you could not be able to build a system with one of these cards. You cannot buy these cards at their current 5 digit pricing. The idea that you could buy it if they dropped the price to $2000 is a fantasy, since others would purchase the supply long before you have a chance to purchase one, just like they do now.
AMD is already selling out at the current 5 digit pricing and Nvidia is not affected, since Nvidia is selling millions of cards per year and still cannot meet demand while AMD is selling around 100,000. AMD dropping the price to $2000 would not harm Nvidia in the slightest. It would harm AMD by turning a significant money maker into a loss leader. It would also likely result in Lisa Su being fired.
By the way, the CUDA moat is overrated since people already implement support for alternatives. llama.cpp for example supports at least 3. PyTorch supports alternatives too. None of this harms Nvidia unless Nvidia stops innovating and that is unlikely to happen. A price drop to $2000 would not change this.
Let's compare to see if it's really the same market:
HGX B200: 36 petaflops at FP16. 14.4 terabytes/second bandwidth.
RX4060 (similar to Intel): 15 teraflops at FP16. 272 gigabytes/second bandwidth
Hmmm.... Note the prefixes (peta versus tera)
A lot of that is apples-to-oranges, but that's kind of the point. It's a different market.
A low-performance high-RAM product would not cut into the server market since performance matters. What it would do is open up a world of diverse research, development, and consumer applications.
Critically, even if it were, Intel doesn't play in this market. If what you're saying were to happen, it should be a no-brainer for Intel to launch a low-cost alternative. That said, it wouldn't happen. What would happen is a lot of business, researchers, and individuals would be able to use ≈200GB models on their own hardware for low-scale use.
> By the way, the CUDA moat is overrated since people already implement support for alternatives.
No. It's not. It's perhaps overrated if you're building a custom solution and making the next OpenAI or Anthropic. It's very much not overrated if you're doing general-purpose work and want things to just work.
What treprinum suggested was AMD selling their current 192GB enterprise card (MI300X) for $2000, not a low end card. Everything you said makes sense but it is beside the point that was raised above. You want the other discussion about attaching 128GB to a basic GPU. I agree that would be disruptive, but that is a different discussion entirely. In fact, I beat you to saying that would be disruptive by about 16 hours:
If Intel or AMD sold a niche product with 48GB RAM even at a loss, but hit high-end consumer pricing, there would be a flood of people doing various AI work to buy it. The end result would be that parts of NVidia's moat would start draining rather quickly, and AMD / Intel would be in a stronger position for AI products.
I use NVidia because when I bought AMD during the GPU shortage, ROCm simply didn't work for AI. This was a few years back, but I was burned badly enough that I'm unlikely to risk AMD again for a long, long time. Unused code sits broken, and no ecosystem gets built up. A few years later, things are gradually improving for AMD for the kinds of things I wanted to do years ago, but all my code is already built around NVidia, and all my computers have NVidia cards. It's a project with users, and all those users are buying NVidia as well (even if just for surface dependencies, like dev-ops scripts which install CUDA). That, times thousands of projects, is part of NVidia's moat.
If I could build a cheap system with around 200GB, that would be incentive for me to move the relatively surface dependencies to work on a different platform. I can buy a motherboard with four PCI slots, and plug in four 48GB cards to get there. I'd build things around Intel or AMD instead.
The alternative is NVidia would start shipping competitive cards. If they did that, their high-end profit margins would dissolve.
The breakpoints for inference functionality are really at around 16GB, 48GB, and 200GB, for various historical reasons.