2 months ago Jensen Huang did an interview where he said xAi built the fastest cluster with 100k GPUs.he said "what they achieved is singular, never been done before"
https://youtu.be/bUrCR4jQQg8?si=i0MpcIawMVHmHS2e
Meta said they would expand their infrastructure to include 350k GPUs by the end of this year. But, my guess is they meant a collection of AI clusters not a singular large cluster. In the post where they mentioned this, they shared details on 2 clusters with 24k GPUs each.https://engineering.fb.com/2024/03/12/data-center-engineerin...
What's singular is putting 100k H100s in a single machine. Which, yay, cool supercomputer, but the distributed supercomputer with 5 times the machines runs just as fast anyways.
Huang is still a CEO trying to prop up his product. He'd tell you putting an RTX4090 in your bathroom to drive an LED screen mirror is unprecedented if it meant it got him more sales and more clout.
