I don't know. Can you bring your GPUs on an inter-continental plane trip and play with LLMs on the plane? It isn't really that slow for 70B 4-q models. These are very good CPU/GPUs, and they are only getting better.
Sure, the GPUs sit in my basement and I can connect to them from anywhere in the world.
My point was not that “it isn’t really that slow,” my point is that Macs are slower than dedicated GPUs, while being just as expensive (or more expensive, given the specific scenario) to purchase and operate.
And I did my analysis using the Mac Studio, which is faster than the equivalent MBP at load (and is also not portable). So if you’re using a MacBook, my guess is that your performance/watt numbers are worse than what I was looking at.
The whole point of having it local is not to use the network, or not need it, or not needing to jump the GFW when you are in China.
Ultra is about 2X of the power of a Max, but the Max itself is pretty beefy, and it has more than enough GPU power for the models that you can fit into ~48GB of RAM (what you have available if you are running with 64GB of memory).
If you travel to China, sure, what I’m talking about probably won’t work for you.
In pretty much any other situation, using dedicated GPUs is 1) definitely faster, like 2x the speed or more depending on your use case, and 2) the same cost or possibly cheaper. That’s all I’m saying.
