Apple may have a bit of a lead in getting it actually deployed end-to-end but given the number of times I've heard "AI accelerator" in reference to mobile processors I'm pretty sure that silicon with 'NPUs' are probably all over the place already, and if they're not, they certainly will be, for better or worse. I've got a laptop with a Ryzen 7040, which apparently has XDNA processors in it. I haven't a damn clue how to use them, but there is apparently a driver for it in Linux[1]. It's hard to think of a mobile chipset launch from any vendor that hasn't talked about AI performance in some regards, even the Rockchip ARM processors seem to have "AI engines".
This is one of those places where Apple's vertical integration has a clear benefit, but even as a bit of a skeptic regarding "AI" technology, it does seem there's a good chance that accelerated ML inference is going to be one of the next battlegrounds for processor mobile performance and capability, if it hasn't started already.
For sure many devices will have them, but the trick will be to build this local web model in a way that leverages all of the local chips. Apple’s advantage is in not having to worry about all that. It has a simpler problem and better access to real local data.
Give my personal local data to a model running in the browser? Just feels a bit more risky.
I think they’ll package a variety of model formats and download the appropriate one for the user’s system. Apple and Microsoft both offer OS-level APIs that abstract over the specific compute architecture so within the platform you don’t need further specialization. On Apple hardware they’ll run their model via CoreML on whichever set of compute (CPU, GPU, NPU) makes sense for the task. On Windows it will use DirectML for the same. I’m not sure if there’s a similar OS abstraction layer on Linux, possibly there they’ll just use CPU inference or whatever stack they usually use on Android.
If you compare a random older Windows laptop to a new or nearly new Mac, sure. But new Windows/x86 PCs have gotten pretty efficient. Here's a review showing a Meteor Lake laptop getting 15 hours of video playback on battery: https://www.anandtech.com/show/21282/intel-core-ultra-7-115h...
Exactly. So feature-wise it can work, and out of our "I proudly spent $2500 on my fancy Apple laptop"-tech bubble, people already learned to settle on something hotter.
And would still need to give Chrome access to your contacts, files etc to make it equivalent. It’s not useless, obviously it’ll be good, it’s just not the same. My original comment was replying to:
“enable approximately what Apple is doing with their AI strategy”
This is one of those places where Apple's vertical integration has a clear benefit, but even as a bit of a skeptic regarding "AI" technology, it does seem there's a good chance that accelerated ML inference is going to be one of the next battlegrounds for processor mobile performance and capability, if it hasn't started already.
[1]: https://github.com/amd/xdna-driver