> The real problem is the ROI on AI spending is.. pretty much zero.
Companies in user acquisition/growth mode tend to have low internal ROI, but remember both Facebook and Google has the same issue -- then they introduced ads and all was well with their finances. Similar things will happen here.
LLMs are a utility, not a platform, and utility markets exert a downward pressure on pricing. Moreover, it's not obvious that -- once trained models hit the wild -- any one actor has or can develop significant competitive moats that would allow them to escape that price pressure. Beyond that, the digital marginal cost of services needs to be significantly reduced to keep these companies in business, but more efficient models leads to pushing inference out to end-user compute, which hollows out their business model (I assume that Apple dropping out of the OpenAI investment round was partially due to the wildly optimistic valuations involved, partially because they're betting on being able to optimize runtime costs down to iPhone levels).
Basically, I'd argue that LLMs look less like a Web 2.0 social media opportunity and more like Hashicorp or Docker, except with operational expenses running many orders of magnitude higher with costs scaling linearly to revenue.
> LLMs are a utility, not a platform, and utility markets exert a downward pressure on pricing.
I think competition exerts a downward pressure on pricing, not being a utility personally. But I guess I agree with the utility analogy in that there are massively initial upfront costs and then the marginal costs are low.
> more efficient models leads to pushing inference out to end-user compute, which hollows out their business model
Faster CPUs have been coming forever but we keep coming up with ways of keeping them busy. I suspect the same pattern with AI. Thus server-based AI will always be better than local. In the future, I expect to be served by many dozens of persistent agents acting on my behalf (more agents as you go further into the future) and they won't be hosted on my smartphone.
> 10 years ago a decade old computer vs a current one would have made a huge difference.
For running Word or Excel or Node.js server apps, I would agree with you. But this is where new applications come in. Modern PCs with either a GPU or an NPU can run circles around your PC when it comes to running Llama or StableDiffusion locally. Same with regards to high end graphics, old PCs can not do real-time raytracing or upscaling with their lesser capabilities. I personally do rendering via Blender or astrophotography via PixInsight and I need all the cores + memory I can get for that.
Faster PCs make for more opportunities that were not possible earlier. But if you do not change your workloads as the hardware evolves, then you do not need to upgrade.
And on other side, how many more paying users will there be? And of users that known about AI or have tried it are happy to use whatever is free at the moment. Or just whatever is on Google or Bing with add next to it?
Social media is free, subscription services are for stuff you can't get for free easily. But will there actually be similar need for AI? Be it any generation.
So I go to ask an LLM to answer to a question and it starts trying to sell me products I don't want or need? I'll need to filter it's output through a locally run adblock LLM which detects/flags/strips out advertising text before delivering the output. Hmmm.
I still have this pipe-dream of an augmented reality system where you can walk through the grocery store and it will superimpose a warning label over any brand connected to a company on the individual user's shit-list.
For many decades consumers have been told that the magnanimous "Free Market" (multiple definitions) is empowering them to vote with their wallets, however some of those same groups start acting rather suspiciously when faced with the possibility consumers might ever exercise that power.
> I still have this pipe-dream of an augmented reality system where you can walk through the grocery store and it will superimpose a warning label over any brand connected to a company on the individual user's shit-list.
There are boycott apps that let you scan barcodes, e.g.:
Companies in user acquisition/growth mode tend to have low internal ROI, but remember both Facebook and Google has the same issue -- then they introduced ads and all was well with their finances. Similar things will happen here.