This article does not touch on the thing which worries me the most with respect to LLMs: the dependence.
Unless you can run the LLM locally, on a computer you own, you are now completely dependent on a remote centralized system to do your work. Whoever controls that system can arbitrarily raise the prices, subtly manipulate the outputs, store and do anything they want with the inputs, or even suddenly cease to operate. And since, according to this article, only the latest and greatest LLM is acceptable (and I've seen that exact same argument six months ago), running locally is not viable (I've seen, in a recent discussion, someone mention a home server with something like 384G of RAM just to run one LLM locally).
To those of us who like Free Software because of the freedom it gives us, this is a severe regression.
Yes, and it's even worse: if you think LLMs may possibly make the world a worse place, you should not use any LLMs you aren't self-hosting, because your usage information is being used by the creators to make LLMs better.
I think that’s a bit of a leap; if you think LLMs make the world a worse place, there are many actions that you might take or not take to try to address that.
To be fair, I don't have any great specific ideas, but "Work Without the Worker" for example talks about how a lot of LLMs are fueled by neo-colonialist exploitation.
So I guess broadly speaking there could be strategies involving attempting to influence governmental policy rather than by consumer choice.
Or more radically, trying to change the structure of the government in general such that the above influences actually are more tractable for the common person.
It's also why local models, even if less powerful, are so important. The gap between "state of the art" and "good enough for a lot of workflows" is narrowing fast
Yeah I am very excited for local models to get good enough to be properly useful. I’m a bit of an AI skeptic I’ll admit, but I’m much more of a SV venture-backed company skeptic. The idea of being heavily reliant on such a company, plus needing to be online, plus needing to pay money just to get some coding done is pretty unpalatable to me.
You can get 90%+ of the way there with a tiny “coder” LLM running on the Ollama backend with an extension like RooCode and a ton of MCP tools.
In fact, MCP is so ground breaking that I consider it to be the actual meat and potatoes of coding AIs. Large models are too monolithic, and knowledge is forever changing. Better just to use a small 14b model (or even 8b in some cases!) with some MCP search tools, a good knowledge graph for memory, and a decent front end for everything. Let it teach itself based on the current context.
And all of that can run on an off the shelf $1k gaming computer from Costco. It’ll be super slow compared to a cloud system (like HDD vs SSD levels of slowness), but it will run in the first place and you’ll get *something* out of it.
DesktopCommander and Taskmaster are great to start with. With just these, you may start to see why OP recommends a memory MCP too (I don’t have a recommendation for that yet)
* Not even counting cellular data carriers, I have a choice of at least five ISPs in my area. And if things get really bad, I can go down to my local library to politely encamp myself and use their WiFi.
* I've personally no need for a cloud provider, but I've spent a lot of time working on cloud-agnostic stuff. All the major cloud providers (and many of the minors) provide compute, storage (whether block, object, or relational), and network ingress and egress. As long as you don't deliberately tie yourself to the vendor-specific stuff, you're free to choose among all available providers.
And many do. The US isn't the entire world, you know.
> ...what kind of software do you write that pays your bills?
B2B software that allows anyone to run their workloads with most any cloud provider, and most any on-prem "cloud". The entire point of this software is to abstract out the underlying infrastructure so that businesses can walk away from a particular vendor if that vendor gets too stroppy.
> ...your setup doesn't require any external infrastructure...
It's Gentoo Linux, so it runs largely on donated infra (and infra paid for with donations). But -unlike Windows or OS X users- if I get sick of what the Gentoo steering committee are doing, I can go to another distro (or just fucking roll my own should things get truly dire). That's the point of my comment.
Of course you can. It's called an AS (autonomous system), I think all you need is an IP address range, a physical link to someone willing to peer with you (another AS), some hardware, some paperwork, etc; and bam you're your own ISP.
My company has set this up for one of our customers (I wasn't involved).
> all you need is an IP address range, a physical link to someone willing to peer with you (another AS), some hardware, some paperwork, etc; and bam you're your own ISP.
I'm pretty sure the connotation of "self-host" entails a strictly substantially smaller scope than starting your own ISP.
Finding someone willing to peer with you also defeats the purpose. You are still fundamentally dependent on established ISPs.
This is why I run a set of rackmount servers at home, that have the media and apps that I want to consume. If my ISP bites the dust tomorrow, I've literally got years worth of music, books, tv, movies, etc. Hell, I even have a bunch of models on ollama, and an offline copy of wikipedia running (minus media, obv) via kiwix.
It's not off-grid, but that's the eventual dream/ goal.
I don't feel like being dependent on LLM coding tools is much of an issue, you can very easily switch between different vendors. And I hope that open weight models will be "good enough" until we get a monopoly.
In any case, even if you are afraid of getting too dependent on AI tools, I think everyone needs to stay up to date on what is happening. Things are changing very quickly right now, so no matter what argument you may have against LLMs, it may just not be valid any more in a few months
> I think everyone needs to stay up to date on what is happening. Things are changing very quickly right now, so no matter what argument you may have against LLMs, it may just not be valid any more in a few months
This actually to me implies the opposite of what you’re saying here. Why bother relearning the state of the art every few months, versus waiting for things to stabilize on a set of easy-to-use tools?
We will have the equivalent of Claude Sonnet 4 in a local LLM that can run well on a modern Mac w/ 36+ GB of ram in a year or two. Maybe faster. The local/open models are developing very fast in terms of quantization and how well they can run on consumer hardware.
Folks that are local LLMs everyday now will probably say you can basically emulate at least Sonnet 3.7 for coding if you have an real AI workstation. Which may be true, but the time and effort and cost involved is substantial.
That will work until there has been a lot of infrastructure created to work with a particular player, and 3rd party software.
See the Microsoft ecosystem as an example. Nothing they do could not be replicated, but the network effects they achieved are strong. Too much glue, and 3rd party systems, and also training, and what users are used to, and what workers you could hire are used to, now all point to the MS ecosystem.
In this early mass-AI-use phase you still can easily switch vendors, sure. Just like in the 1980s you could still choose some other OS or office suite (like Star Office - the basis for OpenOffice, Lotus, WordStar, WordPerfect) without paying that kind of ecosystem cost, because it did not exist yet.
Today too much infrastructure and software relies on the systems from one particular company to change easily, even if the competition were able to provide a better piece of software in one area.
Good thing it's funded by generous investors or groups who are okay with losing money on every sale (they'll make it up in volume), and never stop funding, and never raise prices, insert ads or enshittify.
AKCSHUALLY the M-series CPU memory upgrades are expensive because the memory is on-chip and the bandwidth is a lot bigger than on comparable PC hardware.
In some cases it's more cost effective to get M-series Mac Minis vs nVidia GPUs
They know that, but all accounts I've read acknowledge that the unified memory is worse than dedicated VRAM. It's just much better than running LLMs on CPU and the only way for a regular consumer to get to 64GB+ of graphical memory.
Memory scaling has all but stopped. Current RAM cells are made up of just 40,000 or so electrons (that's when it's first stored. It degrades from there until refreshed). Going smaller is almost impossible due to physics, noise, and the problem of needing to amplify that tiny charge to something usable.
For the past few years, we've been "getting smaller" by getting deeper. The diameter of the cell shrinks, but the depth of the cell goes up. As you can imagine, that doesn't scale very well. Cutting the cylinder diameter in half doubles the depth of the cylinder for the same volume.
If you try to put the cells closer together, you start to get quantum tunneling where electrons would disappear from one cell and appear in another cell altering charges in unexpected ways.
The times of massive memory shrinks are over. That means we have to reduce production costs and have more chips per computer or find a new kind of memory that is mass producible.
That's going full speed ahead though. Every major cloud provider has an AI offering, and there are now multiple AI-centric cloud providers. There is a lot of money and speculation. Now Nvidia has their own cloud offering that "democratize access to world-class AI infrastructure. Sovereign AI initiatives require a new standard for transparency and performance".
How much of useful programming work are you able to do without google? I don't think I even tried to do do any for the last 20 years.
You make a good point of course that independence is important. But primo, this ship sailed long ago, secundo, more than one party provides the service you depend on. If one failes you still have at least some alternatives.
I guess it depends on how you define "using google", but as I've progressed as a developer, I've found myself spending less time googling problems, and more time just looking at official documentation (maybe GitHub issues if I'm doing something weird). And yeah, I guess I technically use google to get to the documentation instead of typing in the URL, but that feels like splitting hairs.
And it's not like people weren't able to develop complicated software before the internet. They just had big documentation books that cost money and could get dated quickly. To be clear, having that same info a quick google search away is an improvement, and I'm not going to stop using google while it's available to me. But that doesn't mean we'd all be screwed if google stopped existing tomorrow.
Takes a little adjustment, but you can do quite a bit of good work without Google (or any search).
Spoken from a fair bit of experience doing software development in closed rooms with strict control of all digital devices (from your phone to your watch) and absolutely no external connections.
There are moments that are painful still, because you'll be trying to find a thing in a manual and you know a search can get it faster - but it's silly to imply this isn't possible.
I can't say I've ever tried to intentionally not use google when working, unless I'm wanting to work offline entirely.
That said I only find google results somewhat helpful. Its a lot like LLM code (not surprising given how they're trained), I may find 5 answers online and one or two has a small piece of what I need. Ultimately that may say me a bit of time or give me an idea for something I hadn't thought of, but it isn't core to my daily work by any stretch.
There's a difference in being dependent on parts of the job versus the entire job.
I mostly write JS today and it either runs in browsers (dependencies) or a host like AAwS (dependencies). I use VS Codium and a handful of plugins (dependencies).
These all help me work efficiently when I'm coding, or help me avoid infrastructure issues that I don't want to deal with. Any one part is replaceable though, and more importantly any one part isn't responsible for doing my entire job of creating and shipping code.
I did code before Google, and I was fine. Yes, it's really convenient, and LLM would be even more convenient if I could trust it just a little bit more, but it's quite possible to do some effective software development without Google.
In 8th grade, I had a little PHP 4 pocket reference book. In classes I didn’t care about, I would have this open inside the book for that class and I would write code for my forums on loose leaf (in a shorthand). I also had printed copies of Mercury Board source code to refer to in the back of my binder. Then I’d get home later, type it in, debug a bit, and have new features :) It’s an entirely alien analog process to modern students, I’m sure, but it was really effective!
There are many alternatives though. It is not like Google has a search monopoly or office product monopoly, or e-mail provider monopoly. It is quite possible to cut out a lot of Google from one's life, and not even complicated to do that.
Not really, no. Though I would argue if Google disappeared tomorrow, as a private person you would probably do mostly fine. The point being, that your dependence is most likely not that strong actually. Unless you got important mail arriving at only your gmail mailbox. That would be dangerous. I lost several good accounts on other websites that way in the past. Now I don't register anything useful on gmail addresses any longer, in fact don't actually use gmail any longer, unless I still have some old accounts that I still didn't migrate away out of laziness.
>To those of us who like Free Software because of the freedom it gives us, this is a severe regression.
It's fair to be worried about depending on LLM. But I find the dependance on things like AWS or Azure more problematic, if we are talking about centralized and proprietary
Well, I'd think of it like being car-dependent. Sure, plenty of suburbanites know how to walk, they still have feet, but they live somewhere that's designed to only be practically traversable by car. While you've lived that lifestyle, you may have gained weight and lost muscle mass, or developed an intolerance for discomfort to a point where it poses real problems. If you never got a car, or let yourself adapt to life without one, you have to work backwards from that constraint. Likewise with the built environment around us; the cities many people under the age of 40 consider to be "good" are the ones that didn't demolish themselves in the name of highways and automobiles, in which a car only rarely presents what we'd think of as useful technology.
There are all kinds of trades that the car person and the non-car person makes for better or worse depending on the circumstance. The non-car person may miss out on a hobby, or not know why road trips are neat, but they don't have the massive physical and financial liabilities that come with them. The car person meanwhile—in addition to the aforementioned issues—might forget how to grocery shop in smaller quantities, or engage with people out in the world because they just go from point A to B in their private vessel, but they may theoretically engage in more distant varied activities that the non-car person would have to plan for further in advance.
Taking the analogy a step further, each party gradually sets different standards for themselves that push the two archetypes into diametrically opposed positions. The non-car owner's life doesn't just not depend on cars, but is often actively made worse by their presence. For the car person, the presence of people, especially those who don't use a car, gradually becomes over-stimulating; cyclists feel like an imposition, people walking around could attack at any moment, even other cars become the enemy. I once knew someone who'd spent his whole life commuting by car, and when he took a new job downtown, had to confront the reality that not only had he never taken the train, he'd become afraid of taking it.
In this sense, the rise of LLM does remind of the rise of frontend frameworks, bootcamps thay started with React or React Native, high level languages, and even things like having great internet; the only people who ask what happens in a less ideal case are the ones who've either dealt with those constraints first-hand, or have tried to simulate it. If you've never been to the countryside, or a forest, or a hotel, you might never consider how your product responds in a poor connectivity environment, and these are the people who wind up getting lost on basic hiking trails having assumed that their online map would produce relevant information and always be there.
Edit: To clarify, in the analogy, it's clear that cars are not intrinsically bad tools or worthwhile inventions, but had excitement for them been tempered during their rise in commodification and popularity, the feedback loops that ended up all but forcing people to use them in certain regions could have been broken more easily.
You can run LLMs locally pretty easily, especially if you have a Mac (the unified memory architecture of Macs is really good at this). It's a niche thing but caring about Free Software is niche.
You think an LLM provider has a bigger moat than an IDE (say pre vs code for a better parallel). MSDN and Jetbrains licenses are far more expensive than Cursor or Windsurf.
Sure, but that is not the point of the article. LLMs are useful. The fact that you are dependent on someone else is a different problem like being dependent on microsoft for your office suite.
I've had the same system (M2 64GB MacBook Pro) for three years.
2.5 years ago it could just about run LLaMA 1, and that model sucked.
Today it can run Mistral Small 3.1, Gemma 3 27B, Llama 3.3 70B - same exact hardware, but those models are competitive with the best available cloud-hosted model from two years ago (GPT-4).
The best hosted models (o3, Claude 4, Gemini 2.5 etc) are still way better than the best models I can run on my 3-year-old laptop, but the rate of improvements for those local models (on the same system) has been truly incredible.
I'm surprised that it's even possible running big models locally.
I agree we will see how this plays out but I hope models might start to become more efficient and it might not matter that much for certain things to run some parts locally.
I could imagine a LLM model with a lot less languages and optimized for one programming language to happen. Like 'generaten your model'
IMO Github doesn't matter for FOSS because you have a lot of local clones, it won't disappear forever if Github goes down or deletes the repo there. Self-hosted alts are not 100% up either. And I actually find collaboration functions / easy PR contribution on Github highly beneficial. At the same time I hate the friction of all those private Gitlabs, Giteas or, God forbid, gitweb.
> And I actually find collaboration functions / easy PR contribution on Github highly beneficial. At the same time I hate the friction of all those private Gitlabs, Giteas or, God forbid, gitweb.
FOSS is more about:
1. Finding some software you can use for your problem
2. Have an issue for your particular use case
3. Download the code and fix the issue.
4. Cleanup the patch and send a proposal to the maintainer. PR is easy, but email is ok. You can even use a pastebin service and post it on a forum (suckless does that in part).
5. The maintainer merges the patch and you can revert to the official version, or they don't and you decides to go with your fork.
Self-hosting has always have a lot of drawbacks compared with commercial solutions. I bet my self-host file server has worse reliability than Google Drive, or my self-host git server has worse number of concurrent user than github.
It's one thing you must accept when self-host.
So when you self-host LLM, you must either accept a drop in output quality, or spend a small fortune on hardware
Wake up, you’re already dependent on everything, unless you stick exclusively to Python std and no outside batteries.
Maven central is gone and you have no proxy setup or your local cache is busted? Poof, you’re fucking gone, all your Springs, Daggers, Quarkuses and every third party crap that makes up your program is gone. Same applies to bazillion JS, Rust libraries.
If PyPI goes out and I cannot use NumPy, I can still roll-out my own implementation of linear algebra library, because I've got the required knowledge, and I've got it because I had to learn it instead rely on LLMs.
I definitely couldn't, because I just use NumPy. Maybe I could roll my own matrix multiplication, but anything more would require cracking open a textbook that I haven't looked at for a decade. And this was true before I touched an LLM.
Panamax works great for mirroring all of crates.io in 300-400GB, which is big but easily small enough for enthusiasts. I've got it on an external USB drive myself, and it's saved my bacon a few times.
We're not yet to that same point for performance of local LLM models afaict, though I do enjoy messing around them.
Unless you can run the LLM locally, on a computer you own, you are now completely dependent on a remote centralized system to do your work. Whoever controls that system can arbitrarily raise the prices, subtly manipulate the outputs, store and do anything they want with the inputs, or even suddenly cease to operate. And since, according to this article, only the latest and greatest LLM is acceptable (and I've seen that exact same argument six months ago), running locally is not viable (I've seen, in a recent discussion, someone mention a home server with something like 384G of RAM just to run one LLM locally).
To those of us who like Free Software because of the freedom it gives us, this is a severe regression.