But it is a programming language. You do not break backwards compatibility lightly. Nobody has ever chosen Python for its runtime performance. Unfortunately, sometimes you have to accept the technical debt cannot be escaped.
> Nobody has ever chosen Python for its runtime performance.
No, they choose it for the ease of using its performant extensions. And those extensions are fundamentally limited in performance by the existence of the GIL. And the authors of those extensions (and their employers) are behind the work to get rid of it.
There are three groups here:
1. "Pure" Python users, whose code makes little/no use of extensions. GIL removal is an immediate win for these uses.
2. "Good" extension users/authors, whose code will support and benefit from no-GIL. GIL removal is an immediate win for these uses.
3. "Bad" extension users/authors, whose code doesn't/can't support no-GIL. GIL removal probably doesn't break existing code, but makes new uses potentially unsafe.
Maintaining the technical debt of the GIL indefinitely so that group 3 never needs to address its own technical debt is not a good tradeoff for groups 1 and 2 which actively want to move the language forwards.
The above arguments could apply to any breaking changes that a host wants to implement. Those who don’t directly consume the change, those who can adapt, and everyone who is fine with the status quo.
Python is free to do as it pleases, but this breaking change is going to result in a lot of churn.
> GIL removal is an immediate win for these uses.
As a minor aside, the GIL free Python version was actually a performance regression, somewhere between 5-10% impact. https://peps.python.org/pep-0703/#performance . So, an immediate loss for nearly everyone.
No, this is false. Relying on the GIL isn't bad practice in a python extension. It's something you have no choice but to do. The GIL is the thread safety guarantee given to the extension by the python intepreter. It's the contract you have and you have no choice but to code to it. Removal of the GIL requires an alternative contract representing a finer-grained set of thread-safety guarantees for people to code against. Programmers who coded against the contract that existed rather than somehow divining the future and coding against a different contract that hadn't been invented yet (while also managing to make it compatible with the existing one) weren't doing something wrong.
But besides being incorrect, the idea that extensions that break on removal of the GIL are "bad" is irrelevant. The question is what the ecosystem will bear. The people who own the codebases I'm talking about are not pushing for GIL removal. They have large, working codebases they're using in production and the cost of redesigning large chunks of them would be real. GILectomy is being pushed by people with varied motives but they certainly don't speak for everyone and their case isn't going to be made successfully by casting aspersions on those with different priorities.
Relying on a published guarantee of CPython is not technical debt!
What will happen is that extensions that don't work with the new behavior will be assimilated by the proponents. Support for the new behavior will be hacked in, probably with many mistakes.
The bad results will then be marketed as progress.
it is now! just like that bill you didn't know you owed until it came in the mail, technical debt builds up regardless of when you know when it's owed.
keeping a gil when entering and exiting c extensions seems like an obvious steppingstone.
That doesn't contradict what the GP said. Those modules are largely written in C, not Python. And that is because nobody would choose Python for its performance.
> And that is because nobody would choose Python for its performance.
Python the runtime may not be the most performant by itself, but python the ecosystem only took of because of performance hacks like those popular modules, pypi, cython, ... .
> Those modules are largely written in C, not Python.
Yet people choose C to implement those modules because Python is not performant enough and isn't that a nice thought? Lets do something good for humanity and drive another stake into that old abominations heart by making it less relevant.
What? Pypy and cython are hardly the reason for python "taking off".
The issue with python is that the ecosystem is so big, everyone thinks their niche is the one python was "built for". Data science, web backends, sysadmin, app scripting, superclusters, embedding... They all think they're the biggest dog, when the reality is that they're more or less all the same. Pypy and cython are probably popular in some of those niches, but python is not just about them.
The reality is that python "took off" because of two things:
- the syntax
- the ease of interfacing with other worlds, typically (but not limited to) C/C++
Removing the GIL seriously threatens the second reason, because integrations that used to be fairly easy will have to be fundamentally rearchitected. Moving from Python 2 to 3 was trivial in comparison, and it took a decade; this transition might take much longer, or never really happen, with massive losses for the ecosystem. And all for what, some purist crusade that interests only a fraction of the whole community...?
