Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You've conveniently described python behavior but lots of C code relies on the gil implicitly and will need to add locking to be correct in a nogil world.

I'm not saying for sure this is bad! I do think it is dishonest though about the potential impact. Lots of critical libraries are written in C.



This is why the proposal is to, by default, reenable the GIL at runtime (and print a warning to stderr) whenever a C extension is loaded, unless that extension explicitly advertises that it does not rely on the GIL.


Dishonest? I didn't hide that fact: "Removing the GIL will require a new ABI, so existing C-API extensions will minimally need to be rebuilt and may also require other changes. Updating C-API extensions is where the majority of work will be if the PEP is accepted."


Yeah but everyone knows that is obviously the biggest issue. Nobody was really concerned about the Python code which is the part that you said will go swimmingly—that's table stakes for a change at all. We already assumed the pure Python code would upgrade easily; a change would have no chance whatsoever of being accepted otherwise. Everyone is worried exclusively about the C extensions, and always has been. This seems to have been presented as a new approach to GIL removal that fixes the problem with C extension breakage but it's just the exact same old approach we've always been considering that breaks the C extensions. No-GIL ain't done until all the C extensions run, especially NumPy.


> Nobody was really concerned about the Python code

This was the top comment when I added my original comment:

"Fundamentally every single piece of python code ever written will have to stop and now worry about potential race conditions."

https://news.ycombinator.com/item?id=36644114

Most of the other comments here at the time were similarly from people who clearly hadn't read the PEP saying it would break all existing Python code.

So I did my best to represent the backwards compatibility section of the PEP. I told folks to go read it and linked to it. I cited the portion relevant to Python code. There were too many bullet points for the C-API, so I summarized it with the disclaimer "Updating C-API extensions is where the majority of work will be if the PEP is accepted."

I also read the PEP discussion thread where there was disagreement from Python maintainers on how much work would be required of C extension authors, but most of the folks stating it would be a lot of work hadn't seemed to actually have tried to port anything to the new API. Meanwhile Sam had asserted that:

> Most C API extensions don’t require any changes, and for those that do require changes, the changes are small. For example, for “nogil” Python I’m providing binary wheels for ~35 extensions that are slow or difficult to build from source and only about seven projects required code changes (PyTorch, pybind11, Cython, numpy, scikit-learn, viztracer, pyo3). For four of those projects, the code changes have already been contributed and adopted upstream. For comparison, many of those same projects also frequently require changes to support minor releases of CPython.

I don't think I've misrepresented anything.

> No-GIL ain't done until all the C extensions run, especially NumPy.

Sam already got NumPy working without the gil:

https://youtu.be/9OOJcTp8dqE?t=1958


I think the word "minimally" makes it sound like the changes to existing libraries are "to an extremely small extent; negligibly." "At a minimum" would fit better, because the minimum of a set of things can still be very large whereas minimally implies the quantity is very small.


But it was downplayed in the comment when it was always the #1 reason for keeping the GIL. Python level changes were never a serious part of the argument.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: