My headline feature is the new “abi3t” stable ABI for the free-threaded build. While Petr Viktorin did most of the CPython implementation, I’ve been trying to make sure ecosystem support is ready. It’s been a rewarding but quite challenging project to make sure everything is working. There were some late nights leading up to the beta1 release when we found a Windows-specific issue that needed a fix.
I’m particularly proud that the cryptography project is already shipping a single abi3.abi3t wheel for each platform on Python 3.15 or newer. The GIL-enabled build and free-threaded build can both use the same wheel now, because PyObject is opaque.
If you want to learn more about this, I gave a talk at EuroPython this year on Python’s ABI and the road to building and releasing abi3t today. See https://youtu.be/An8lO29SxXE.
As a maintainer of a Python module that uses abi3 now, thank you!
Just one question: any plans to promote APIs like PyUnstable_EnableTryIncRef() and PyUnstable_TryIncRef() into the limited API? So far we have found that these APIs are necessary to implement the weak-valued caches we've always used in the past: https://github.com/protocolbuffers/protobuf/blob/6559a9f9622...
I’m glad to see there’s ongoing work to get protobuf working!
I doubt PyUnstable APIs will get promoted directly to the limited API without at least a release or two in the regular version-specific API first.
That said, if you would be willing to start a thread on discourse describing your need and use-case, that is probably the first step to stabilizing the APIs you want. It may also turn out there’s an alternate way to get protobuf working.
Please do feel free to reach out privately. I’d love to be able
to chat with people working on protobuf and grpcio, they come up reasonably often and it’s hard to get official updates from inside Google.
It should be good to go, but of course I don’t know what your codebase looks like and issues are always possible in any code, “good to go” or no. Please do give it a try and try testing on 3.15 and 3.15t.
Came here to say this. A stable ABI will mean that libraries can make a single free-threaded build that'll last at least a couple of Python versions into the future.
There’s also the fact that INCREF and DECREF on shared objects is a lot more expensive than plain integer addition, so stuff becomes a bottleneck that was never a bottleneck. Kumar also fixed a bottleneck caused by a lock added only for safety on the free-threaded build. It’s hard to tell in advance than a fancy lock-free data structure is needed for something.
You're talking about the same thing as the article: why were things not as fast as they could be in free threaded mode?
But this comment thread is more of a meta discussion: why would lock contention issues only show up now, in free threaded mode, if numpy already released the GIL anyway? That's what I answered above: yes the GIL was sometimes released by numpy, but these operations really were happening with the GIL locked (in non free threaded build).
Right. The original commenter already seemed to understand that.
It's a new issue because the reference count needs locking, whereas previously it didn't because it was implicitly synchronised due to the GIL being locked.
But the original commenter asked: hang on, I thought the GIL wasn't locked for numpy?
Now you have reached the start of the conversation.
I’m not sure why it took me, a NumPy developer, looking at the benchmark numbers and saying “hmm, this is a bug”. But that is what it took. People are sometimes slow to treat behavior in dependencies like NumPy as bugs.
I’m particularly proud that the cryptography project is already shipping a single abi3.abi3t wheel for each platform on Python 3.15 or newer. The GIL-enabled build and free-threaded build can both use the same wheel now, because PyObject is opaque.
If you want to learn more about this, I gave a talk at EuroPython this year on Python’s ABI and the road to building and releasing abi3t today. See https://youtu.be/An8lO29SxXE.