State of Python 3.13 performance: Free-threading(codspeed.io) |
State of Python 3.13 performance: Free-threading(codspeed.io) |
Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "real" semantics.
On the other hand, a lot of these changes to try and speed up the base language are going to be highly disruptive. E.g. disabling the GIL will break tonnes of code, lots of compilation projects involve changes to the ABI, etc.
I guess getting loops in Python to run 5-10x faster will still save some people time, but it's also never going to be a replacement for the zoo of specialized python-like compilers because it'll never get to actual high performance territory, and it's not clear that it's worth all the ecosystem churn it might cause.
But then a few hours later, I tried running a very small project I wrote last year and it turned out that a bunch of my dependencies had changed their APIs. I've had similar (and much worse) experiences trying to get older code with dependencies running.
My meaning with this comment is, that if the average developer's reality is that backwards compatibility isn't really a thing anyway, then we are already paying for that downside so we might as well get some upside there, is my reasoning.
Stuff that is actually included with Python tends to be more stable than random Pypi packages, though.
NPM packages also sometimes change. That's the world.
I thereby kind of feel like this might have happened in the other direction: a ton of developers seem to have become demoralized by python3 and threw up their hands in defeat of "backwards compatibility isn't going to happen anyway", and now we live in a world with frozen dependencies running in virtual environments tied to specific copies of Python.
This was very much the opposite of my experience. Consider yourself lucky.
If the dependency was in external modules and you didn't have pinned versions, then it is to be expected (in almost any active language) that some APIs will break.
> A majority of Python projects use a scheme that resembles semantic versioning. However, most projects, especially larger ones, do not strictly adhere to semantic versioning, since many changes are technically breaking changes but affect only a small fraction of users...
[0] https://github.com/pydata/xarray/issues/6176
[1] https://numpy.org/doc/stable/dev/depending_on_numpy.html
[2] https://packaging.python.org/en/latest/discussions/versionin...
I like python (and swift for that matter) but I don't like the feeling that I am building on quicksand. Java, C++, and vanilla javascript seem more durable.
...
> I wrote last year and it turned out that a bunch of my dependencies had changed their APIs
these two things have absolutely nothing to do with each other - couldn't be a more apples to oranges comparison if you tried
Scientific computing community have a bunch of code calling numpy or whatever stuff. They are pretty fast because, well, numpy isn't written in Python. However, there is a scalability issue: they can only drive so many threads (not 1, but not many) in a process due to GIL.
Okay, you may ask, why not just use a lot of processes and message-passing? That's how historically people work around the GIL issue. However, you need to either swallow the cost of serializing data over and over again (pickle is quite slow, even it's not, it's wasting precious memory bandwidth), or do very complicated dance with shared memory.
It's not for web app bois, who may just write TypeScript.
Accellerated sub-languages like Numba, Jax, Pytorch, etc. or just whole new languages are really the only way forward here unless massive semantic changes are made to Python.
Kind of related, the other day I was cursing like a sailor because I was having issues with some code I wrote that uses StrEnum not working with older versions of Python, and wondering why I did that, and trying to find the combination of packages that would work for the version of Python I needed-- wondering why there was so much goddamn churn in this stupid [expletive] scripting language.
But then I took a step back and realized that, actually, I should be glad about the churn because it means that there is a community of developers who care enough about the language to add new features and maintain this language so that I can just pipe PyQt and Numpy into each other and get paid.
I don't have any argument, just trying to give an optimistic perspective.
I don't even understand what this means. If I write `def foo(x):` versus `def foo(x: int) -> float:`, one is a restricted subset of the other, but both are the language's "real" semantics. Restricted subsets of languages are wildly popular in programming languages, and for very varied reasons. Why should that be a barrier here?
Personally, if I have to annotate some of my code that run with C style semantics, but in return that part runs with C speed, for example, then I just don't really mind it. Different tools for different jobs.
You either are performing some wordplay here or you don't understand but type hints are not part of the semantics at all: since they are not processed at all they do not affect the behavior of the function (that's what semantics means).
EDIT: according to the language spec and current implementation
`def foo(x: int) -> float`
and
`def foo(x: float) -> int`
are the same exact function
What makes Python brilliant is that it’s easy to deliver on business needs. It’s easy to include people who aren’t actually software engineers but can write Python to do their stuff. It’s easy to make that Wild West code sane. Most importantly, however, it’s extremely easy to replace parts of your Python code with something like C (or Zig).
So even if you know performant languages, you can still use Python for most things and then as glue for heavy computation.
Now I may have made it sound like I think Python is brilliant so I’d like to add that I actually think it’s absolute trash. Loveable trash.
On the other hand though, Python is so big and there's so many corps using it with so much cash that maybe they can get away with just breaking shit every few releases and people will just go adapt packages to the changes.
It's just that the CPython developers and much of the Python community sat on their hands for 15 years and said stuff like "performance isn't a primary goal" and "speed doesn't really matter since most workloads are IO-bound anyway".
I think with the GIL some people are overreacting: most python code is single threaded because of the GIL. So removing it doesn't actually break anything. The GIL was just making the use of threads kind of pointless. Removing it and making a lot of code thread safe benefits people who do want to use threads.
It's very simple. Either you did not care about performance anyway and nothing really changes for you. You'd need to add threading to your project to see any changes. Unless you do that, there's no practical reason to disable the GIL for you. Or to re-enable that once disabled becomes the default. If your python project doesn't spawn threads now, it won't matter to you either way. Your code won't have deadlocking threads because it has only 1 thread and there was never anything to do for the GIL anyway. For code like that compatibility issues would be fairly minimal.
If it does use threads, against most popular advise of that being quite pointless in python (because of the GIL), you might see some benefits and you might have to deal with some threading issues.
I don't see why a lot of packages would break. At best some of them would be not thread safe and it's probably a good idea to mark the ones that are thread safe as such in some way. Some nice package management challenge there. And probably you'd want to know which packages you can safely use.
Because the language's semantics promise that a bunch of insane stuff can happen at any time during the running of a program, including but not limited to the fields of classes changing at any time. Furthermore, they promise that their integers are aribtrary precision which are fundamentally slower to do operations with than fixed precision machine integers, etc.
The list of stuff like this goes on and on and on. You fundamentally just cannot compile most python programs to efficient machine code without making (sometimes subtle) changes to its semantics.
_________
> I don't see why a lot of packages would break. At best some of them would be not thread safe and it's probably a good idea to mark the ones that are thread safe as such in some way. Some nice package management challenge there. And probably you'd want to know which packages you can safely use.
They're not thread safe because it was semantically guaranteed to them that it was okay to write code that's not thread safe.
It can be so fast that it completely mooted the discussions that often happen when wanting to move from a python prototype to 'fast enough for production'.)
Seems like a lose-lose to me. (which is presumably why it never caught on)
I would recommend being less reductively dismissive, after claiming you “don’t really have a dog in this race”.
Edit: Lots of recent changes have done way more than just loop unrolling JIT stuff.
My alternative is to serialise in heavy processes and then incur a post process unification pass, because the cost of serialise send/receive deserialise to unify this stuff is too much. If somebody showed me how to use shm models to do this so it came back to the cost of threading.lock I'd do the IPC over a shared memory dict, but I can't find examples and now suspect multiprocessing in Python3 just doesn't do that (happy, delighted even to be proved wrong)
The real barrier my thought experiment hit were the extensions. Many uses of Python are glue around C extensions designed for the CPython interpreter. Accelerating “Python” might actually be accelerating Python, C, and hybrid code that’s CPython-specific. Every solution seemed like more trouble than just rewriting those libraries to not be CPython-specific. Or maybe to work with the accelerators better.
Most people are just using high-level C++ and Rust in the areas I was considering. If using Python, the slowdown of Python doesn’t impact them much anyway since their execution time is mostly in the C code. I’m not sure if much will change.
- People choose Python to get ease of programming, knowing that they give up performance.
- With multi-core machines now the norm, they’re relatively giving up more performance to get the same amount of ease of programming.
- so, basically, the price of ease of programming has gone up.
- economics 101 is that rising prices will decrease demand, in this case demand for programming in Python.
- that may be problematic for the long-term survival of Python, especially with new other languages aiming to provide python’s ease of use while supporting multi-processing.
So, Python must get even easier to use and/or it must get faster.
Threads break down so many bottlenecks of CPU resources, memory, data serialization, waiting for communications, etc. I have a 16-core computer on my desk and getting a 12x speed-up is possible for many jobs and often worth worse less efficient use of the individual CPU. Java has many primitives for threads that include tools for specialized communications (e.g. barrier synchronization) that are necessary to really get those speed-ups and I hope Python gets those too.
But then I do have a lot of data, there's a lot of trial&error there for me so I'm not doing these tasks only once, and I would have appreciated a speedup of up to 16x. I don't know about "high performance", but that's the difference between a short coffee break and going to lunch.
And if I was a working on an actual data-oriented workstation, I would be using only one of possibly 100+ cores.
That just seems silly to me.
Sometimes the optimal level of parallelism lies in an outer loop written in Python instead of just relying on the parallelism opportunities of the inner calls written using hardware specific native libraries. Free-threading Python makes it possible to choose which level of parallelism is best for a given workload without having to rewrite everything in a low-level programming language.
There are lots of use case where Python's performance is acceptable but a 10x speed boost would be much appreciated. Or where Python's performance is not acceptable but it would be if I could fully utilize my 32 cores.
For example look at Instagram. They run tons of Python but need to run many processes on each machine wasting memory. I'm sure they would love to save that memory, the environment would too. Sure, they could rewrite their service in C but that is most likely not the best tradeoff.
Very little of what I use it for has performance bottlenecks in the Python. Its the database or the network or IO or whatever.
On the few occasions when it does I can rewrite critical bits of code.
I definitely care more about backward compatibility than I do about performance.
It feels like Python development is being driven by the needs of one particular group (people who use ML heavily, possibly because they have deep pockets) and I wonder whether this, and a few other things will make it less attractive a language for me and others.
Have you heard of Mojo? It is a very performant superset of Python. https://www.modular.com/mojo
If you have 128 cores you can compromise on being a bit slow. You can't compromise on being single-threaded though.
MP has the stupid serialization issues and anything involving the Python coder doing locking or mutexes is not Pyrhinic as they will struggle.
- https://github.com/scipy/scipy/issues/21479
it might be the same as this one that further involves OpenMP code generated by Cython:
- https://github.com/scikit-learn/scikit-learn/issues/30151
We haven't managed to write minimal reproducers for either of those but as you can observe, those race conditions can only be triggered when composing many independently developed components.
Learning it has only continued to be a huge benefit to my career, as it's used everywhere which underlies how important popularity of a language can be for developers when evaluating languages for career choices
3.13t doesn't seem to have been meant for any serious use. Bugs in gc and so on are reported, and not all fixes will be backported apparently. And 3.14t still has unavoidable crashes. Just too early.
I don't think anyone would suggest using it in production. The point was to put something usable out into the world so package maintainers could kick the tires and start working on building compatible versions. Now is exactly the time for weird bug reports! It's a thirty year old runtime and one of its oldest constraints is being removed!
That's interesting, I wouldn't have expected performance regressions coming from those releases. How can that be?
An oversimplified explanation (and maybe wrong) of it goes like this:
problem:
- each object needs a reference counter, because of how memory management in Python works
- we cannot modify ref counters concurrently because it will lead to incorrect results
- we cannot make each ref counter atomic because atomic operations have too large performance overhead
therefore, we need GIL.
Solution, proposed in [0]:
- let's have two ref counters for each object, one is normal, another one is atomic
- normal ref counter counts references created from the same thread where the object was originally created, atomic counts references from other threads
- because of an empirical observation that objects are mostly accessed from the same thread that created them, it allows us to avoid paying atomic operations penalty most of the time
Anyway, that's what I understood from the articles/papers. See my other comment [1] for the links to write-ups by people who actually know what they're talking about.
He also had a meeting with Python core. Notes from this meeting [1] by Łukasz Langa provide more high-level overview, so I think that they are a good starting point.
[0] https://docs.google.com/document/u/0/d/18CXhDb1ygxg-YXNBJNzf...
[1] https://lukasz.langa.pl/5d044f91-49c1-4170-aed1-62b6763e6ad0...
Does it eliminate the need for a GC pause completely?
I tend to use C++, so use SWIG [1] to make python code to interface with C++ (or C). You can nearly just give it a header file, and a python class pops out, with native types and interfaces. It's really magical.
The Go was a dead simple my first Go project sort of implementation and used 32 cores and therefore worked much better right out of the gate. (I mean I did have go routine worker pools for each step of the processing, but the division of work into the stages was already in the Python code).
So yeah Python is easy to make less of a mess of for lots of people, until you want to use all your cores (which again means rerunning your things goes from four minutes to 15 seconds on that fancy laptop).
In fact, Sam, the man behind free-threading, works on PyTorch. From my understanding he decided to explore nogil because GIL is holding DL trainings written in PyTorch back. Namely, the PyTorch DataLoader code itself and almost all data loading pipelines in real training codebases are hopeless bloody mess just because all of the IPC/SHM nonsense.
> Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.
But where this flexibility isn't required, which is a lot of performance sensitive number crunching code the cost of the flexibility bites you. You can't "turn it off" when you want control down to the instruction for a truly massive performance win. Which is why I think the model Python has of highly expressive and flexible language backed by high-performance compiled libraries is so successful.
Python will never be number crunching or parsing with the best of them because it would require essentially a whole new language to express the low-level constraints but for high-level code that relies on Python's semantics you can get performance wins that can't be accomplished just by switching to a compiled language. We've just taken the "embedded scripting language" and made it the primary interface.
It was not that simple, but it was not that hard either.
It took the industry years because Python 2.7 was still good enough, and the tangible benefits of migrating to Python 3 didn't justify the effort for most projects.
Also some dependencies such as MySQL-python never updated to Python 3, which was also an issue for projects with many dependencies.
Why not? Python does make breaking changes to the standard library when going from 3.X to 3.X+1 quite regularly.
Now even asyncore is gone -_-' Have fun rewriting all the older async applications!
https://peps.python.org/pep-0703/#reference-counting
If by GC you mean the cyclic GC, free-threaded Python currently stops all threads while the cyclic GC is running.
I wasn't aware of that, that's actually insane. It's odd to me that it took so long to get f-strings and Enums right in Python, I assumed those would be pretty easy language features to implement.
I know that Python 3.11 added some things, like StrEnum; those obviously won't work on older Python versions. But I'm not aware of things that work in a certain Python 3 version but don't work in newer ones. You're even talking about incompatibilities between different 3.11.x versions? Can you give some more detail on that?
I don't agree that it is "insane stuff", but I agree that Python is not where you go if you need super fast execution. It can be a great solution for "hack together something in a day that is correct, but maybe not fast", though. There are a lot of situations where that is, by far, the preferred solution.
julia is a great example of a highly dynamic language which is still able to compile complicated programs to C-equivalent machine code. An older (and less performant but still quite fast) example of such a language is Common Lisp.
Python makes certain choices though that make this stuff pretty much impossible.
People don't pick languages for language features, mostly. They pick them for their ecosystems -- the quality of libraries, compiler/runtime support, the network of humans you can ask questions of, etc.
> The list of stuff like this goes on and on and on. You fundamentally just cannot compile most python programs to efficient machine code without making (sometimes subtle) changes to its semantics.
People said the same thing about JavaScript, where object prototypes can change at any time, dramatically changing everything. But it turned out JavaScript can run fast if you try hard enough. I suspect the same would be true for Python if a similar amount of resources was poured into it (probably with a similar "try the fast path, abort and fall back to a slow path in the extremely rare case that someone actually is doing funky metaprogramming" approach).
And that while ignoring all the engineering that has gone into PyPy, largely ignored by the community.
There is no need to eliminate the 10% or make it 5% or whatever, people happily pay 10% overhead for convenience, but being limited to 10 threads is a showstopper.
Because of GIL, there may be at most one thread in a Python process running pure Python code. If I have a computation which takes 10% time in pure Python, 90% time in C extensions, I can only launch at most 10 threads, because 10 * 10% = 100%, and expect mostly linear scalability.
> the pure Python code in between calls into C extensions is irrelevant because you would not apply multithreading to it
No. There is a very important use case where the entire computation, driven by Python, is embarrassingly parallel and you'd want to parallize that, instead of having internal parallization in each your C extensions call. So the pure Python code in between calls into C extensions MUST BE SCALABLE. C extensions code may not launch thread at all.
In so far as it is all threaded for C and Python you can parallelize it all with one paradigm that also makes a mean dynamic web server.
None of the lisps have anything close to julia's ecology in numerical computing at least. Can't really speak to other niches though.
> People don't pick languages for language features, mostly. They pick them for their ecosystems -- the quality of libraries, compiler/runtime support, the network of humans you can ask questions of, etc.
Sure. And that's why Python is both popular and slow.
Try Hissp (https://github.com/gilch). Lisp programming and Python's ecosystem.
Just this week I failed to convince a team to migrate their ten year old maintenance mode demon to Python 3.
The difference is in what motivates getting to the other end of that transition bump and how big the bump is. That’s why it took till past 2.7’s EOL to actually get people on to 3 in a big way because they’d drag their feet if they don’t see a big enough change.
Compiled languages have it easier because they don’t need to mix source between dependencies, they just have to be ABI compatible.
The main thing is that unlike the 2 to 3 transition, they're not breaking syntax (for the most part?), which everyone experiences and has an opinion on, they're breaking rather deep down things that for the most part only the big packages rely on so most users don't experience it much at all.
The Python community consisted of tons of developers including very wealthy companies. At what point in the last few years would you even say they became “rich enough” to do the migration? Because people are STILL talking about trying to fork 2.7 into a 2.8.
I also disagree with your assertion that 3.x releases have significant breaking changes. Could you point to any specific major breaking changes between 3.x releases?
2 to 3 didn’t break syntax for most code either. It largely cleaned house on sensible API defaults.
I think it’s a larger problem with interpreted languages where all the source has to be in a single version. In that case I cant think of much.
This is how Common Lisp people can claim that the language is both performant and flexible. The performant parts and the flexible parts are more disjoint than one might expect based on the way people talk about it.
But anyways, Common Lisp does manage to give a high degree of dynamism and performance to a point that it surely can be used for any of the dynamic stuff you'd want to do in Python, while also giving the possibility of writing high performance code.
Python did not do this, and so it'll be impossible for them to offer something like common lisp perf without breaking changes, or by just introducing a whole new set of alternatives to slow builtins like class, int, call, etc.
> Because the language's semantics promise that a bunch of insane stuff can happen at any time during the running of a program, including but not limited to the fields of classes changing at any time.
You originally claimed Python is slow because of its semantics and then compare later to CL. CL has a very similar degree of dynamism and remains fast. That's what I'm saying makes for a poor comparison.
CL is a demonstration that Python, contrary to your initial claim, doesn't have to forfeit dynamism to become fast.
But not the dynamic parts remain "really" fast. Common Lisp introduced very early a lot of features to support optimizing compilers -> some of those reduce "dynamism". Code inlining (-> inline declarations), file compiler semantics, type declarations, optimization qualities (speed, compilation-speed, space, safety, debug, ...), stack allocation, tail call optimization, type inferencing, ...
Not all dynamism is the same, even if the end result can feel the same. Python has a particularly difficult brand of dynamism to deal with.
You can also treat Julia as C and recompile vtables on the fly.
I just installed Python 3.13 with pip 24.2, created a venv and installed a package - and nothing, no file was created and nothing was saved. Even if I touch requirements.txt and pyproject.toml, pip doesn't save anything about the package.
This creates a massive gap in usability of projects by people not very familiar with the languages. Node-based projects sometimes have issues because dependencies changed without respecting semver, but Python projects often can't be installed and you have no idea why without spending lots of time looking through versions.
Of course there are other package managers for Python that do this better, but pip is still the de-facto default and is often used in tutorials for new developers. Hopefully uv can improve things!
It is very fast and tracks the libraries you are using.
After years of venv/pip, I'm not going back (unless a client requires it).
So no need to mess around with brew/deadsnakes and multiple global python versions on your dev system.
This is actually an improvement over the node/nvm approach.
I think if you are comparing with what NPM does then you would have to say that native pip can do that too. It is just one command
`pip freeze > requirements.txt`
It does include everything in the venv (or in you environment in general) but if you stick to only add required things (one venv for each project) then you will get requirements.txt files
As some posters mentioned uv takes care of a lot of that and you can even pin it to a version of python.
If it’s just a one off script you can get all the dep information in the script header and uv can take care of all the venv/deps for you if you transfer the script to another machine by reading the headers in a comment section at the start of the script.
All this is based on PEPs to standardise packaging. It’s slow but moving in the right direction.
Turns out one dependency had 3 major releases in the span of a year! (Which basically confirms what I was saying, though I don't know how typical that is.)
Which is, fairly often, pinning your python version.
Last week I was trying to install snakemake via Conda, and couldn't find any way to satisfy dependencies at all, so it's not just pypi, and pip tends to be one of the more forgiving version dependency managers.
It's not just Python, trying to get npm to load the requirements has stopped me from compiling about half of the projects I've tried to build (which is not a ton of projects). And CRAN in the R universe can have similar problems as projects age.
I'm not sure it is discouraged so much as just not what people did in Python-land for a long time. It's obviously the right thing to do, it's totally doable, it's just inertia and habit that might mean it isn't done.
Pinning obviously the wrong thing, it only works if everyone does it and if everyone does it then making changes becomes very hard. The right thing is to have deterministic dependency resolution so that dependencies don't change under you.
Conda is it’s own kettle of fish especially given different channels and conda-forge which you have to remember.
Pip is nice in that you can install packages individually to get around some version conflicts. But with conda and npm and CRAN I have always found my stuck without being able to install dependencies after 15 minutes of mucking.
Its rare that somebody has left the equivalent of the output of a `pip freeze` around to document their state.
With snakemake, I abandoned conda and went with pip in a venv, without filing an issue. Perhaps it was user error from being unfamiliar with conda, but I did not have more time to spend on the issue, much less doing the research to be able to file a competent issue and follow up later on.
There are different types of dependencies, and there are different rules for them, but here's an overview of the best practices:
1. For applications, scripts and services (i.e. "executable code"), during development, pin your direct dependencies; ideally to the current major or minor version, depending how much you trust their their authors to follow SemVer. Also make sure you regularly update and retest the versions to make sure you don't miss any critical updates.
You should not explicitly pin your transitive dependencies, i.e. the dependencies of your direct dependencies -- at least unless you know specifically that certain versions will break your app (and even then it is better to provide a range than a single version).
2. For production builds of the above, lock each of your dependencies (including the transitive ones) to specific version and source. It is not really viable to do it by hand, but most packaging tools -- pip, Poetry, PDM, uv... -- will happily do that automatically for you. Unfortunately, Python still doesn't have a standard lock format, so most tools provide their own lock file; the closest thing to a standard we have at the moment is pip's requirements file [0].
Besides pinned versions, a lock file will also include the source where the packages are to be retrieved from (pip's requirements file may omit it, but it's then implicitly assumed to be PyPI); it can (and should, really) also provide hashes for the given packages, strengthening the confidence that you're downloading the correct packages.
3. Finally, when developing libraries (i.e. "redistributable code"), you should never pin your dependencies at all -- or, at most, you can specify the minimum versions that you know that work and have tested against. That is because you have no control over the environment the code will eventually be used and executed in, and arbitrary limitations like that might (and often will) prevent your users to update some other crucial dependency.
Of course, the above does not apply if you know that a certain version range will break your code. It that case you should most definitely exclude it from your specification -- but you should also update your code as soon as possible. Libraries should also clearly specify which versions of Python they support, and should be regularly tested against each of those versions; it is also recommended that the minimal supported version is regularly reviewed and increased as new versions of Python get released [1].
For more clarity on abstract vs concrete dependencies, I recommend the great article by Donald Stufft from 2013 [2]; and for understanding why top-binding (i.e. limiting the top version pin) should be avoided there is a lengthy but very detailed analysis by Henry Schreiner [3].
[0] https://pip.pypa.io/en/stable/reference/requirements-file-fo...
[1] https://devguide.python.org/versions/
That is true for all formats of lock files, by definition.
So I am not sure when this will become a problem.
How can you reasonably expect to work with any tech that breaks itself by not controlling its dependencies? You’re absolutely correct that this is probably more likely to be an issue with Python, but that’s the thing with freedom. It requires more of you.
> numpy isn't written in Python. However, there is a scalability issue: they can only drive so many threads (not 1, but not many) in a process due to GIL.
Now you have concocted this arbitrary example of why you can't use multithreading that has nothing to do with your original comment or my response.
> instead of having internal parallelization in each your C extensions call ... C extensions code may not launch thread at all.
I don't think you understood my comment - or maybe you don't understand Python multithreading. If a C extension is single threaded but releases the GIL, you can use multithreading to parallelize it in Python. e.g. `ThreadPool(processes=100)` will create 100 threads within the current Python process and it will soak all the CPUs you have -- without additional Python processes. I have done this many times with numpy, numba, vector indexes, etc.
Even for your workload, using multithreading for the GIL-free code in a hierarchical parallelization scheme would be far more efficient than naive multiprocessing.
I apologize if I can't make you understand what I said. But I still believe I said it clearly and it is simply correct.
Anyway, let me try to mansplain it again, "numpy isn't written in Python" - and numpy releases GIL. So as long as a Python code calls into numpy, the thread on which the numpy call runs can go without GIL. The GIL could be taken by another thread to run Python code. I have easily have 100 threads running in numpy without any GIL issue. BUT, they eventually needs to return to Python and retake GIL. Say, for each such thread and for every 1 second there is 0.1 seconds they need to run pure Python code (and must hold GIL). Please tell me how to scale this to >10 threads.
> Now you have concocted this arbitrary example of why you can't use multithreading that has nothing to do with your original comment or my response.
The example is not arbitrary at all. This is exactly the problem people are facing TODAY in ANY DL training written in PyTorch.
I have a Python thread, driving GPUs in an async way, it barely runs any Python code, all good. No problem.
Then, I need to load and preprocess data [1] for the GPUs to consume. I need very high velocity changing this code, so it looks like a stupid script, reads data from storage, and then does some transformation using numpy / again whatever shit I decided to call. Unfortunately, as dealing with the data is largely where magic happens in today's DL, the code spend non-trivial time (say, 10%) in pure Python code manipulating bullshit dicts in between all numpy calls.
Compared to what happens on GPUs this is pretty lightweight, this is not latency sensitive and I just need good enough throughput, there's always enough CPU cores alongside GPUs, so, ideally, I just dial up the concurrency. And then I hit the GIL wall.
> I don't think you understood my comment - or maybe you don't understand Python multithreading. If a C extension is single threaded but releases the GIL, you can use multithreading to parallelize it in Python. e.g. `ThreadPool(processes=100)` will create 100 threads within the current Python process and it will soak all the CPUs you have -- without additional Python processes. I have done this many times with numpy, numba, vector indexes, etc.
I don't think you understood what you did before and you already wasted a lot of CPUs.
I never talked about doing any SINGLE computation. Maybe you are one of those HPC gurus who care and only care about solving single very big problem instances? Otherwise I have no idea why you are even talking about hierarchical parallelization after I already said that a lot of problem is embarrassingly parallel and they are important.
[1] Why don't I simply do it once and store the result? Because that's where actual research is happening and is what a lot of experiments are about. Yeah, not model architectures, not hyper-parameters. Just how you massage your data.
> I don't think you understood what you did before and you already wasted a lot of CPUs.
No CPUs were wasted lol. You are clearly confused about how threads and processes in Python work. You also don't seem to understand hierarchical parallelization, which is simply a pattern that works well in cases where you can better maximize parallelism using combination of processes and threads.
There are probably better ways to address your preprocessing problem, but I get the impression you're one of those people only incidentally using Python out of necessity to run PyTorch jobs and frustrated or haven't yet come to the realization that you need to learn how to optimize your Python compute workload because PyTorch doesn't do everything for you automatically.
Again even if you are going to spend sometime to learn something that will have better tool for doing that like uv and poetry package managers. This is no massive amount of time. And eveb pip freeze is just one standard command and will give you portable environment be everything will be pinned in your environment. You just don't want to do everything with system global environment which is a common sense not a huge ask.
So I am not sure what is the massive amount of debugging needes for that.
You might explain that away by asking why I'd want to run these projects, but a large percentage of data science projects I've tried to run end up in this scenario, and it simply doesn't happen with npm. A command existing is not a valid replacement for good defaults, since it literally affects my ability to run these projects.
Using uv as an example[0]:
uv add "tqdm >=4.66.2,<5"
[0]https://docs.astral.sh/uv/concepts/dependencies/#project-dep...
I don't have access to uv to test that command at the moment, but that should work. uv then installs the dependency in the .venv directory in the project directory. This may include a specific version of python as well, if you pin one.
But having that lock file will allow somebody to reconstruct your particular moment in time in the future. Its just that those lock files do not exist for 99.9% of Python projects in time.
I think this is the core to much misunderstandings and arguments around this question. Some people are writing code that only they will run, on a python they've installed, on hardware they control. Others are writing code that has to work on lots of different versions of python, on lots of different hardware, and when being run in all kinds of strange scenarios. These two groups have quite different needs and don't always understand each other and the problems they face.
It is only a matter of tooling. Locking ones dependencies remains the right thing to do, even for a lib.
At which point one might directly not pin, but that's "insecure" (https://scorecard.dev/)
I can not, but I can tell you that anything AI often requires finding a proper combination of python + cuXXX + some library. And while I understand cu-implications, for some reason python version is also in this formula.
I literally have four python versions installed and removed from PATH, because if I delete 3.9-3.11, they will be needed next day again and there’s no meaningful default.
That said, 3.12 doesn't even have python312.exe in its folder. If 3.13 follows, having both of them in PATH is useless.
Sure I'm not that knowledgeable in this topic (in python). But you're telling me they go to the lengths of supporting e.g. 3.9-3.11.2, but out of lazyness won't just compile it to 3.12?
I can only hypothesize that 3.9-3.xxx had the same ABI and they don't support multiple ABIs out of principle, but that sounds like a very strange idea.
Quite a bit of that could be fixed by automated tooling, but not all of it, and the testing burden was huge, which meant a lot of smaller packages did not convert very quickly and there were ripple effects.
Regarding breakage in 3.x, all I know is that I recall several times where I did a linux system update (rolling release), and that updated my Python to a newly released version which broke various things in my system. I'm pretty sure one of these was v3.10, but I forget which others caused me problems which I could only solve by pinning Python to an older release.
It's entirely possible though that no actual APIs were broken and that this was just accidentaly bugs in the release, or the packages were being naughty and relying on internals they shouldn't have relied on or something else.
Python isn’t fully ABI stable (though it’s improved greatly) so you can’t just intermix compiled dependencies between different versions of Python.
This is true for many packages in your distro as well.
- standard library modules removed
- zip error handling behaves differently
- changes to collections module
- new reserved keywords (async, await, etc.)
You can argue how big of a deal it is or isn't, but there were definitely breakages that violate semantic versioning
Almost every major noteworthy Python package uses the ABI, so instability there is going to constantly be felt ecosystem wide.
Python 3 made no sense from a cost/risk perspective for teams with a lot working and mostly finished Python 2 code.
https://docs.python.org/3/whatsnew/3.3.html#porting-to-pytho...
> Hash randomization is enabled by default. Set the PYTHONHASHSEED environment variable to 0 to disable hash randomization. See also the object.__hash__() method.
https://docs.python.org/3/whatsnew/3.4.html#porting-to-pytho...
> The deprecated urllib.request.Request getter and setter methods add_data, has_data, get_data, get_type, get_host, get_selector, set_proxy, get_origin_req_host, and is_unverifiable have been removed (use direct attribute access instead).
https://docs.python.org/3/whatsnew/3.5.html#porting-to-pytho...
https://docs.python.org/3/whatsnew/3.6.html#removed
> All optional arguments of the dump(), dumps(), load() and loads() functions and JSONEncoder and JSONDecoder class constructors in the json module are now keyword-only. (Contributed by Serhiy Storchaka in bpo-18726.)
https://docs.python.org/3/whatsnew/3.7.html#api-and-feature-...
> Removed support of the exclude argument in tarfile.TarFile.add(). It was deprecated in Python 2.7 and 3.2. Use the filter argument instead.
https://docs.python.org/3/whatsnew/3.8.html#api-and-feature-...
> The function time.clock() has been removed, after having been deprecated since Python 3.3: use time.perf_counter() or time.process_time() instead, depending on your requirements, to have well-defined behavior. (Contributed by Matthias Bussonnier in bpo-36895.)
https://docs.python.org/3/whatsnew/3.9.html#removed
> array.array: tostring() and fromstring() methods have been removed. They were aliases to tobytes() and frombytes(), deprecated since Python 3.2. (Contributed by Victor Stinner in bpo-38916.)
> Methods getchildren() and getiterator() of classes ElementTree and Element in the ElementTree module have been removed. They were deprecated in Python 3.2. Use iter(x) or list(x) instead of x.getchildren() and x.iter() or list(x.iter()) instead of x.getiterator(). (Contributed by Serhiy Storchaka in bpo-36543.)
> The encoding parameter of json.loads() has been removed. As of Python 3.1, it was deprecated and ignored; using it has emitted a DeprecationWarning since Python 3.8. (Contributed by Inada Naoki in bpo-39377)
> The asyncio.Task.current_task() and asyncio.Task.all_tasks() have been removed. They were deprecated since Python 3.7 and you can use asyncio.current_task() and asyncio.all_tasks() instead. (Contributed by Rémi Lapeyre in bpo-40967)
> The unescape() method in the html.parser.HTMLParser class has been removed (it was deprecated since Python 3.4). html.unescape() should be used for converting character references to the corresponding unicode characters.
https://docs.python.org/3/whatsnew/3.10.html#removed
But I still find the high level of instability in Python land rather disturbing, and I would be unhappy if the languages I used constantly did these sorts of breaking changes
I'm even more extreme in that I also think the ABI instability is bad. Even though Python gives no guarantee of its stability, it's used by so many people it seems like a bad thing to constantly break and it probably should be stabilized.
The model here isn't something being actively maintained but some 100k utility that has been quietly working for five or ten years and has receded out of the awareness of everyone. Touching it is risky and it's working. I don't want to change it. SSL deprecations are bad enough but at least justifyable but removing things that have been deprecated is not.