Faster Parallel Python Without Python Multiprocessing(towardsdatascience.com) |
Faster Parallel Python Without Python Multiprocessing(towardsdatascience.com) |
The beauty of python for someone like me is that I can get my job done without actually having to do it. I free up my own time to leverage more of my creativity and have a multiplicative effect on my productivity.
The reason I bring all of this up is that so many of the examples for advanced libraries I see are geared towards seasoned software engineers. The examples include false arrays of data so that you can "just see" how to use it. I don't think anyone realizes how confusing this is to the guy who is a restaurant manager, or the gal who is a researcher that just needs to know how to make this work for them.
If it's an image, how about putting image = cv2.imread("C:\image.jpg") or whatever?
Anyway, a bit of a rant but there are people who are very thankful that smart people in this world like yourself write libraries we can use to make our daily lives better. Including example code that is stupid simple would make me so much happier.
I have used ray (a year ago) in one of the advertised basic applications: parallelising the environments for RL. It was unusable back then, as it was clogging up the memory.
The plasma store which is backend for arrow was never cleaned which made the computation stop after 3 hours
Here’s the issue:
https://github.com/ray-project/ray/issues/2128
Or perhaps this has been fixed already?
It's nice to have the option to speed up python in certain cases though.
https://github.com/stackless-dev/stackless/wiki
It got less attention than pypy, but might be more pragmatic wrt parallel perf. There's also some work in pypy to remove the GIL, but not sure if there's been any news on that lately.
Python has a lot of merit as an easy to use language, a modern BASIC, but "strongly typed" is marketing that confuses rather than enlightens (to quote Prof. Harper). The term "strongly-tagged" is perhaps more apt, but sadly that ship has sailed.
Strongly typed used to mean safe, as in you can't add strings to ints or reinterpret values. As opposed to C and others where anything goes. I think that's a useful distinction.
Imagine all that lovely energy poured into understanding the compromises involved in designing realistic type systems. But its messy, and you don't become famous from dealing with messy problems.
I can't think of a static language that doesn't support a dynamic type, it's just that most require explicit casts or unpacking to use with existing typed interfaces, e.g. "Object" in Java.
> Strongly typed used to mean safe
It still does, it's just that type theory doesn't distinguish between a runtime failure with a nice error message versus a segfault, both are programs that "go wrong".
I would be in favour of "strong dynamic types" instead of "strongly typed".
Yes, you can write Java using nothing but Object. And no, the experience won't be comparable to using a dynamically typed language.
Map is not terrain, if theory doesn't match reality then changing the former is the only thing that makes sense.
Categories are generalizations, making them more specific weakens them. A RISC approach works better from my experience, where categories are kept trivial and members belong to multiple categories.
Something like a simplified nim/f# with type inference. So it feels dynamic as you code but isn't really.
Or yes if the normal way to run python was with a top notch jit that would be a big plus.
IMHO, Julia is that language: https://julialang.org/
That's a pretty big "if". Unfortunately making python a lot faster is difficult because python is such a dynamic language. The pypy folks have done their best, with a respectable amount of success.
The 10x figure I was quoting was from the comment I was responding to. I do think people need to look at what was accomplished by the Strongtalk team and various other projects dealing with dynamic languages including some Lisp implementations. I'm pretty sure Python can be a lot faster, but the legacy code is going to be a big, big problem.
Not all dynamic languages are born the same.
> I'm pretty sure Python can be a lot faster
You might like you join the PyPy team then and find out.
Frankly, I honor the Python folks and all their achievements, but I really dislike the language. I mostly am researching another paradigm.