Julia Computing raises $24M Series A(hpcwire.com) |
Julia Computing raises $24M Series A(hpcwire.com) |
I tried Julia for the first time last week and it was great.
I've been playing with the idea of defining a hash function for lists that can be composed with other hashes to find the hash of the concatenation of the lists. I tried to do this with matrix multiplication of the hash of each list item, but with integer mod 256 elements random matrices are very likely to be singular and after enough multiplications degenerates to the zero matrix. However, with finite field (aka Galois fields) elements, such a matrix is much more likely to be invertible and therefore not degenerate. But I don't really know anything about finite fields, so how should I approach it? Here's where Julia comes in: with some help I was able to combine two libraries, LinearAlgebraX.jl which has functions for matrices with exact elements, with GaloisFields.jl which implements many types of GF, and wrote up a working demo implementation of this "list hash" idea in a Pluto.jl [2] notebook and published it [0] (and a question on SO [1]) after a few days without having any Julia experience at all. Julia seems pretty approachable, has great libraries, and is very powerful (I was even able to do a simple multithreaded implementation in 5 lines).
[0]: https://blog.infogulch.com/2021/07/15/Merklist-GF.html
[1]: https://crypto.stackexchange.com/questions/92139/using-rando...
I would never use it for producing software meant for distribution, but people mainly hate on it because it's 'cool', without realising that it excels at what it does. I fucking love matlab.
I am a python developer who has dabbled with Julia but it never stuck for me.
I think Julia was built by academics for other academics running innovative high performance computing tasks. It excels at the intersection of 1) big data, so speed is important, and 2) innovative code, so you can't just use someone else's C package. Indeed, Julia's biggest successful applications outside academica closely resemble an academic HPC project (eg Pumas). I think it will continue to have success in that niche. And that's not a small niche! Maybe it's enough to support a billion dollar company.
But most of us in industry are not in that niche. Most companies are not dealing with truly big data, on our scale, it is cheaper to expand the cluster than it is to rewrite everything in Julia. Most who ARE dealing with truly big data, do not need innovative code; basic summary statistics and logistic regression will be good enough, or maybe some cloud provider's prepackaged turn key neural nets scale out training system if they want to do something fancy.
I think for Julia to have an impact outside of academia (and academia-like things in industry) it will need to develop killer app packages. The next PyTorch needs to be written in Julia. Will that happen? Maybe! I hope so! The world would be better off with more cool data science packages.
But I think the sales pitch of "it's like Pandas and scikit but faster!" is not going to win many converts. So is Jax, Numba, Dask, Ray, Pachyderm, and the many other attempts within the Python community of scaling and speeding Python, that require much less work and expense on my part for the same outcome.
Again, congrats to the team, I will continue to follow Julia closely, and I'm excited to see what innovative capabilities come out of the Julia ecosystem for data scientists like me.
1. https://julialang.org/blog/2019/02/julia-entities/#julia_com...
Happy to be wrong here.
What products/services does Julia Computing sell to justify that Series A? The article doesn't mention anything.
Although the company website lists some "products," there are no price tags or subscription plans attached to anything.
And even if there are revenue-generating products/services on the horizon, how will the company protect itself from smaller, more nimble competitors that don't have a platform obligation to fulfill?
How is this not another Docker? Don't get me wrong, both Julia and Docker are amazing, but have we entered the phase of the VC-funded deliberate non-profit?
That's the entire second paragraph of the article. JuliaHub is a paid cloud computing service for running Julia code and JuliaSim/JuliaSPICE/Pumas are paid domain-specific modeling and simulation products. See also some of the other comments here from Keno[1] and Chris Rackauckas[2]:
What we've noticed is the vast majority of the time it's the data scientist's code that's slow not the actual ML model bit. So allowing them to write very performant code with a dumpy-like syntax and not have to deal with painfully slow pandas, lack of true parallelism, etc. would be a true game changer for ML in industry.
There are a number of components here which enable (what I would call) the expression of more advanced models using Julia's nice compositional properties.
Flux.jl is of course what most people would think of here (one of Julia's deep learning frameworks). But the reality behind Flux.jl is that it is just Julia code -- nothing too fancy.
There's ongoing work for AD in several directions -- including a Julia interface to Enzyme: https://github.com/wsmoses/Enzyme.jl
Also, a new AD system which Keno (who you'll see comment below or above) has been working on -- see Diffractor.jl on the JuliaCon schedule (for example).
Long story short -- there's quite a lot of work going on.
It may not seem like there is a "unified" package -- but that's because packages compose so well together in Julia, there's really no need for that.
The reason for the lag is that Julia has been focusing on general composable compiler, codegen and metaprogramming infrastructure which isn't domain specific, whereas pytorch and friends has been putting lots of dev money into c++ ML focused optimizers.
Once the new compiler stuff is in place, it would be relatively trivial to write such optimizations, in user space, in pure Julia. Then exceeding that would be fairly simple also, plus things like static analysis of array shapes
I'm wondering if most statisticians or researchers deal with data big enough that massively better performance would be enough motivation to switch.
And here Julia is a complete Godsend, since it makes it a joy to implement things from the bottom up.
Sure, you also need a language that already has dataframe libraries, plotting, editor support et cetera, and Julia is lacking behind Python and R in these areas. But Julia's getting there, and at the end of the day, it's a relatively low number of packages that are must-haves.
I remember when they hit $10M and assumed they had continued to grow somewhat. But I didn't realized we'd blown well past $20M — when did that happen?
Most of these arguments are re-hashed on each new Julia post here. A few comments:
For most Julia users, any supposed rift between Python and Julia is not really a big deal -- we can just use PyCall.jl, a package whose interop I've personally used many times to wrap existing Python packages -- and supports native interop with Julia <-> NumPy arrays. Wrapping C is similarly easy -- in fact, easier than "more advanced" languages like Haskell -- whose C FFI only supports passing opaque references over the line.
Ultimately, when arguments between languages in this space arise -- the long term question is the upfront cost, and the maintenance cost for a team. Despite the fact that Julia provides an excellent suite of wrapper functionality, I'm aware that introducing new Julia code into an existing team space suffers from the above issues.
I'm incredibly biased, but I will state: maintaining Julia code is infinitely easier than other uni-typed languages. I've had to learn medium-sized Python codebases, and it is a nightmare compared to a typed language. This total guessing game about how things flow, etc. It really is shocking. Additionally, multiple dispatch is one of those idioms where, one you learn about it, you can't really imagine how you did things before. I'm aware of equivalences between the set of language features (including type classes, multi-method dispatch, protocols or interfaces, etc).
Ultimately, the Julia code I write feels (totally subjectively) the most elegant of the languages I've used (including Rust, Haskell, Python, C, and Zig). Normally I go exploring these other languages for ideas -- and there are absolutely winners in this group -- but I usually come crawling back to Julia for its implementation of multiple dispatch. Some of these languages support abstractions which are strictly equivalent to static multi-method dispatch -- which I enjoy -- but I also really enjoy Julia's dynamism. And the compiler team is working to modularize aspects of the compiler so that deeper characteristics of Julia (even type inference and optimization) can be configured by library developers for advanced applications (like AD, for example). The notion of a modular JIT compiler is quite exciting to me.
Other common comments: time to first plot, no native compilation to binary, etc are being worked on. Especially the latter with new compiler work (which was ongoing before the new compielr work) -- seems feasible within a few quarter's time.
Stream 1: Build amazing products for particular domains, charge license fees
Stream 2: Build a great SaaS platform for running Julia, charge for compute
Since all of our domain products are built in Julia and often involve significant compute cost for their intended application, hopefully both at the same time :).
It's a bit less of a pure language / infra play and more a product play. Ie, docker / containers was almost a pure infra play in the end. These guys make actual things you can use.
The later sells better into business I think and is less likely to be competed against. Google / AWS et al are generally pretty quick to compete on the infra play level.
hopefully will soon be a dominant force in this direction
Is the Julia business model similar to Redhat or Canonical where they sell consulting services?
Such as:
- the first Julia IDE Juni now depreciated.
- On premise package server
- A wrapper over AWS called JuliaRun that has a nice web interface
- paid "we make a core developer stare at RR traces of your problems"
- FDA approved software for drug development and pharmacokinetics as https://juliacomputing.com/products/pumas/
It is not the programming language that is the product.
Well, I still might use Julia personally in the future, but my days of contributing to the community are over.
The julia community is thankless, and the people at Julia computing are mean spirited and they think it's acceptable to harm people having good intentions (like me), having contributed thousands of hours of free time to help people on their forum and releasing my own packages used by thousands of people.
Julia Computing just raised a bunch of money, but they are just pocketing it for themselves while they stomp their foot on people they perceive as unworthy.
Yea, it's a good language, but the people behind it are very selfish and harmful to the free software community.
The Two Language Problem makes this more likely than one might think. Those high level python packages that plaster over python's many mediocrities have to be written and maintained by someone, and while extremistan bears the brunt of the pain and has done a remarkable job shielding mediocristan, it's extremistan that gets to decide which language to use for the next killer app.
Of course, python has more inertia than god, so it won't go quietly.
I think this is a good deciding factor. Not just for “big data”. In my experience, with its combination of flexibility and raw speed, Julia makes implementing new algorithms (from scratch) a breezy experience. The strong research/academic presence in the community also helps towards encouraging decent Julia libraries for a lot of cutting edge work.
So if you are working in an area where that could make a significant difference, it’s an excellent reason to use Julia.
> will need to develop killer app packages. The next PyTorch needs to be written in Julia. Will that happen?
If enough cutting-edge work happens in Julia, it’s likely that a few great tools/platforms will emerge from that. We’re already seeing that in the scientific modeling ecosystem (as an example), with Differential Equations infrastructure and PumasAI.
On the other hand it makes function discovery almost impossible [1]. Combined with the way how 'using' keyword exports a predefined subset of functions, this makes the language doomed from larger adoption outside of academia at least as long as there is no superb autocompletion and IDE support.
[1] https://discourse.julialang.org/t/my-mental-load-using-julia...
Why's that? What features or lack thereof of Julia contribute to that experience?
If a major company would pick up Flux.jl and fill out the ecosystem that would be AMAZING.
PyTorch and Tensorflow feel like duct tape and chewing gum all day, every day.
Yes, and even with languages that have these abstractions, they aren't as pervasive as in Julia. Ex. There's a fundamental difference between typclasses and functions in haskell. In julia, there's no such distinction and code is more generic
1 https://www.hpcwire.com/off-the-wire/julia-joins-petaflop-cl...
2 https://arstechnica.com/science/2020/10/the-unreasonable-eff...
(Not a hater of Julia at all, very much think it's a cool language and an increasingly vibrant ecosystem and have been consistently impressed when Julia devs have spoke at events I've attended)
If Julia can provide both the REPL/debugging experience of a language like Python or MATLAB with a fast enough JIT to use in production it would be an enormous boon to productivity and robustness.
There are a few limiting factors but I don't think they're absolute.
Then there's Matlab, Mathematica, and they are also pretty good but they're closed source/proprietary, so their ecosystem is mostly limited and driven by commercial interests. Nothing wrong with that intrinsically and they're all widely used but it's one way Julia differentiates itself, by making the language open and making money through services.
But indeed, Julia Computing differentiates itself from something like MATLAB or Mathematica by leveraging a strong open source community on which these products are developed. These products add a lot of the details that are generally lacking in the open source space, such as strong adherents to file formats, regulatory compliance and validation, GUIs, etc. which are required to take such a product from "this guy can use it" to a fully marketable product usable by non-computational scientists. I will elaborate a bit more on this at JuliaCon next week in my talk on the release of JuliaSim.
It also pretty much solved my version of the two-language problem, but that means different things to different people so ymmv.
Such a question seems sort of in bad faith (or loaded), since the selling points of Julia have been hammered time and again on HN and elsewhere, and are prominent on its website. It's a 1 minute search to find them, and if someone is already aware that there's this thing called Julia to the point that they think it's made to be "a big deal", they surely have seen them.
So, what could the answer to the question above be? Some objective number that shows Julia is 25.6% better than Java or Rust or R or whatever?
But first, who said it's a "big deal"? It's just a language that has some development action, seems some adoption, and secured a modest fundng for its company. That's not some earth shattering hype (if you want to see that, try to read about when Java was introduced. Or, to a much lesser degree, Ada, for that matter).
You use a language because you've evaluated it for your needs and agree with the benefits and tradeoffs.
Julia is high level and at the same time very fast for numerical computing allowing you to keep a clean codebase that's not a mix of C, C++, Fortran and your "real" language, while still getting most of the speed and easy parallelization. It also has special focus on support for that, for data science, statistics, and science in general. It's also well designed.
On the other hand, it has slow startup/load times, incomplete documentation, smaller ecosystem, and several smaller usability issues.
That’s not to say that there are no disadvantages to Julia. I personally see Julia as a beefed up, new and improved R.
However, Julia is perfect if you write mathematical/physical/engineering simulations and data analysis codes, which is my typical use case. Its support for multiple dispatch and custom operators lets you to write very efficient code without sacrificing readability, which is a big plus. Support for HPC computing is very good too.
That's the case now, because Julia made a design decision to focus on extreme composability, dynamism, generic codegen etc which involved compiler tradeoffs...but it's not inherent to the langauge.
For scripting, interpreted Julia is coming. For executables, small binary compilation is as well...particularly bullish on this given the new funding
That's a bit of an understatement. It's about as fast as C and Rust (ignoring JIT compilation time).
designers seems to have a good amount of PLT knowledge and made good foundations
https://twitter.com/evalparse/status/1416039770833096706
And https://github.com/JuliaPlots/AlgebraOfGraphics.jl >>> GoG
It depends on the field, there’re hundreds of biological publications each month that just use existing software. And if I’m developing a new tool for single-cell analysis, it’s either going to be interoperable with Seurat or Bioconductor tools.
Citation for this? Julia has had a built-in interpretted since 1.0, in 2017 use `--compile=min`, or `--compile=none` to make use of it. And JuliaInterpretter.jl has been working since 2018. Both are very slow -- slower than adding in the compile time for most applications. As I understand it, this is because a number of things in how the language works are predicated on having a optimizing JIT compiler. As is how the standard library and basically all packages are written.
Julia is going to over time become nicer for scripting, just because of various improvements. In particular, I put more hope on caching native code than on any new interpreter.
If you're curious: https://github.com/aviatesk/JET.jl
This may seem more about performance (than IDE development) but Shuhei is one of the driving contributors behind developing the capabilities to use compiler capabilities for IDE integration -- and indeed JET.jl contains the kernel of a number of these capabilities.
https://juliahub.com/lp/pricing
I see Free, Pay as You Go, and Contact Sales.
Clicking the middle option, I see that I must be logged in to view anything.
What you want to do with this information is up to you, but it doesn't look good from the customer side. And from the perspective of someone trying to figure out if Julia Computing has a future as a money-making enterprise, it looks iffy at best.
If Julia had its own lean realtime database implementation, then I can see it becomming a killer language for finance. JuliaDB/OnlineStats is probably 60% of the way there.
YMMV, but I find it is fantastic in this use case. And I don't have to worry about semantic space.
Diffeq are one of those things that Julia is a particularly good choice for that benefits strongly from its types and what not. Give it a shot if you do a lot of this.
Julia syntax resembles matlab more than python or R.
However, Numba is quite limited because it only works well for mathematical code (it is not able to apply its optimizations to complex objects, like lists of dictionaries), while on the other side Julia's compiler applies its optimizations to everything.
[1] https://discourse.julialang.org/t/comparing-python-julia-and...
* Julia can do loop fusion when broadcasting, while numpy can't, meaning numpy uses a lot more memory during complex operations. (Numba can handle loop fusion, but it's generally much more restrictive.)
* A lot of code in real applications is glue code in Python, which is slow. I've literally found in some applications that <5% of the time was spent in numpy code, despite that being 90% of the code.
That said, if your code is mostly in numba with no pure python glue code (not just numpy), you probably won't see much of a difference.
EDIT: And last time I checked, Numpy only parallelizes calls to supplied linear algebra routines, and only if you have the right library installed. A simple vector arithmetic operation like a + b will execute on one core only.
Most people use a ton of numpy and scipy. It turns out that phrasing things as array operations with numpy operators is quite natural in this field, including for things like galaxy merger simulations.
I work, in particular, on asteroid detection and orbit simulation, and it's all pretty much Python.
You can get very far with these approaches I python, but having these at the language level just has more potential for optimization and less friction.
The debugability of numba code is very limited and code coverage does ot work at all.
Having a high level language that has scientific use at its core is just great.
Python has the maturity and community size on its side, but Jul is catching up on that quickly.
“It turns out that phrasing things as array operations with numpy operators is quite natural in this field”
But if A and B are numpy arrays, then A + B will calculate the elementwise sum on a single core only, correct? It will vectorize, but not parallelize. All large-scale computation is multi-core.
That's basically the answer.
1.2 is pretty ancient. Current, or even recent, versions of Julia have a fraction of the startup time (https://lwn.net/Articles/856819/). Package management has been refined further, as well.
We absolutely cannot upgrade Julia version right now, dozens of repos full of complicated scientific code. Management doesn’t care as far as it barely runs. I don’t think it’s fair to blame Julia for it but it just shows how much more it needs to go. That should be looked at as a positive thing.
I have one more complain to Julia community - please don’t be too defensive. Accept and embrace criticisms and turn that into a propellant for improving the language. I’ve seen a cult-like behavior in Julia community that is borderline toxic. Sorry, it’s the truth and needs to be said. Speed isn’t everything and people are magnetized by the benchmarks on the Julia website, especially non-software engineers.
Instead of Pip, virtualenv, conda, etc etc there's one package manager that resolves and installs native and binary dependencies, ensures reproducibility with human readable project files, is accessible from the REPL etc.
You can get an entire GPU based DL stack up and running on a windows machine in under 30 min, with a few keystrokes and no special containers or anything like that. Don't even have to install cuda toolkit. It's a dream, and I've heard the same from recent python converts
A lot of problems are fixable with time and money. Maybe the Series A will help!
But some problems might be related to Julia's design choices. One thing I really missed in Julia is Object Oriented Programming as a first class paradigm. (Yes I know you can cobble together an approximation with structs.)
OOP gets a lot of hate these days. Mostly deserved. But in some large complex projects it's absolutely the right abstraction. I've used OOP in several Python projects. Most of the big Python DS/ML packages use OOP.
Maybe you think PyTorch, SciKit, etc are all wrong, and would be better off with a more functional style. I know it's fashionable in some circles to make fun of the "model.fit(X,Y); model.predict(new_data)" style of programming but it works well for us busy professionals just trying to ship features.
I don't think Julia is wrong for enforcing a more functional style. It probably makes it easier for the compiler to generate great performance.
But Python has achieved its success because of its philosophy of being the "second best tool for every job" and that requires a more pragmatic, multiparadigm approach.
Julia is object oriented in a broad sense, it just uses multiple dispatch which is strictly more expressive than single dispatch, so doesn't make sense to have dot notation for calling methods because types don't own methods.
For giving up some facility in function discover, you get speed, composability, generic code...and a net gain in usability because you can have one array abstraction for GPUs, CPUs etc etc, which is just an instance of having common verbs across the ecosystem (enabled by MD). Instead of everyone having their own table type or stats package, you have Tables.jl or Statsbase.jl that packages can plug into and extend without the issues inherent in subclassing, monkeypatching etc.
This is a much better, more powerful and pleasant experience
Closing the gap in Method discovery will simply require a language feature with tooling integration, where you write the types and then tab to get the functions. There's already an open issue/PR for this
Julia peeps would tell you that the multiple dispatch used by Julia is a generalization of OOP. And that they like multiple dispatch
as opposed to Python and JVM traces? LOL
Also, python has been around for alot longer. So perhaps that Python has "everything" is a feature of time and user base.
Maybe Julia will grow to have a large enough user-base to have those things.
Gosh, that's really old and it's not even an LTS version. I fear many of your woes stem from that. Julia 1.6 has huge improvements, and the community has rallied around VS Code now that Atom seems to be dying.
It really shouldn't be too bad to update.
Python having a broader ecosystem is a very good point, but I've found pycall to be very helpful
The Julia-for-astronomy community is just microscopic right now, so it's hard to find useful libraries. Nothing comes close to, say, Astropy[0].
I'm not a huge fan of the current numpy stack for scientific code. I just don't think anyone should get too carried away and claim that Julia is taking the entire scientific world by storm. I don't know anyone in my department who has even looked at it seriously.
Thanks for writing this. I think it is an important concept for getting started with Julia. When I tried Julia, I was initially confused and concerned about the subtyping hierarchy, which as far as I understand is undocumented. "Apart from a partial description in prose in Bezanson [2015], the only specification of subtyping is 2,800 lines of heavily optimized, undocumented C code." [0]. Assurance that users can safely ignore the subtyping hierarchy if we maintain semantic equivalence between methods, and that this actually works out in practice, makes it easier to commit to using the language.
For a negative example of this, C++ introduced function overloading — which is a static and therefore broken version of multiple dispatch (that calls the wrong “methods” based only on static type information). They then immediately decided to abuse the bitshift operators for I/O — in the standard library, no less. (There are other abuses of overloading but this is the most famous one.) So that didn’t exactly create a culture where people respect the semantic meaning of overloaded functions. As a result, C++ has given function overloading a really horrible reputation. Partly because it is likely to be not actually so what you want because it’s static rather than based on the actual types of arguments, but even more so because there’s no culture of semantic consistency in C++, starting from the standard library itself — you cannot trust anyone to respect the meaning of anything, not even the language authors.
In Julia, on the other hand, we’ve always been very strict about this: don’t add a method to a function unless it means the same thing. We would never dream of using bitshift operators to do I/O. Since meaning is the level where human thinking operates, this makes it reasonable to write generic code that works the way you meant it to: you can call `+` on two things and just trust that whoever has implemented `+` for those objects didn’t decide to make it append a value to an array or something wonky like that. Not that people haven’t proposed that kind of thing, but because of the culture, it gets shot down in the standard library and elsewhere in the ecosystem.
But yeah, it’s hard to see how this would happen because there is nothing technical preventing the same kinds of abuses that are rampant in C++, it’s just the invisible but extremely force of culture.
But this only concerns when I'm writing the code myself. If I read some code and I see a few nested function calls, there's a combinatorial explosion of possible types that gives me vertigo.
> complex and rational numbers can be constructed using anything that has arithmetic defined.
seriously? this does not seem right, it cannot be like that. If I build a complex number out of complex numbers, I expect a regular complex number, not a "doubly complex" number with complex coefficients, akin to a quaternion. Or do you? There is surely some hidden dirty magic to avoid that case.
help?> Complex
search: Complex complex ComplexF64 ComplexF32 ComplexF16 completecases
Complex{T<:Real} <: Number
Complex number type with real and imaginary part of type T.This still seems like an overstatement, but maybe it depends on what you mean by "most demanding level." I work on systems for the Rubin Observatory, which is going to be the largest astronomical survey by a lot. There's a bunch of C++ certainly, but heaps of Python. For example, catalog simulation (https://www.lsst.org/scientists/simulations/catsim) is pretty much entirely in Python.
Take a look at `lsst/imsim`, for example, from the Dark Energy collaboration at LSST: https://github.com/LSSTDESC/imSim.
Maybe this isn't the "most demanding" but I don't really know why.
> But if A and B are numpy arrays, then A + B will calculate the elementwise sum on a single core only, correct? It will vectorize, but not parallelize.
That's correct, but numba will parallelize the computation for you (https://numba.pydata.org/numba-doc/latest/user/parallel.html). It's pretty common to use numba's parallelization when relevant.
EDIT: Yes, numba is impressive.
Benchmarks: https://www.ritchievink.com/blog/2021/02/28/i-wrote-one-of-t...
And that's the killer feature of Julia. It is easier to micro-optimize Julia code than any other language, static or dynamic. Meaning if Julia is not best-in-class in a certain algorithm, it will soon.
Julia's DF library is generic and allows user defined ops and types. You can put in GPU vectors, distributed vectors, custom number types etc. Julia optimizes all this stuff.
data.frame is just a giant chunk of c (c++) code that one must interact with in very specific ways
These features aren't of interest to practicing statisticians, which the parent comment was talking about.
> data.frame is just a giant chunk of c (c++) code that one must interact with in very specific ways
I don't understand this criticism: yes, data.table has an API.
DataFrames.jl is very rapidly catching up and starting to surpass it. After hitting a stable v1.0 they've begun focusing on performance and those benchmarks have changed significantly over the past three months. Here's the live view: https://h2oai.github.io/db-benchmark/
When the trick to writing fast R code is to rely on C as much as possible, that feels less compelling.
Differential equation solvers that need to take a very custom function that works on fancy R/Python objects is another example of clumsiness in these drop-to-C-for-speed languages. It works and as a performance-nerd I enjoy writing such code, but it is clumsy.
That type interoperability is trivial in Julia.
The only difficulty with Rcpp-based R packages is you have to ensure the target system can compile the code, which means having a suitable compiler available.
For instance, I imagine there is an R library that makes it easy to automatically run R code on a GPU. Can that library also work with Rcpp functions?
I am very surprised by this. Given how R is extremely dynamic. and has things like lazy-evaluation, that you can rewrite before it is called with substitute. Which I am sure some packages are using in scary and beautiful ways.
I've been impressed with Julia, but it's hard to beat 25 years of package development.
In other words, you can (empirically) get a lot done that way, but there is always friction.
In my case I went to deploy on a musl system and things with the two language just were a pain to get up and running.
Conversely, everything that was native python ran fine in a musl based python container.
Your native python code just moves also nicely between windows / linux / etc
It's pretty convenient for things like uncertainty propagation and data cleaning...all things statisticians should care about.
>I don't understand this criticism: yes, data.table has an API
A relatively limited API, walled off from the rest of the language.
I think all languages have this dynamic...I've seen it with python and R. To some extent it's fed by what we perceive as criticisms from people defending their favorite incumbent language with arguments that aren't at all informed- such as a focus on speed and how numba achieves parity there.
In the same vein, I and many Julia users are enthusiastic precisely because of thing other than speed, such as the type system, abstractions, differentiability and other things that make programming more joyful, fluid and productive.
Agree though, that we could always improve on acceptance of criticism.
I expect it to continue to improve; note that it's starting to be the fastest implementation on some of the groupby benchmarks.
1. https://discourse.julialang.org/t/release-announcements-for-...
2. https://discourse.julialang.org/t/the-state-of-dataframes-jl...
Consider if we want to run many different types of models. Logistic regression, gradient boosting, NNs, etc. We want the ability to easily plug in any type of model into our existing code base. That's why model.fit(X,Y) is attractive. I just need to change "model = LogisticRegressionModel" to "model = GradientBoostingModel" and the rest of the code should still Just Work. This is a big part of SciKit's appeal.
But all these different models have very different training loops. So with "fit!(model,X,Y)" I need to make sure I am calling the compatible "fit" function that corresponds to my model type.
You might now say "Ah! Multiple dispatch handles this for you. The 'fit' function can detect the type of its 'model' argument and dispatch execution to the right training loop sub function." And I suppose that's theoretically correct. But in practice I think it's worse.
It should be the responsibility of the developer of "model" to select the "fit" algorithm appropriate for "model." (They don't have to implement it, but they do have to import the right one.) The developer of "fit" should not be responsible for handling every possible "model" type. You could have the developer of "model" override / extend the definition of "fit" but that opens up its own can of worms.
So is it possible to do the same thing with "fit!(model,X,Y)"? Yes of course it is. It's possible to do anything with any turing complete language. The point is, which system provides the best developer ergonomics via the right abstractions? I would argue, in many cases, including this one, it's useful to be able to bundle functions and state, even if that is in theory "less flexible" than pure functions, because sometimes programming is easier with less flexibility.
>It should be the responsibility of the developer of "model" to select the "fit" algorithm appropriate for "model." (They don't have to implement it, but they do have to import the right one.) The developer of "fit" should not be responsible for handling every possible "model" type. You could have the developer of "model" override / extend the definition of "fit" but that opens up its own can of worms.
It's really the same thing as python, just better...I don't see the distinction you are drawing.
In python you have a base class with default behavior. You can subclass that and inherit or override.
Julia has abstract types with interfaces...instead of relying on implementation details like fields, you provide functions so that more types of models can work even if they don't have that one specific field. Otherwise everything is the same where it counts,- you can compose, inherit and override. Even better, you can work with multiple models and types of data, inheriting where you see fit.
I don't see any benefit to python's restrictions here, either in ease of use or in expressiveness.
For all intents and purposes it's a strict superset.
Even better, you can use macros and traits to group different type trees together.
https://www.stochasticlifestyle.com/type-dispatch-design-pos...
These seem to be in contradiction:
>It should be the responsibility of the developer of "model" to select the "fit" algorithm appropriate for "model.
>You could have the developer of "model" override / extend the definition of "fit" but that opens up its own can of worms.
It's the same in python, either you inherit Fit or you can override it. What's the difference with Julia?
Except in julia all types and functions have a multiple dispatch, parametric type and possible trait lattice of things you can override, customize and compose, so that even if the model author has to override fit, they can do it using small composable building blocks.
I think abstract types are a brittle solution. The "can of worms" I alluded to is something like this: Library TensorFlow implements model "nn" and Library PyTorch also implements model "nn" and they both want to override "fit" to handle the new type "nn"... Good luck combining them in the same codebase. This problem is less pronounced in OOP where each development team controls their own method. Julia devs can solve this by having every developer of every "fit" function and every developer of every "model" struct agree beforehand on a common abstraction, but that's an expensive, brittle solution that hurts innovation velocity.
I think the closest I can do in Julia via pure structs is for the developer to define and expose their preferred fit function as a variable in the struct, something like "fit = model['fit_function']; fit(model,X,Y)" but that introduces a boilerplate tax with every method I want to call (fit, predict, score, cross validate, hyperpameter search, etc). (EDIT: indeed, I think this is pretty much what MLJ is doing, having each model developer expose a struct with a "fit" and "predict" function, and using the @load macro to automate the above boilerplate to put the right version of "fit" into global state when you @load each model... but as described above, I don't like macro magic like this.)
Each develop will develop their model in their own package. But they just define this fix function. E.g.
``` # in PkgA function fit!(model::ModelTypeA, X, y) # some code ... end ```
``` # in PkgB function fit!(model::ModelTypeB, X, y) # some code ... end ```
Each `fit!` function is distinct. It's the same as `model.fit` where the `model` is controlled by the individual developers.
Or the sum operator for string concatenation, which is the epitome of non-commutative operation.
I like your point of view, but still I'm skeptical of function and especially operator overloading. Shouldn't these semantic constraints that you mention be enforced by the language? For example, the language does not let you overload + for a non-commutative operation, and so on.
I was surprised to find Julia uses the multiplication method for string concatenation. For me it feels as wrong as using bitshift operators for I/O but maybe I'm missing something.
Multiplication for concatenation makes perfect sense: it is commutative, associative, and consistent with mathematical notation. A space would be even better. And the empty string would be great, but maybe difficult to implement if you want to allow multiple-letter variable names, as julia seems to do.
> For example, the language does not let you overload + for a non-commutative operation
It does allow it? I'm not even sure how one would disallow non-commutative operations. How would you know if a definition is commutative?
> how one would disallow non-commutative operations. How would you know if a definition is commutative?
I don't think that this is possible without solving the halting problem. But in practice, you can do that by documenting this enforcement and making it unfeasible to overload a commutative operator with non-commutative code. For example, the callers of the overloaded commutative operator can and do assume commutativity to optimize compilation; as in, you fill a matrix with "f(i+j)" and it may only evaluate the upper-half of the matrix (this is even more interesting for the associative case). I think that Mathematica does a similar thing, I recall several symbols for operators to be overloaded assuming certain symmetries. As another example, in C++ you must overload "<" with something that is an order relation, otherwise the sort function fails. Being more fancy, a test option of the interpreter may run each call of the operator in a random order, etc.
Also, related to Mathematica, using juxtaposition for product is very natural to mathematicians. It is indeed hard point of friction when moving from Mathematica to Maple or Matlab.
If (and only if, I have not looked at our hypothetical model and fit GF, but for the sake of argument, I will assume that it does) "fit" specialises on "model" will a "mode = AnOtherModel" cause "fit(model, x, y)" to be exquivalet to Python's "model.fit(x, y)". If you need to provide a custom "fit" method, you do so by providing a method specialised on AnOtherModel to the "fit" GF.
At no point is there a macro involved.
As for the "module a, model nn" and "module b, model nn", I would naively assume that they actually are different models, and therefore something specialising on "a.nn" will not get dispatched to when you pass a "b.nn".
Disclaimer: I don't actually know Julia, at all. But I have written substantial amounts of CLOS code (and Python, but I like CL and CLOS better).
I never said macros were required. I said implementing this type of code without OOP required more boilerplate, and MLJ uses macros to reduce that boilerplate.
As I understand module imports in Julia: Each module developer exports a list of publicly facing objects. Obviously "fit" and "model" would be among them. If you import two modules that both export a new "nn" subtype of shared parent type "model", and both extend "fit" and "predict" and etc to accept their own subtype "nn", then you have to manually specify which module you are referring to every time you call nn, or fit, or predict, or whatever. Is that wrong? If I just import PyTorch, import TensorFlow, and then call "mymodel = TensorFlow.nn; fit(mymodel, mydata)" then Julia doesn't know that the "fit" I am calling is the TensorFlow implementation and not the PyTorch implementation; what if I had WANTED to use module A's "fit" on module B's model, and they intentionally adopted the same abstract type system to enable this interoperability? So instead I have to write "mymodel = TensorFlow.nn; TensorFlow.fit(mymodel, mydata); TensorFlow.predict(mymodel, mynewdata)". Obviously the extra typing is mildly annoying but the bigger problem is potentially introducing bugs by mismatching modules, and the developer's cognitive overload of having to keep track of modules. Python style OOP is a more elegant solution to the namespace problem and results in more readable, maintainble code, at least in my opinion. Anyways, maybe Julia has a more elegant solution I'm not aware of, if so I'd love to hear it.
In other words, one package would define `fit(mymodel::TensorFlowModel)` and the other would define `fit(mymodel:PyTorchModel)`, and then when you call `fit` it'll just dispatch to the appropriate one depending on the type of `mymodel`.
This dispatch-oriented style also allows a shocking degree of composability, e.g. [1], where a lot of packages will just work together, such that you could for example just use the equivalent of PyTorch or TensorFlow on the equivalent of (e.g.) NumPy arrays without having to convert anything.
If you mean "what about the case where both packages just call their model type `Model`", while I've never run into that, the worst case scenario is just that you have to fall back to the Python style explicit Module.function usage (which was always allowed anyways...). And if you if you don't like names being exported, you can always just `import` a package instead of `using` it:
help?> import
search: import
import
import Foo will load the module or package Foo. Names from the imported Foo module can
be accessed with dot syntax (e.g. Foo.foo to access the name foo). See the manual
section about modules for details.
Even if using multiplication to join strings may be consistent with some properties of mathematical notation, it's less consistent with the usual meaning of the words (I checked the cosine similarity using GloVe 6B words embeddings, the similarity between "add" and "join" is 0.4 while between "multiply" and "join" is only 0.03), so it's probably the balance between being the general purpose and math oriented language.
I am aware of the ability to do eg "import TensorFlow; model = TensorFlow.model; TensorFlow.fit(model,data)"
As I mentioned previously, I find Python's OOP "model.fit" syntax to be better, for a variety of reasons.
Thank you for your engagement. Have a nice day.
m1 = TensorFlow.model()
fit(m1, data)
m2 = Pytorch.model()
fit(m2, data)
YensorFlow.fit and Pytorch.fit are just different methods of the same function.
You've formed some strong opinions based on a pretty big misunderstanding.
<edit>I'm also nervous of relying on the recourse of asking package maintainers to edit their variable names to improve compatibility with the random third package I want to use; maybe the culture is different in Julia but in Python that's a good way to get laughed out of the issue tracker :) </edit>
If I were you I would maybe ask people like @systemvoltage who took the plunge and wrote a big project in Julia only to find they had trouble maintaining the project. Maybe one reason he can't upgrade without breaking everything is because of namespace collisions amongst his many dependencies? If not that, it's something like that.
I know in Julia I can just precede every function call and object instantiation with "modulename." and solve the namespace problem that way. What I want to do instead, what I do in python, is bind one namespace of function methods to each object, so that as I code, I don't have to remember which module each each object came from. That is the appeal to me of "model.fit" over "module.fit(model)".
EDIT: This is not some "shave a few seconds off coding time" quality of life issue. This is a mission critical requirements in many enterprise contexts.
Scenario A: Model Development Team trains a model, serializes it, and sends the serialized model object to Production. You want Production to have to lookup which software Package the Model Development Team used for each model, just so they know which "predict" function to call? No, "predict" should just be packaged with the model object as a method, along with "retrain", "diagnose", etc.
Scenario B: I want to fit a dozen different types of models, from multiple packages, on the same data, to evaluate how they each do, and build build some sort of ensemble meta model. So I need to loop over each model object in my complicated ensembling logic. You want me to also programmatically keep track of which module each model comes from?
These are important, happens-every-day use cases in the enterprise.
I think the best solution for this in Julia is to package the model state with all the "method" functions in one struct. Again, this is what MLJ does. This is the closest Julia gets to OOP. But then you either need a lot of boilerplate code, or a Macro to unpack it all for you behind the scenes.
The bigger issue for production in my experience is about packaging the right model with the right version. I don't think anyone has to do `module1.fit` everywhere, since `fit` would've likely come from the same source.
That solves both of your scenarios
No.
As described in the great-great-great-great-great-great-great-great-great-parent comment, the problem I am describing is that of trying to combine models from multiple software authors. You may not have that problem. It may not be a common problem among Julia's academic user base. I do have that problem.
Thanks for reading.
I'm a bit confused still though why you say it's a "missing" feature, given that as we discussed above, there is absolutely nothing to stop anyone who wants to use the "Python OOP" style of namespacing in Julia from doing so? Most of us don't seem to find it necessary or to prefer it personally, but that doesn't restrict anyone else from choosing it.
Can I trouble you to make a post either on discourse or on the slack? I'd really like this to get in front of the broader julia community and core devs, and you're the best person to do that. Maybe there's a solution of which I'm unaware....or there could be some design discussions to come out of this.
https://julialang.org/community/ (slack and discourse link)
To be 100% honest with you, there's pretty much 0% chance of me adopting Julia in the next 12 months. I evaluated it before embarking on my current project, but ended up going with Python, and now I have several thousand lines of Python code that work fine, and I'm not going to rewrite it all in the near future. At some point in the medium term, I'll re-evaluate Julia. But until then, I don't want to lead anyone on any wild goose chase. Even if you solved this problem, and all my problems, I'm just not in a position to switch right now. So for that reason I think I'll hold off on issuing a community wide call for help. But I'm cautiously optimistic that at some point in the future I'll be writing Julia professionally.
A lot of this is also probably cultural rather than language features. The first thing they teach any Python data science boot camp initiate is: never "from numpy import *", always "import numpy as np" and yet in Julia "using" appears more common than "import"...
I also wouldn't read too much into my one example. It was initially meant just as an illustrative point, but somehow I was so bad at explaining it that it took tons of comments for me to get my minor point across. I do think the MLJ guys are properly on the right track, and that should work fine for most people who don't mind Macros. Maybe I'm in the minority in hating Macros.
The more commonly cited issues around compile time caching, tooling, etc. are boring to list off yet again, but probably the right areas of focus for the community, in terms of both impact and ease of implementation.
More generally, I really do think you're better off talking to people like @systemvoltage, who have actually given Julia more of a chance than I have. If I worked for Julia Computing I'd be reaching out to him and begging to get his brain dump on all the challenges he faced. In any business, it's always easier to reduce churn and learn from your (unhappy) customers, then it is to convert new prospects, whether that business is programming languages or investment banking.
Best of luck. Sincerely.