OTP 23(erlang.org) |
> Elixir v1.10.3 is fully compatible with Erlang/OTP 23.0 - so up to date Elixir users are ready to give it a try!
> CPU quotas are now taken into account when deciding the default number of online schedulers. > Thus, automatically making Erlang a good citizen in container environments where quotas are applied, such as docker with the --cpus flag.
Super happy about this!
I had originally taken a Programming Language Paradigms class in college with racket, and I really didn't appreciate functional ideas(i.e. syntax is my excuse). I'm now a really big fan of functional language idioms.
Anyone interested should try the futurelearn class, https://www.futurelearn.com/courses/functional-programming-e....
I mean, unlike in typical hard FP languages like Haskell or Elm, mutable state is rampant in your average Elixir app, it's just spread out across many (global singleton) little processes. Only inside a process you're doing "true" FP but given how small the average process is in scope, in practice the only real big difference is that you can't to an `i++` style for loop. Oh no!
But once you leave the process boundary, and often even before it, all bets are off. The amount of Elixir forum messages I've read that go "you can't do X at that point, because Y hasn't completed yet" is nuts.
Eg you can't broadcast in a phoenix channel `join` because the channel hasn't been fully initialized yet. So you send yourself an :after_join message and do the broadcast in there. I don't know about you but to me this feels a lot more like C++ than like Haskell.
Or consider the library module Agent which is exactly identical in semantics to a global singleton variable in an OO/imperative language. It's just a blob of data that you can get or set.
Now, I don't think any of these are disadvantages. I did mostly C# and JavaScript before Elixir, so I'm used to the occasional mutable state flying around.
But I'll never understand that people like Elixir for being FP. You just get such a small subset of the usual advantages of FP that it feels like an implementation detail. There's lots of advantages, but freedom from thinking about state isn't one of them.
To be clear, the Elm model of putting everything into a big tree and and transforming that at every user input is very unusual among even FP languages, and is not the model typically used in Haskell. This might be what you are thinking of.
Edit: I also want to point out that Elixir processes can be registered globally to act as singletons, but by default you can spawn any number of agents or other processes at runtime, meaning they are not singletons.
To me the answer is: using mutable state is opt-in. I disagree that "mutable state is rampant".
By opting in to the mutable state constructs you are basically saying "I know what I am doing, let me do my work" which is IMO quite fine because "pure" FP languages like Haskell can be a huge hassle when you actually need to deal with the real world.
To me Elixir is a very nice compromise.
What is a "typical" FP language? Are Scheme, Common Lisp, OCaml, SML, atypical?
> mutable state is rampant in your average Elixir app
I don't think this is generally true, or at least not on my experience, but I guess it may depend on what would one consider rampant.
Yeah but you're using that small subset in 90% of your code.
Look, this is not a thing to worry about in elixir:
def p(my_array):
do_something_with_this_array(my_array)
return my_array
I personally think elixir is really special and an amazing fit for any project that needs high IO concurrency.
I would say the only other functional language that has the same bent is Julia, which is "functional for the working scientist". It makes different choices about where to expose state, understandable, since scientific computing has different tradeoffs from systems programming.
Rust and Scheme have been gateway drugs ha. I found I really like OCaml-y languages.
Now, I’m very interested in Erlang (and to some degree elixir). I’m learning as much as I can about Erlang and the ecosystem; I’m trying to answer the question, “why isn’t Erlang more popular?”
Any guesses? Reasons? (syntax aside)
Habit, confirmation bias, sunk cost fallacy.
Namely: people have gotten a lot of battle scars by working with what pays their bills -- PHP, Ruby, Python, C#, Java -- and they refuse to look at an alternative because that would render their huge time and energy investment moot (in their eyes at least; I don't see why this has to be the case but plenty of people have been adamant about this without giving an explanation).
I've only become a better programmer since I adopted Elixir but I never stopped using other languages.
All of that plus what PG calls "the middlebrow dismissal" are the main reasons IMO. People are just too set in their ways.
Can some erlanger explain why you are excited about certain things from the update?
Anyone know about the status / rational for the `erl_interface.h` API?
1: 1: https://github.com/elcritch/einode/ 2: https://github.com/elcritch/einode/tree/master/tests/c-nodes...
erl_interface itself is not going away but it is evolving with the BEAM VM. Deprecation warnings should be checked when compiling code on each major release to avoid surprises as features are usually deprecated for one release and then removed in the next release each year. The erl_interface documentation should be up to date with regards to new APIs and might be worth browsing again to get an idea of the what changes look like.
Yay! We need to actively and in a strong forceful manner remove broken security products.
Erlang/OTP 22
Interactive Elixir (1.10.3)
defmodule Test do
def matchme(value) do
x = :foo
%{^x => bar} = value
IO.puts(bar)
end
end
> Test.matchme(%{foo: "bar"})
bar
:ok > <<len::8, val::binary-size(len)>> = <<3>> <> "abc"
<<3, 97, 98, 99>>
> val
"abc"
(Your example demonstrates 'pinning' which has been a thing since always I think).
I've always wanted to try OTP for a real world project (simply because it looks more elegant and has less parts), and now may be the time, but i'm so scared of discovering huge pitfalls down the road that are better solved by more recent techs..
There are drawbacks of course but having worked with Python, PHP and JS. I haven't seen any real drawbacks. I wouldn't choose it for minimal overhead or the fastest compute maybe..
Edit: typo
I'm asking, because to my knowledge, the only "huge number of users" recent success story is whatsapp, and after having viewed all the talks about their stack and experience, it seems that the team was so extremely talented that they probably would have made something good with any tech.
That's why the value of OTP seems to be less obvious at the moment (from the point of view of someone who doesn't use it). But there's surely something i'm missing.
- from what I understand Akka (Scala) heavily borrows from Erlang/OTP - is this correct? What would be the biggest similarities/differences?
- to what extent OTP is unique to Erlang and to what extent it can be implemented in another language?
- any articles/materials out there showing the most valuable bits of OTP applied in JavaScript/NodeJS?
I'm too used to the safety that rust or F# gives at compile time to give that up.
It is nowhere near as ancient as sharks, nor as vicious, but it employs some new physiology, and can survive just as long.
But that was a very different world and it sticks out like an outsider, when really it's been there the whole time.
(sorry, this analogy got away from me a bit).
the socket library (if I am correct) is the beginning of redoing the entire network stack in a way that is more commensurate with the way that typical programmers expect (but with the right erlang-ey).
> Allow underscores in numeric literals to improve readability
This is great whenever a language does it.
> The embedded documentation is created as docchunks (EEP 48)
It looks like the erlang community is converging on the elixir strategy for documentation. This is fantastic, since elixir documentation is quite good, and it shows that the two languages are starting to put their friendly animosity aside and really working together.
> distributed named process groups is introduced
This is kind of huge. Sometimes you want a locking, fully consistent process management (so CRDTs are inappropriate), and you don't want a full raft/paxos under the hood.
Basically, what is exciting is how well engineered it is.
Other than a bunch of code reliant on the stuff removed from erl_interface, of course. No idea how big of an impact that has on real-world projects, though.
The deprecation of erlang:get_stacktrace/0 also gave me a brief heart attack for one of my projects¹, but luckily I had the foresight to gate that to older OTP versions (and with newer versions the relevant code already removes dependence on that function), so I can rest at least a little bit easy.
----
¹: https://github.com/otpcl/otpcl/blob/73167c58771f700963cbd9f7...
I personally find Elixir's functional programming is easier, because there are fewer things you have to deal with than object oriented. There are basically no constructs in Elixir that you don't have out of the box in an OO language like Javascript or Python. You just have to deal with that you can't do things like assign variables in if statements (you just have to explicitly export them) and you don't have for loops.
Having said that, Enum.map and Enum.reduce are really hard to get used to and can be an obstacle in terms of "why can't I do this thing that is so easy in X".
In the long run, once you get used to reading them (which honestly took me about 4-5 months, I have like 20+ years of programming experience) I think they are easier because they are declarative -- you can see exactly what is happening to each piece of your list and in the case of reduce, what state is being passed through each iteration.
For loops are far more unstructured, literally anything in any of your parent scopes could be what you're keeping track of through iterations, which means, your mistake surface area is much much higher. The same goes for getting used to "if statements that export values" but, maybe, less dramatically.
I know this is difficult to hear since you gave up, but I promise you if you push through it, eventually it will feel like writing elixir is "programming with training wheels on" you can do crazy hard things without worrying about large classes of logical and structural programming errors. Hope you come back to it!
The paradigm of Erlang is message passing. The primary idiom is logic programming (like Prolog). Functional programming is related to logic programming. Between message passing, logic programming, and simple primitives for concurrency; Erlang programs tend to look a little strange. But it all hangs together in a way that has been field tested for thirty years. Erlang is a language that was engineered for use by engineers for solving engineering problems.
Elixr is mostly a way of avoiding learning Erlang and making the Beam VM popular. It is a procedural programming abstraction layer. For me, Elixr's abstractions generate an impedance mismatch and Erlang's syntax better expresses the engineering mechanisms of concurrent systems and logic programming. YMMV.
Try sticking through it.
OTP paradigms are definitely intended for a larger system that needs its own distribution schemes, handles its own internal process uptime and restarts, and for some use cases, hot code updates (running servers that update without restarting). OTP nodes can also easily and natively communicate between each other across the network, allowing for some pretty sophisticated distributed systems.
There are many solutions that could be more easily implemented on some sort of serverless architecture, but in cases where you want to maintain a service with a high degree of reliability and distributed capability, OTP is a good choice. After all, it was developed for the telecom industry to handle phone system communications loads.
Worth noting: there are other solutions (some of which are already in serverless environments) that solve the same problems of service uptime during code updates/deployment by keeping an older instance running until the new instance is ready to accept traffic. Services like Kubernetes also implement a lot of cluster management capabilities that solve the same problems OTP was attempting to solve, and you can use whatever language and engine/framework you want. Personally, I think if you wrote a service purely in Erlang/Elixir and directly applied all OTP solutions for uptime, process communication, and distribution, it would be faster and more efficient than a typical serverless or container-based system. But, it would also lock you into that system and make any non-OTP tech more difficult to integrate.
Together, Erlang and OTP provide a batteries included environment for building and running scalable concurrent systems. Scalable down to any device that will run Linux, not just up into the cloud.
Unlike OTP it must be language-agnostic and also aware of latency and bandwidth.
One common use-case, is where a payload's prefix-length field encodes the number of bytes of payload minus one. You'll see this in many protocols as an optimization: if the payload is always at least one byte, then one byte of payload can be represented as a length of 0, allowing up to 256 bytes of payload rather than 255.
Here's an example of how the new syntax would be used for that, from the EEP-52 proposal (http://erlang.org/eeps/eep-0052.html):
example1(<<Size:8, Payload:(Size+1)/binary, Rest/binary>>) ->
{Payload,Rest}.
example1(<<Size:8, Rest0:binary>>) ->
PayloadSz = Size + 1,
<<Payload:PayloadSz/binary, Rest1:binary>> = Rest0,
{Payload, Rest1}.
Oh, and since the newer code is all in a head-clause, if it fails on matching Payload, it'll move on to attempting to match on the next head-clause, rather than generating a badmatch error. Just like if you produce an error in a head clause's guard clause expression.
-----
On a separate note, despite this example being simple, you can actually do arbitrarily-complex things:
example2(<<HeaderSize:8, BodyChunkSize:8, BodyNumChunks:8,
Payload:(HeaderSize + (BodyNumChunks * BodyChunkLen))/binary,Rest/binary>>) ->
{Payload,Rest}.There are a lot of Elixir products and systems out there and before those there were quite a few in Erlang. I don't have a sufficient sample size from my own experience but it is hardly old and disused or too new to be sure.
Changelog podcast runs on it. Dockyard build with it. It seems the new default for many previously Rails shops.
I think that's the perception, but it isn't necessarily true. I work on an Elixir team that quite successfully runs a stateful server. The service is pretty stable and the stateful portion is never the source of problems. That's one anecdote, but if you look at servers for online games, they're often quite stateful and work rather well.
If your system is designed properly, it's not unreasonable to have a node out of service for a couple hours while it's repaired. Nodes which don't have database state can often wait until business hours to be repaired, if there are enough nodes that you can have a few down.
That said I have grown an appreciation for more esoteric languages and enjoy seeing the way they handle various problems it's just likely they'll never have a place in my toolkit.
You have a list of N elements and want to trasform it into another list of N elements -> map
You have a list and want to transform it into a single value -> reduce
Fin.
But hey, you ask, can’t a list be considered a single value? Yes, of course, so map can be implemented using reduce, making it the most useful FP construct
What I like of Elixir is not functional programming, it's the extensive use of pattern matching. What I like less is the convoluted syntax of GenServer with all those handle_cast/calls that obfuscate the real code. As a GenServer is usually a way to store a state, they should have had the courage of calling them objects and give them an object like syntax. After all they initially sold Elixir to developers coming from OO languages. If they had given it a Java like syntax maybe it would be 10 or 100 times as big by now.
While GenServer's store state they are, in my mind, more akin to microservices than Java/C++ Objects. They're like microservices, but without needing a separate service bus, global naming system (pg2), etc. Now I do wish Dialyzer/Dialyxer also had better support for checking GenServer handler's and messages, especially intra-process. You have a point the syntax their could be spruced up some perhaps. The process / GenServer paradigm kind of remind me of Smalltalk in a way, where you're passing messages that object may or may not want to respond to. That's not really possible with C++/Java objects.
GenServers do have their own state, but they are used for way more than just storing state. It's probably best to literally think of it as a server in your running application, analogous to a server running on a network.
If storing state is all you're trying to do you might want to look at Agent or possibly ETS.
Further, GenServers come from Erlang, not Elixir. They're intended to model all sorts of runtime properties of code.
I got severely disheartened that 2-3 casual mentions of Elixir were enough for several people in this thread to attack and quickly stereotype me. I think I'll just keep quiet, or at least not mention what I work with. This seems to get the message across much better.
Is it? When I looked at it, it just seemed to be a Ruby-ish syntax on top of Erlang. It still felt very functional.
It's not about how the Erlang VM is implemented. You could implement it in pure Haskell and compile it so some hypothetical pure CPU. Wouldn't change that part of how you interact with it that is stateful as described.
(And I liked programming in Erlang.)
If you hit a performance bottleneck you might as well just use RabbitMQ or Kafka to queue up stuff and process it as it comes along. Or apply back-pressure if your current code allows for it.
On Haskell, yes, you can just work off a thread and write shared buffers. But you are better off using STM or Async (these are a bit like futures), which compose, and you can write pipelines out of them.
Not sure what you mean by "actor frameworks don't compose"?
Maybe composition for compositions' sake isn't that important?
This isn't meant as criticism -- I think Erlang/OTP is a brilliant piece of work. But arguably Erlang/OTP is a practical tool that achieved practicality by embracing dogma.
I wonder though if Erlang might have been a footnote in programming history if it weren't for the mindshare that OTP generated. "Simple functional language" is attractive, but "simple functional language with world-class platform" turned out to be a game-changer.
It absolutely would be. There's plenty of "simple FP languages" out there. OTP is very definitely the main selling point of Erlang, and nowadays of Elixir as well.
But maybe we'll talk the same language if you give a few examples. I was under the impression that my parent poster made the point of "but your FP code gets compiled to imperative mutable code so FP is not good" or something. If I was mistaken in my interpretation then we're talking past each other.
Have you ever programmed in Erlang?
About your first paragraph: not all databases should be append only, but it would be a good idea to make append-only the default and only deviate for a good reason. (Instead of making mutable the default, and only weirdos go for append-only.)
Though I don't think anyone ever suggested the actor model (or Erlang) was doing C/C++ craziness.
Definitely. I was about to ask "but when did you actually need to compose actors and how does that even make sense?" -- and while I am sure there are people who would find a scenario I feel that would still be tarrying on minutiae.
Obviously the actor model is not a panacea. But for my commercial work Elixir -- and thus Erlang's OTP -- has been a true blessing. There aren't many commercial scenarios where OTP is a very poor fit. They do exist but I'd dare saying they are no more than 10-15% of everything you can stumble upon out there.
It's quite sad how fiercely the average HN audience is attacking people who are trying to show an alternative way of doing things.
But if you have the choice, composability always seems much nicer to me and stops me wanting to tear my hair out.
Yes, exactly. No one was implying that Erlang's model is mimicking those particularly weird languages. The statement was much more narrow.
In the sense of being failure domains, OTP "actors", or processes, ARE composable. With very little boilerplate, that is built in as primitive BIFs in the standard library, I can bundle failure domains together and orchestrate them using links, monitors, and supervision strategies, with the ultimate failure domain being a single erlang VM operating inside a cluster of erlang VMs, all supported without third party libraries as a part of the way of doing things.
Then the idea of adding a supervising and observability was a natural addition to these systems.
That said, I think OTP is a great tool for modeling failure domains and I think it does a great job.