I'm not saying that all our problems are solved and the programming world will now be rainbows and butterflies, I'm just saying that this feature is the correct framing and abstraction for issues we've run into many times in the past, and it has the potential to greatly simplify and unify the hacky, bespoke, situational solutions we've found.
I guess if it helps you understand typed effects if you describe it as "java checked exceptions with an option to resume" then I'm glad that works for you, but for me, Java exceptions have so much other baggage surrounding their design that I would prefer describing it from the other direction: "typed effects would enable you to implement a host of cross-stack features, including a checked exception system like Java's".
https://overreacted.io/algebraic-effects-for-the-rest-of-us/
https://users.scala-lang.org/t/from-scala-monadic-effects-to...
And this is how it should be. Not a language feature, but library. Dealing with language feature you deal with compiler and may affect more people than needed, with library you can use (and extend) it as you wish.
Wouldn't hold my breath
If someone has experience with algebraic effects, I have a question to ask. Why are they needed at all as a type system extension and why can't they just be represented with function types? (excuse my Haskell pseudocode, I'm just a filthy C++ programmer abusing the notation I don't really know the language, also don't assume lazy semantics):
newtype Effect a = Effect (a -> Effect a)
newtype EffectHandler a = EffectHandler (() -> (a, EffectHandler a))
The only magic would be in the function that creates the continuation:
typed_callcc1 :: (Effect a -> Effect a) -> EffectHandler a
newtype Cont a b = Cont (a -> (b, Cont a b))
The original plan was to upstream only the multicore GC. This was sped up on the suggestion of the core developers and now 5.0 will bring parallelism and effect handlers (though without syntactic support for the latter).
https://discuss.ocaml.org/t/multicore-ocaml-september-2021-e... has a good explanation of effect handlers, syntax and what will be available in 5.0.
Totally blew my mind
I imagine exposure to algebraic-effects systems must make one feel the same way: like it's such an awful hack when a language has to have async support baked into its syntax!
Also you can start playing with effects today using the 4.12+domains branch on https://github.com/ocaml-multicore/ocaml-multicore
A neat side-effect (no pun intended) of doing things this way is that, unlike Promises, Plans can be stored as constants (or cached) and re-used multiple times.
I'm sure it's nowhere near as advanced or flexible as the OP, but it seems to be in the same general spirit
How does this compare with IO monads? Seems like they accomplish roughly the same goal
Monad transformers allow you to separate types of effects (so you can specify e.g., "this code only needs environment variables, not database access"), and, at least at compile time, select a different implementation for each effect. In Haskell, at least, though, they have a drawback of needing to define typeclass instances (interpreters) for every concrete monad stack (basically explicitly describe how they interact with each other - the n-squared instances problem. In practice, there's a bunch of template code to help mitigate the boilerplate).
Somewhat relatedly, Haxl, in an attempt to optimize effects, introduced a compiler change to identify less dynamic code (code that only needed Applicative), and Selective Functors, to allow for more optimization based on what's coming next.
Algebraic Effects (assuming I'm not incorrect to conflate them a bit with free effects/extensible effects) make things more dynamic, so you're instead effectively building the AST you want, and separately interpreting it at runtime. This should let you look at more of the call tree to decide on an execution plan. Since you'd also not be relying solely on the typeclass mechanism to pick an interpretation strategy, you should also be able to more easily describe how the interpreters compose, saving you from all the boilerplate of the transformers approach.
See e.g.: https://www.microsoft.com/en-us/research/publication/desugar... "Furthermore, 10,899 (28.0%) were fully desugared into Applicative and Functor combinators and thus would not require a Monad constraint." б) "The Haxl codebase at Facebook. [...] and 7,600 (26.9%) were fully desugared into Applicative and/or Functor."
https://www.youtube.com/watch?v=hrBq8R_kxI0
As well as this post, which relates react hooks to algebraic effects.
https://overreacted.io/algebraic-effects-for-the-rest-of-us/
> Over time, the runtime system itself tends to become a complex, monolithic piece of software, with extensive use of locks, condition variables, timers, thread pools, and other arcana.
I'm not an expert on this, but my understanding is that the problem that algebraic effects tries to solve is to improve language semantics to make it easier to separate different levels of abstraction (e.g. separating the what from the how), while also encoding the performed effects into the type system.
https://medium.com/traveloka-engineering/cooperative-vs-pree...
As far as I understand effects are similar to delimited continuations in the way the effect handler is found via dynamic scoping, but in addition there is an extension to the type system to guarantee that at least one effect handler of the correct type handler is in place.
So I'm wondering if it wouldn't it be better, or at least equivalent , to simply pass the continuation around (i.e with lexical scope) as a first class value and attach the effect type to it, obviating the need for an ad hoc type system extension.
I'm must be missing something and there must be some use cases that can't be easily expressed this way.
Let’s imagine two simple effects. One prints a string (I’ll call this ‘printer’) and one reads an int entered by the user (let’s call it ‘reader’) In this case, how would those effects be modelled with the types you wrote?
print :: String -> Effect () -> Effect ()
hello () = fst (typed_callcc1 (print “Hello”) ())
input_int :: () -> Effect Int -> Effect Int
In your example of reading an int the EffectHandler and Effect are simply switched (better names are sink and source). And yes, for IO you will need a scheduler, but streams are much more straightforward.
I have reached my limit of pure functional language knowledge, but I can offer you a working implementation [1] in an imperative language.
I've actually implemented these typed continuations in c++ years ago, and I'm trying to understand how they differ from effects (aside for the whole imperative thing).
In the C++ implementation, for convenience the continuation object is replaced with the next continuation when invoked, but internally actually invoking a continuation function returns the yielded value and the next continuation as for my EffectHandeler example.
https://github.com/gpderetta/libtask/blob/master/tests/conti...
So, whereas exceptions only jump backwards in the stack, resuming a continuation sorta lets you jump forwards again, back to where you were. It's really powerful stuff.
Project Loom will not change that, but it will improve performance for certain scenarios.
Scala’s varied async/concurrency libraries are implemented in user land, and still use threads underneath. Mechanically, you must opt in to these and have to work to interop with code that might use other primitives. Scala can handle a lot of this complexity at compile time w/ types, but it’s not perfect, and certain runtime behaviors will always be out of scope.
Loom improves this by allowing any language that runs on the JVM (Java, Scala, Clojure) to opaquely use virtual threads to run their existing synchronous, scheduler-unaware code on the new Loom concurrency primitives implemented in the JVM. That’s powerful!
Concurrency is not mainly about threads or performance, it is about program behavior semantics. Loom does not do anything about that, it "merely" improves performance. Well, you could say it actually makes semantics worse (you used (TM) for good reasons).
In that sense, I believe that Scala or any language with good concurrency semantics will benefit from Loom more than Java, unfortunately. But Java can of course still catch up on a language level. Even after such a long time, there are still new and interesting libraries (e.g. looking at JOOQ).
I'm not sure how one would get there with the JVM's memory model. you'd need something like actors and a preemptive scheduler per core at the VM level with a share nothing state between actors/virtual threads. Erlang utilizes message passing and immutability to do this.
Project Loom is bringing green threads back, now officially as virtual threads.
Additionally there is java.util.concurrent and for those that care to really go deep enough, custom schedulers.
Part of what Project Loom is doing is bringing lightweight usermode threads, called "Virtual Threads" and it's own scheduler.
Importantly you don't need share nothing state or immutability to add preemption, the JVM already has points during code execution where it knows the full state of the program. They call these "safepoints" and they're important for the GC to work properly. With the current implementation of Loom virtual threads are preempted when they do any blocking I/O or synchronization, but there's no reason why in the future they couldn't preempt them at any safepoint.
I don't know what you define as a few thousand active threads, but running the following C++ code let me run 70,000 threads before I got a resource error: