Go is a next-gen trumpian language that rejects sum types, pattern matching, non-nil pointers, and for years, generics; it's unhinged.

A modern thread-per-core shared-nothing architecture takes this even further and tries to eliminate blocking at the hardware level for the same basic reason.

You used to be able to depend on OS threads getting reused because you were pooling them. You can do the same with virtual threads if you wish and you will get the same behavior. The difference is we ought to spawn new threads per task now.

Side note, you have to specifically use InheritableThreadLocal to get the inheritance behavior you speak of.

How do you developers spew such strong opinions without taking a moment to think about what you're about to say. Rust cannot be directly compared to C#, Java or even Go.

You don't get a runtime or a GC with rust. The developer experience is excellent, you get a lot of control over everything you're building with it. Yes it's not as magical as languages and runtimes like you've mentioned, but the fact that I can at anytime rip those abstractions off and make my service extremely lightweight and performant is not something those languages will allow you to do.

And this is coming from someone who's written non blocking services before Async rust was a thing with just MIO.

The very fact Rust gets mentioned between these languages should be a tribute to the efforts of it's maintainers and core team. The amount of tooling and features they've added into the language gives developers of every realm liberty to try and build what they want.

Honestly, you can hold whatever opinion you want on any language but your comparison really doesn't make sense.

For example ktor works on the JVM but you can also build native applications with it. And I use ktor client in the browser. When running in the browser it uses the browser fetch API. When running on the jvm you can configure it to use any of a wide range of Java http clients. On native it uses curl.

Some operations that don't cause a task switch to another virtual thread are:

- If you've called into a native library and back into Java that then blocks. In practice this never happens because Java code doesn't rely on native libraries or frameworks that much and when it does happen it's nearly always in-and-out quickly without callbacks. This can't be fixed by the JVM, however.

- File IO. No fundamental problem here, it can be fixed, it's just that not so many programs need tens of thousands of threads doing async file IO.

- If you're holding a lock using 'synchronized'. No fundamental problem here, it's just annoying because of how HotSpot is implemented. They're fixing this at the moment.

In practice it's mostly the last one that causes issues in real apps. It's not hard to work around, and eventually those workarounds won't be needed anymore.

Does the same apply to virtual threads?

Edit: I think I answered my own question. Java virtual threads have the same memory model as regular java threads so yes, I need to use the same semantics. That rules replacing the eventloop model for us.

I’m not sure I said outdated, but I can see what you mean by how I called Javas approach “more modern”. What I should have called Javas approach was “correctly designed”.

C#’s async/await isn’t all terrible as you point out, but it’s designed wrong from the bottom up because computation should always be blocking by default. The fact that you can accidentally start running your code asynchronous is just… Aside from trapping developers with simple mistakes, it’s also part of what has lead to the ecosystem irrecoverably being split into two.

I was actually a little surprised to see Microsoft make their whole .Net to .Net core without addressing some of the glaring issues with it, when that massive disruption process uprooted everything anyway.

For example, in Erlang the entire VM is built around green threads with a huge amount of guarantees and mechanisms: https://news.ycombinator.com/item?id=40989995

When your entire system is optimized for green threads, the question of "it still needs to map onto OS threads" loses its significance

That means that Java programmers have to be very careful when writing code, lest they block the entire underlying (OS) thread!

Again, Go already went through that experience. It was painful. Java should have learned and implemented it from the start

With virtual threads, it seems like if you don't hit gotchas, you can spawn a thead, and run straight through blocking code and not worry about too many threads, etc. So you could do thread per connection/user chat servers and http servers and what not.

Yes, it's still shared memory, so you can miss out on the simplifying effect of explicit communication instead of shared memory communication and how that makes it easy to work with remote and local communication partners. But you can build a mailbox system if you want (it's not going to be as nice as built in one, of course). I'm not sure if Java virtual threads can kill each other effectively, either.

JVM Green Threads here serve predominantly back-end scenarios, where most of the items on the list are not of concern. This list also exists to address bad habits that carried over from before the tasks were introduced, many years ago.

In general, the perceived want of green threads is in part caused by misunderstanding of that one bad article about function coloring. And that one bad article about function coloring also does not talk about the way you do async in C#.

Async/await in C# in back-end is a very easy to work with model with explicit understanding where a method returns an operation that promises to complete in the future or not, and composing tasks[0] for easy (massive) concurrency is significantly more idiomatic than doing so with green threads or completable futures that existed in Java before these. And as evidenced by adoption of green threads by large scale Java projects, turns out the failure modes share similarities except green threads end up violating way more expectations and the code author may not have any indication or explicit mechanism to address this, like using AsyncLocal.

Also one change to look for is "Runtime Handled Tasks" project in .NET that will replace Roslyn-generated state machine code with runtime-provided suspension mechanism which will only ever suspend at true suspension points where task's execution actually yields asynchronously. So far numbers show at least 5x decrease in overhead, which is massive and will bring performance of computation heavy async paths in line with sync ones:

https://github.com/dotnet/runtimelab/blob/feature/async2-exp...

Note that you were trivially able to have millions of scheduled tasks even before that as they are very lightweight.

[0]: e.g. sending requests in parallel is just this

    using var http = new HttpClient() {
        BaseAddress = new("https://news.ycombinator.com/news")
    };

    var requests = Enumerable
        .Range(1, 4)
        .Select(n => $"?p={n}")
        .Select(http.GetStringAsync);

    var pages = await Task.WhenAll(requests);

Async runtimes themselves are simply attempts to bolt-on green threads on top of a language that doesn't support them on a language level. In JavaScript, async/await uses Promises to enable callback-code to interact with key language features like try/catch, for/while/break, return, etc. In Python, async/await is just syntax sugar for coroutines, which are again just syntax sugar for CPS-style classes with methods split at each "yield". Not sure about Rust, but it probably also uses some Rust macro magic to do something similar.

While 'async' is just a name, underneath it's epoll - and the virtual threads would not perform better than a proper NIO (epoll) server. I dont consider myself an 'async expert' but I have my share of writing NIO code (dare say not terrible at all)

The key with virtual threads is they are so light weight that you can have thousands of them running concurrently: even when they block for I/O, it doesn't matter. It's similar to light weight coroutine in other language like Go or Kotlin.

This is a common problem with people using Java parallel streams, because they by default share a single global thread pool and the way to use your own thread pool is also extremely counterintuitive, because it essentially relies on some implicit thread local magic to choose to distribute the stream in the thread pool that the parallel stream was launched on, instead of passing it as a parameter.

It would be best if people came up with more dynamic back pressure strategies, because this is a more general problem that goes way beyond thread pools. In fact, one of the key problems of automatic parallelization is deciding at what point there is too much parallelization.

Also the virtual threads run on a "traditional" thread pool to my understanding, so you can just tweak the number of worker threads to cap the total concurrency.

The benefit is it's overall more efficient (in the general case) and lets you write linear blocking code (as opposed to function coloring). You don't have to use it, but it's nice that it's there. Now hopefully Valhalla actually makes it in eventually

Async/await is able to achieve good UX around explicitly defining what goes on your Main thread and what goes elsewhere. Its trivial to mix UI thread and background thread code by bouncing between synchronization contexts as needed.

When the threading model is implicit its impossible to have this control.

Also, nothing prevents you from building your own, more limited but safer (the two always come together!) abstraction on top, but you couldn’t express Loom on async as the primitive.

Go made half a step in the right direction with goroutines, but never committed fully

Java virtual threads are stackful; they have to save and restore the stack every time they mount a different virtual thread to the platform thread. They do this by naive[0] copying of the stack out to a heap allocation and then back again, every time. That's clearly a context switch that you're paying for; it's just not in the kernel. I believe this is what the person you're replying to is talking about.

[0] Not totally naive. They do take some effort to copy only subsets of the stack if they can get away with it. But it's still all done by copies. I don't know enough to understand why they need to copy and can't just swap stack pointers. I think it's related to the need to dynamically grow the stack when the thread is active vs. having a fixed size heap allocation to store the stack copy.

Goroutines are preemptible by the runtime (since https://go.dev/doc/go1.14#runtime) but they're still not addressable or killable through the language itself.

No.

The goal is to maximize the number of tasks you can run concurrently, while imposing on the developers a low cognitive load to write and maintain the code.

> Why did you need a thread pool before but not any more?

You still need a thread pool. Except with virtual threads you are no longer bound to run a single task per thread. This is specially desirable when workloads are IO-bound and will expectedly idle while waiting for external events. If you have a never-ending queue of tasks waiting to run, why should you block a thread consuming that task queue by running a task that stays idle while waiting for something to happen? You're better off starting the task and setting it aside the moment it awaits for something to happen.

> What resource was exhausted to prevent you from putting every request on a thread?

You can create a thread in your code and not worry whether that thing will then be some day run in a huge loop or receive thousands of requests and therefore spend all your memory on thread overhead. Go and other languages (in Java's ecosystem there's Kotlin for example) employ similar mechanisms to avoid native thread overhead, but you have to think about them. Like, there's tutorial code where everything is nice & simple, and then there's real world code where a lot of it must run in these special constructs that may have little to do with what you saw in those first "Hello, world" samples.

Java's approach tries to erase the difference between virtual and real threads. The programmer should have to employ no special techniques when using virtual threads and should be able to use everything the language has to offer (this isn't true in many languages' virtual/green threads implementations). Old libraries should continue working and perhaps not even be aware they're being run on virtual threads (although, caveats do apply for low level/high performance stuff, see above posts). And libraries that you interact with don't have to care what "model" of green threading you're using or specifically expose "red" and "blue" functions.

I think it’s definitely worth a read.

I'm not familiar with windows, but this certainly isn't the case on Linux. It only costs 2mb-8mb of virtual address space, not actual physical memory. And there's no particular reason to believe the JVM can have a list of threads and their states more efficiently than the kernel can.

All you really save is the syscall to create it and some context switching costs as the JVM doesn't need to deal with saving/restoring registers as there's no preemption.

The downside though is you don't have any preemption, which depending on your usage is a really fucking massive downside.

I suspect if you care about threads overhead, you won't pick Java, because there will be overhead in other areas too

> From what I gather virtual threads are an alternative to “callback-hell” (js) or async coloring (python).

there is also existing ExecutorService and futures in Java

except now you need to prove somehow that all 100 libs in your project support virtual threads.

> Virtual threads improve throughput because the moment a task is waiting for anything like IO, the thread is able to service any other task in the queue.

from reading similar discussions, linux for example doesn't have true IO async API, you just push lock of Java thread to lock of thread in the kernel

Virtual threads usually require significantly fewer resources to spawn and run. And, if the underlying system is implemented with them in mind, they can use fewer context switches, and possibly even fewer cache misses etc.

So I think it's meaningful to define what we mean by context switch here.