I too love reading the code for nptl(7). It's a riot. :D

Also: please don't spawn goroutines during init(). There's generally a better time and place, and in the event that you justifiably need a package-level background routine you can spawn it on demand with a sync.Once.

Oh, and I haven't even mentioned the absolute shitfest that is the cgroup namespace and how you have to set up cgroups before you unshare it because its behaviour changes based on what cgroups you were in when you unshared it.

Not for a long time, for all the reasons the parent mentioned (and then some).

I want to mix C++11/14 into that also. Now that the typesystem is used by the standard library to describe resource ownership in code whole categories of errors can be found at compile time. If you are new this it can seem like you are fighting the compiler, but once you learn that the compiler just won't let you make certain kinds of mistakes you get access to every level of abstraction with C++11/14 and Rust. You can code in terms of passing database bindings betweens threads in threadpools or you can twiddle individual bits in specific registers and everything in between.

I don't know Go well, but it seems really limited. You can't write certain classes of bugs, but you also can't write many design patterns.

Go is deliberately opinionated about threading of the runtime. I think it's unlikely Go will offer much more control over these internals (beyond GOMAXPROCS), given the philosophy around e.g. GC tuning.

Cgo is a beefed-up runtime.LockOSThread() that could be used to avoid having the author resort to a helper process.

> CGo is slow as balls to call into and return from.

They're slower than Go function calls, but they still take only nanoseconds. This is negligible on the authors' scale of "launching an entire container".

For their case of "it is not possible to guarantee that a new OS process ... will run in a given namespace", you're not even returning from Cgo after exec.

> It's interacted with via comments.

Do you use build tags? Or go:generate? Like it or not, it's idiomatic.

I'm with you on the types, though! `go tool cgo -godefs` helps, but it would be great to see improvements especially in the reverse case of exporting Go buildmode=c-shared for C consumption. Still, a little marshaling seems tidier than a whole helper process.

It is entirely possible to write C-style programs in C++ (very few classes, globals all over the place) and so on, and at least for me, C-style software in C++ are much safer/easier to debug than C-style software in C .

Sort of. Go binaries are statically linked by default which is a must in situations where you are e.g. unsure about what libs are available in the current environment. You have to go through a lot of hoops to make sure your C executable is really fully statically linked.

No, but there are more options than just Go and C. Rust is an option that I'm shilling at the moment (though I've only started learning it, so take that with a grain of salt). The main reason I would want to write it in C is because there's a lot of string parsing code you have to write in order to make container runtimes work -- and as we all know that's probably the #1 source of security vulnerabilities.

> Are there no advantages to Go for runc?

There are, mainly due to network effect (everything else is written in Go) and getting contributions from the community (Go is easy to pick up). Unfortunately there are also disadvantages, and quite a few of those disadvantages are present in Go but are not present in other memory-safe and low-level languages (Rust is a good example because to be quite honest it's the only player in this space that doesn't try to do more than necessary).

Go is a good language for it's designed for (Web servers and similar things), but from my experience it's not the best choice for low-level tasks. We've seen cases where long-running container daemons (not naming any names) will crash if you run more than 1000 containers on a single system. They don't crash because of the actual daemon code but because of issues with Go's GC (it doesn't actually free memory sanely, it uses MADV_DONTNEED which inflates RSS and causes OOM to kill your daemon).

> And, would it be possible for someone to write a somewhat standardized Go library for doing this grunt work?

Of course (and you could argue that we have done that in runc with github.com/opencontainers/runc/libcontainer/nsenter), but the thing to note is that in order to get around problems in the Go runtime you have to split out a single piece of code into separate processes and have to now redesign how a single function would work. So moving it to a separate library means that development is even more frustrating (you've created an API around the internal implementation of whatever thing you're working on).

> Is it merely fear of C that keeps so much of the container infrastructure on Go?

I think the network effect is the main reason. Most of the people I've worked with know quite a lot of C (we do kernel work sometimes) so writing a runtime in C would be frustrating but entirely doable. The problem is that you couldn't just import it into a Go project (and people don't like cgo because it makes binaries harder to build in certain cases).

> I've only spent a couple of weeks looking peripherally at Go, and I already like it better than C

Go definitely has it's uses, and I still use it for new projects. For example I recently wrote a tool for dealing with OCI container images in Go[1]. The standard library of Go is quite nice (though I found some bugs in archive/tar but let's not go there) and I'm always quite amazed just how much you can do before you have to import external libraries.

But I recently started learning Rust and I am _really_ enjoying being able to understand what my program will actually look like when compiled. If you've ever had to strace a Go program, you'll know exactly what I mean. Debugging Go programs is basically fucking impossible.

[1]: https://github.com/openSUSE/umoci

In contrast, one of the reasons I'm a big fan of go is it is extremely easy to read for almost anyone. People pick it up very quickly.

C let's me focus on the underlying system and it is what Linux is written in, Python is quick to write and has great libraries, and Go is amazingly easy to pick up/read.

C++ always felt like it was more so a language designed for programmers who love code than something that lets me focus on what I love (systems).

Trust me, we've been trying to get Go to co-operate with containers for quite a few years. It's not as simple as just reading the standard library docs. ;)

It's not as easy as threads (M:N vs 1:1 is a red herring as far as this is concerned), but there's no free lunch.

1. https://www.reddit.com/r/rust/comments/6enj5d/what_does_rust...

I hope it continues to play nice, as I have a project depending on this behavior (bindings for a CPU emulator which spawns a thread for the CPU main loop, and calls back into my Go code on some events).

Thanks for the info. I guess the biggest problem is making sure `go` in a locked goroutine doesn't release threads to the scheduler from the locked goroutine? That's a weird thing to reason about. I initially guessed you'd need a separate feeder thread, but you want `go` from a locked thread to keep the uid and namespace of the locked thread.

So I guess there are two potential solutions:

- goroutines spawned from locked threads are pure coroutines and will not be scheduled on another thread

- goroutines spawned from locked threads can only execute on the locked thread, or on threads spawned from the locked thread that don't predate the goroutine. Child threads can also only be used for matching child goroutines, and will be more aggressively collected. This would converge closer to an oldschool threading model than green threads, but should hopefully prevent bugs like UID/namespace hopping.

This is the kind of thing you could possibly do with a small patch to the Go runtime. Go makes it so easy to build the compiler/runtime, perhaps forking Go for this is reasonable until there's an official solution.

There's also the issue of what to do with the primary thread that has been marked unsafe when it returns. Instead of LockOSThread, you probably want "give this OS thread and its children magic sandbox scheduling behavior, which includes collecting the thread when it returns, because we're going to call stuff like setuid".

Arguably it has been implemented with clone(2) which has flags.

> in fact, it's easier to reason about fork than something like POSIX threads, IMHO.

Not if you call fork() in a multi-threaded program. That ends _exceptionally_ badly (let's just say there's a reason Go doesn't expose syscall.Fork and it has to do with horrific deadlocks).

> I just didn't see the problem as being all that damning of Go.

The problem is more subtle, and it comes down to maintainability and understandably. If you ever decide to read the runc codebase, I apologise. One of the reasons the codebase is so scattered is because of these sorts of hacks where you have to work around issues in the Go runtime (because it doesn't give you enough control). In the article, whenever you read the small function you have to keep in mind that it's actually spawning a subprocess (which then means you have to think about what namespaces had the process joined and so on). Go is an okay language, but it simply wasn't designed for stuff this low-level. We would be much better served with Rust in my opinion.

> But, are you saying that to switch to a namespace in Go one must fork, whereas one wouldn't need to fork in C unless you need to switch namespace and you need concurrency

In C you don't need to fork to switch namespaces, you just call setns(...). For the PID namespace you need to fork, but that's just a quirk of the interface.

In Go, you theoretically don't need to fork either (syscall.Setns is available). However, there is no real way to safely use it. First of all, the namespace interfaces in Linux are quite fragile when it comes to multi-threaded processes, but combine that with a runtime that will switch you between OS threads at random. And while the documentation on runtime.GOMAXPROC and runtime.LockOSThread might trick you into believing it's possible to stop the Go runtime from doing a clone(CLONE_THREAD|CLONE_PARENT), you can't.

The article is stating that to control the namespace that the threads execute in, that a separate process must be spawned so that the entire process can be forced into the correct namespace. Otherwise, the runtime can spawn new threads as it sees fit and you don't have control over which namespace they are in.

It might be possible to work around this from within the same process if it were possible to force the runtime to not spawn new threads in particular cases. If you could control when the runtime was allowed to spawn new threads, then you could organize the program / threads in a way that would keep the correct code operating in the correct namespace. Unfortunately, you can't.

Disclaimer: I don't know Go, but this is my understanding from reading the article.

When changing namespace happens often in a hot path, forking might not be fast enough.

I disagree with Walton that the blame is on the N:M threading. It's really on the Linux kernel that has never made a clean distinction between threads and processes, both in kernel and in the APIs.

In addition fork() style concurrency was essentially nonexistent outside of Unix.

My favorite was the Comair christmas failure back in 2004 or 2005. I tried googling the root cause, but I think it was an Integer overflow in the 16 bit integer they used for storing the flight number. The system was designed in the mid 80s and 65535 flights was an insane amount, but but when the software failed on Christmas it interupted more than 1,100 flights.

Go hijacks control flow and makes it more difficult to reason about how your code will be compiled and executed. At least that is true in my personal experience.

Agreed, that's why I don't use it for anything super important yet. Nim is approaching a 1.0 release soon. Go has a similar problem in that it is maintained almost entirely by Google who has a history of dropping projects without warning.

> Go is more versatile and at the same time more mature than anything out there.

This is objectively incorrect. In fact, Go aims precisely to be non-versatile for the sake of simplicity. That is why Go does not have generics for instance.

> It is a different design and it excels in what it does (considering all tradeoffs now)

I don't think it's design is all that different. It looks like a stripped down version of C and it's definitely not the first PL focused on concurrency.

> Will Nim be as versatile and solid as Go in the future?

Nim is already leagues above Go in the versatility(I assume you mean flexibility?) department. As far Nim being as "solid" as Go, I'm not entirely sure what you mean. If you're asking about stability, I believe that Nim can reach a similar level of stability as Go, yes.

> You need a solid financial backing and certain amount of adoption where people actually write software that makes them money.

I agree with this. However, it's not always a quick process. The only reason Go is as popular as it is is because of Google's size and reputation(edited). Every programmer on Earth heard of Go within a few days of it's official release. Nim is taking a slow roll approach. Look at Python. It took almost 15 years before it started getting really popular.

All that said, I didn't come into this thread to argue about Go vs Nim. I've been accused of shilling Nim in the past. I'm sorry that I like talking about PLs I enjoy using.

This doesn't seem like a very credible statement on the face of it. Is Go more mature and more versatile than Python? Than Java? Than...Scala? Go does seem to share a lot of use cases (and limitations) with Java; and it would be hard to call it more mature.

It was the same for Linux, Python, Ruby. Being community-driven can be a bug or a feature.

> Go is more versatile

Nim has macros, templates, overloading and compiles to C, JS, Objective-C. Runs on more architectures than Go including arduino microcontrollers.

> mature than anything out there

Go is not more mature than C, C++, Java, Python, Perl...

> You need a solid financial backing

See Linux, Python, Ruby... many projects had no big company or funding behind them.

This isn't true in Linux or Windows so I don't see why Go would make this assumption other than poor design.

>The property should be accessible via a handle that can be shared amongst all threads.

It literally is. The namespace information is shared with all threads in a PID group globally available in the /proc fs.

---

What OP is doing is effectively having 1 program run part of itself in 1 container, and another part outside of that container.

Go isn't a systems programming language so this level of fine grain control isn't possible. Hell its pretty difficult in a C/Rust/C++ environment.

The flow of data through channels can be confusing until the concept becomes familiar, but the behavior is well-defined, so the behavior is as easy or difficult to predict as of any program of the given complexity.

    Pid ! {self(), 42}

    receive
        {From, Data} when is_pid(From) ->
            handle_data(From, Data)
    end

Too bad we can't use Erlang for systems programming, but you can't have everything!

A channel is just a thread-safe queue. An unbuffered channel blocks until something else receives the sent value. A buffered channel allows n sends until it blocks. Using a channel will never create a new goroutine, its just a way send data. You can use a select statment to wait on sending or reciving from multiple channels or to send or get without being blocked by a full or empty channel.

In C you could put together a RW mutex, condition and a linked list or array and have the same thing pretty much.... If you want to create a goroutine you need to invoke "go X".

Out of curiosity, do you know C/C++ or Java?

If you have one thread in one namespace and another in another you now have to worry about what you can do in the context of a callback. This asymmetry just makes any multi-threaded program more complicated than it needs to be (and already is).

It is fair to say that without those names attached I would have likely passed it by. Google is boring, but those three names, for me at least, I had to take a look.

> Google & adoption

I think it helped get around the initial cycle of drawing in the curious. An argument counter to claims regarding the importance of the people involved would point out Plan9 etc. that are hardly widely known, much less adopted.

In sum, I think the role of Google in one becoming a Go programmer depends entirely on when it happened. Today, of course Go has its own brand name. 8 years ago, it was Google that eased the debutante phase of the language.

Like I said, the objections are practical, not philosophical:

They published four past design docs for generics in Go that simply didn't pass technical muster (https://github.com/golang/proposal/blob/master/design/15292-...)

Also, rsc stated he plans to understand generics better in 2017: https://research.swtch.com/go2017#generics

Also, bradfitz said recently on the GoTimeFM podcast that doing Generics and Go2 together makes sense.

> I didn't necessarily mean their marketing budget. I should probably edit that. I meant that anything they do is news.

I feel like the comments from others about Dart give the lie to this one. Go is fantastically, dramatically, massively more popular than Dart, which is also from Google.