Go channels, goroutines and GC available in Nim

Go channels, goroutines and GC available in Nim(forum.nim-lang.org)

147 points by pwernersbach 11 years ago | 83 comments

stcredzero 11 years ago |

This document describes how the GC works and how to tune it for (soft) realtime systems.

The basic algorithm is Deferred Reference Counting with cycle detection.

I'm sitting here feeling very impressed. Deferred reference counting has very good semantics for games. Even better, you can control the cycle detection part separately and run that part at an advantageous time. (Though it's probably better to just let GC do its thing, unless you really know what you're doing.)

I am currently writing a multiplayer game server in golang, by making sure almost everything is allocated on the stack, and heap sizes are small. This gives me an efficient, nearly pauseless server. However, something like Nim could give me even more flexibility.

pcwalton 11 years ago | |

> I'm sitting here feeling very impressed. Deferred reference counting has very good semantics for games.

Not if you need to be thread-safe.

> Even better, you can control the cycle detection part separately and run that part at an advantageous time.

How would that work with multithreading? (Assuming you had a thread-safe GC, which Nim's isn't.)

stcredzero 11 years ago | | |

Not if you need to be thread-safe.

Everything that has to be thread-safe uses channels. I use channels to sanitize everything for a purely synchronous game loop. As an optimization, in cases where there are atomic operations available, there are places where concurrent code can mutate values visible to the game loop, but this is strictly an optimization technique, to be used judiciously. (So only values like "speed" can use this technique. Anything that's a reference is verboten.)

How would that work with multithreading? (Assuming you had a thread-safe GC, which Nim's isn't.)

I didn't realize Nim's GC wasn't thread safe. In my current architecture, you'd only have to worry about the part using channels to sanitize things for the synchronous game loop. If everything outside of the game loop was written such that most everything was allocated on the stack, the GC would never have to collect anything outside of the game loop. So maybe it could work as a port. I couldn't say for sure, though.

Pharohbot 11 years ago | | |

A more general approach to shared memory via lockable heaps is a feature Nim seems will implement soon.

vvanders 11 years ago | | |

If you're crossing threads in games stuff you're doing it wrong. There's a good chance you're running into false-sharing and other pitfalls.

Most games use worker-queues(in which case you can use non-GC objects) to deal with architectures like the CELL and for better cache coherency. In that case Nim is a pretty good fir.

fjehcjriendje 11 years ago | | |

So, I've noticed that over the past six months that every time something about Nim gets posted to HN, you make an effort to discredit the language.

Care to offer an explanation why?

kodablah 11 years ago |

I always wanted to see a Go/Nim interop project in a little different vein. Since Nim is a superset of Go (and Go is fairly simple), why not a Go implementation in Nim via cross-compilation? Then you can even build a Nim macro that does a "Go get" and Nim developers can use Golang libraries right in their code as imports. Not to mention we get a full, alternative Go impl.

It should be easy to bootstrap by first using Go's parser to dump some AST that Nim code can then translate, and then once that version is done, just reference the Go parser using this new impl. The only real struggle with the project from what I can see is deciding which standard library items to use the Go transpiled implementation (probably most of them) vs which need to have Nim backends (e.g. runtime package). Meh, just rambling thoughts in my head I was considering playing with given the time...

duaneb 11 years ago | |

> Since Nim is a superset of Go

I seriously doubt that Nim is out of the box binary compatible with another runtime's fundamental types.

kodablah 11 years ago | | |

I am not looking for binary compatibility. I am looking for feature parity. Go interfaces can be handled with concepts [1] or just macros. I am talking about an alternative Go implementation (i.e. just sitting on top of Nim), not ABI compat between the two runtimes.

1 - http://nim-lang.org/docs/manual.html#generics-concepts

aikah 11 years ago | |

> Since Nim is a superset of Go

what ?

kodablah 11 years ago | | |

I was under the impression this was true feature-set wise. I may be wrong though and am happy to be corrected.

Manishearth 11 years ago |

This is awesome! Go's channels and green threads were one of the features I really liked about Go. One more reason for me to go past basic prime number programs in Nim :)

I'm hoping to see Rust get green threads/tasks/goroutines too. I'm working on a GC myself, and hopefully someone is trying out a green thread scheduler.

bmurphy1976 11 years ago |

I'm very curious to hear the pros and cons of this from somebody who has intimate knowledge of Nim internals. I really really like Go's CSP model and would love to see it properly supported in another lightweight non-jvm language (yes I know about Erlang, it doesn't fit the bill for me).

vbit 11 years ago | |

Well you can already use Nim threads (http://nim-lang.org/docs/threads.html) and channels (http://nim-lang.org/docs/channels.html) - the model is similar although the implementation uses system threads rather than coroutines.

Nim also has lightweight coroutines using `async` and `await` (http://nim-lang.org/docs/asyncdispatch.html) - you can run a bunch of these within one thread.

Also, have a look at gevent for Python.

rspeer 11 years ago | | |

Don't look at gevent for Python, look at a responsible async library: asyncio if you like Py3, Twisted if you like Py2 and want something that's very featureful but old-school, Trollius if you wish you could be using asyncio but you're stuck on Py2.

gevent is too much magic. It aims to give you async without changing any of your code. To accomplish this, it monkey-patches the entire Python standard library, in a way that is 99% compatible with Python, but the 1% will constantly surprise and infuriate you. Its compatibility shows no signs of increasing given how much its development has slowed down.

You can use gevent as a quick hack, but you will hate yourself if you have to maintain gevent code.

dom96 11 years ago | | |

In addition to the heavyweight threads (http://nim-lang.org/docs/threads.html) there is also `spawn` (http://nim-lang.org/docs/manual.html#parallel-spawn-spawn-st..., http://nim-lang.org/docs/threadpool.html)

digitalzombie 11 years ago | |

Ada have a CSP model too, just fyi.

It's a mature language. You can look into that?

jeremyjh 11 years ago |

I thought the Go runtime runs foreign code on M:M threads; e.g. when Go time calls foreign code, it dedicates a thread to it. This is so foreign libraries (which are unaware of yielding to the Go scheduler when they do I/O) don't block a thread with multiple goroutines scheduled on it. I do not think Nim code can run "in a goroutine".

stefantalpalaru 11 years ago | |

> I do not think Nim code can run "in a goroutine".

It does. Don't forget that this is gccgo so it is possible to use plain C functions as goroutines. Nim is translated to C and with the help of a macro I convert Nim functions with an arbitrary number of arguments into ones with a single void* arg that gccgo wants for its 'go' keyword implementation:

    extern void* __go_go(void (*f)(void *), void *);

jeremyjh 11 years ago | | |

Yes, but when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? If it doesn't, then nothing prevents foreign code from blocking other goroutines. How does the Nim code yield the thread for other goroutines, does it have to register a callback?