Rewriting the Buildbox Agent in Go

Rewriting the Buildbox Agent in Go(buildbox.io)

43 points by keithpitt 12 years ago | 6 comments

gecko 12 years ago |

I'm not familiar with the Ruby ecosystem anymore, so this is a genuine question: does the memory usage improvement here indicate a typical Ruby workload; if so, is this optimal, or are there profiling tools that can meaningfully help; and how much of these gains are the Ruby garbage collector v. the Go garbage collector?

Part of my curiosity here is that Go's GC isn't yet very good, so I'm assuming that something about typical Ruby patterns results in a lot of not-garbage on the heap, but I'm also wondering if this could simply reflect bad design.

aaronblohowiak 12 years ago | |

You can FAR more easily avoid generating garbage in Go than in Ruby.

In Go, it would be very awkward to use map[string]interface{} everywhere, whereas that is VERY common in ruby. sizeof(map) > sizeof(struct).

Ruby stores all of the source of all of the files at runtime in memory, and IIRC, the AST nodes themselves are still run through the GC. If you have a load of dependencies, this is "fun".

I believe everything in Ruby is heap-allocated. The profiling tools for Ruby can help you identify "leaks", by telling you the classes of live objects. In go, you can get a dump of the number of allocations attributable to each line of code.

Ruby has the interesting quirk that every interned string (symbol) is retained forever.

With Go, you can set a quick flag that will force the GC to run after the percentage of new allocations exceeds a certain ratio. With ruby, you can tweak the GC with certain compile and environment flags, but this is not regularly practiced by the community and is ruby version dependent.

artursapek 12 years ago | | |

I've been happily using Go to build a side project for a while, but I don't do anything special to profile the memory usage or optimize it. Do you have any resources you recommend for learning more about it? You seem to know a good bit.

densone 12 years ago |

Loving this line. Always nice to see a crafty/simple way to check for many different http responses.

https://github.com/buildboxhq/buildbox-agent/blob/master/bui...