Practical Garbage Collection - Part 1: Introduction

Practical Garbage Collection - Part 1: Introduction(worldmodscode.wordpress.com)

181 points by cgbystrom 14 years ago | 38 comments

bitops 14 years ago |

A very good writeup, but one thing always confuses me.

What is meant specifically by the "heap" and "stack"? I know what a stack is, but "heap" gets thrown around in many different contexts and I've yet to find any explanation that made it clear.

If anyone has a good explanation or good links for those two terms in this context, I'd be very grateful. Thanks!

[EDIT: thanks everyone for the answers so far!]

lmkg 14 years ago | |

This looks pretty well-answered already, but I'll throw in my $0.02.

The stack and heap are the two main regions of memory where data can live in a program. They are (somewhat unhelpfully) named after the data structures originally used to represent them. Pretty much all modern languages have a concept of a stack and a heap, although there are differences both in how explicit they are, and in implementation details.

The stack is the function call stack, used to store data local to a function call. It behaves like a traditional stack, with push and pop operations. When you call a function, a new stack frame is pushed onto the stack, which contains all of the local variables that are only accessible inside that function call. When that function returns, it pops its stack frame (and sends its return value, somehow). The language scoping rules determine how one function may access the contents of an earlier stack frame that called it, but stack frames are basically deleted and inaccessible after they are popped (a source of errors in C).

The heap is the region of memory where you put data that lives longer than the function that created it. Heap behavior is far less specified, and more variable, between languages than stack behavior is (and involves the OS more), but if you're passing around pointers or references to an object you're probably using the heap. In languages that have them, the malloc and new operators usually mean "put this on the heap." In languages with a notion of reference types and value types, reference types are usually heap-allocated by default.

The distinction between the two types of storage is more important in some languages that others. It's really important in C, where you have to manually manage all your memory, but some of the more dynamic languages (especially everything-is-an-object ones) use the heap for all user data, and the call stack only for implementation details.

Languages also vary greatly in how much control you have over where things get put. On one extreme, C lets you put any type in any location, with malloc, while on the other I am not aware of any way in JavaScript to control object lifetime. Java and C# both distinguish between value-types (stack-allocated by default) and reference-types (heap-allocated by default), but C# allows user-defined value-types while Java does not. Common Lisp has a standardized way to tell the compiler that it is safe to stack-allocate a value (of any type). And so on.

nhebb 14 years ago | | |

There are several things I've never had a good understanding of regarding the stack. How big is the stack? Is it a fixed size dictated by the architecture of the processor? I know I've written recursions that overflow it. Other than gut feel and catching an exception after the fact, how do programmers know when their program might overflow it?

bitops 14 years ago | | |

Just wanted to say thanks again for the thorough answer. This really made it clear for me.

Tuna-Fish 14 years ago | |

A stack is a data structure that grows and can be accessed from one end.

The stack usually means the call stack, or c stack, where one of the registers of the cpu is always pointed to the top of a stack structure in memory. That way it can be used to quickly allocate and free memory for function calls and function-scoped temporaries, and it has no trouble handling (limited) recursion.

Heap refers to the part of the memory that is fundamentally just a flat address space asked for from the OS. On top of that lives the heap allocator, whose job is to portion that into usable chunks and try to be able to reuse returned space. In C that is malloc/free, in c++ new and delete.

In the olden days heap and stack actually referred to the same area of memory, the heap growing upwards from the bottom and the stack growing down from the top. Luckily, today there is enough address space that we don't have to do this anymore.

kbolino 14 years ago | | |

AFAIK, heap and stack are still in the same "area of memory;" the discipline of separating the two is still enforced (or not) by the operating system. The heap-grows-up, stack-grows-down convention is still common on (32-bit, maybe 64-bit?) x86 too.

eternalban 14 years ago | |

Generally, the "heap" is the addressable memory assigned to the process. E.g. you malloc a byte array and gain a reference to the (byte[] rep.) memory object in heap. The life-cycle of a memory object in heap is explicitly controlled by OS (and sometimes the language e.g. C/Obj-C/etc via freeing the heap memory object). Languages with garbage collectors do this implicitly.

The "stack" is the dynamic structure -- its a stack :) -- that maintains the nested call/method/function frames from the initial entry point (e.g. main(..) in C family) so it maintains the path traversing the invocation/call graph to the current executing call/method/function. The 'objects' in a stack are call/function/method parameters, various metadata (e.g. obj ptr in some oo langauges). The life cycle of objects in stack is scoped by the call/function/method itself. Further, typically attempts are made to store (current) stack objects in CPU registers so as to avoid incurring memory access latencies.

faboo 14 years ago | |

Usually when people say "the heap" (especially in the context of garbage collection) they mean the memory where new (non-stack) data/objects are allocated from. Usually this is the bulk of memory an application uses.

There are other meanings of the word "heap" in the realm of data structures, but I haven't personally seen "heap" the data structure talked about (or used) much in the wild.

anamax 14 years ago | | |

> I haven't personally seen "heap" the data structure talked about (or used) much in the wild.

Maybe not, but the concept of computing with addresses is reasonably common.

That's why programming interviews often have some variation on heap sort or radix sort.

dons 14 years ago | |

In programming languages, generally:

* Stack: "control stack" -- a structure that tracks the context of an evaluation "step" in the program

* Heap: an environment that maps symbols to values (i.e a data structure that tracks the bindings of names to computations and results).

jws 14 years ago | |

Traditionally, the stack is a FILO (first in, last out) allocation area. Generally they start at the top of memory and work their way down as space is allocated, then back up as it is freed.

The heap might start at the low end of unused memory (above the program and libraries) and is used for allocating space with arbitrary release order. This means it has to keep track of what is used and what is available as well as when to "grow the heap" by allocating more pages of memory.

Garbage collection is one way of managing a heap.

pixie_ 14 years ago |

The complexity of generational garbage collection vs. the speed of manual collection, makes me feel like the happy medium of speed and simplicity is reference counting, like that found in objective-c. iPhone apps are fast, but take a bit longer to design, develop, and debug due to memory management issues. Though with experience these can be minimized.

It probably isn't possible without a ton of modification, but I wish the JVM/CLR had an option to garbage collect through reference counting.

stcredzero 14 years ago |

What are the long term implications of the increasing amount of parallel computing resources available to programmers? Are we getting to a point where there is enough excess CPU available, that the extra instruction on every assignment for reference counting is no big deal? (In certain contexts. In some contexts extra instructions are always a big deal, but these don't span all of computing.) Combine that with incremental algorithms for cycle reclamation, and you'd have great low-latency GC.

larsberg 14 years ago | |

It's still not trivial even with massive parallelism. Performance is easily killed if you:

1) Completely saturate the memory interconnect between the processors and RAM chips

2) If your parallel collector threads are touching the same memory that your worker threads are, you're going to have some cache contention

3) One of the big challenges with cache lines that have writes in them is if they are shared between processors. For example, create two ref cells and have two parallel threads set them -- if the compiler did not happen to pad them out to cache line size, they'll be fighting for the same 64 bytes (8 64-bit words/pointers) of memory

We (Manticore) have done a lot of work to isolate the per-CPU work, heap pages, etc. (http://arxiv.org/abs/1105.2554 , MSPC 2011 proceedings should be out on acm.org soon-ish). But it's not easy and requires rearchitecting your whole compiler and runtime system. GHC is also in the middle of some similar work, and they have a much harder problem both due to the challenges of implementing laziness and they have a full-featured system and real user base, so they can't just change things willy-nilly.

Further, once you get the parallelism right, you end up in a different situation: Amdahl's Law bites. Even if 90% of your program is amenable to parallel work, if you make that portion go infinitely fast, you still have a sequential portion that takes 10% of the time and therefore limits you to a maximum of 10x speedup.

We are hitting that limitation right now in our research and going back to do more sequential performance work. I've got a small army of undergraduates implementing compiler optimizations and analyses :-)

martincmartin 14 years ago | |

The the object can be accessed (not modified, just accessed) from multiple threads, the extra instruction needs to be a thread safe atomic increment, which is hugely expensive, and becomes more expensive the more cores you have, especially in a NUMA architecture.

Also, when an object is freed, you need to decrement the reference count of every single object it points to, even objects that are referenced from many other places and clearly won't be reclaimed for a long time.

ruggeri 14 years ago |

I enjoyed this basic introduction to GC; upvoted. What I'd look forward is further discussion of incremental and concurrent GC algorithms.

Until then, does anyone know what makes concurrent GC non-trivial? It seems like it shouldn't be too hard to trace concurrent to program execution. And if you don't compact, collection seems to just involve updating some structure tracking free blocks. I'd imagine it's possible to write a thread-safe version of that structure where every "free" request doesn't need to block every "malloc" request. But I must have missed something.

I'd also be interested to read how compaction works; how are references remapped from the old address to the new one? Is it possible that a reference value is a pointer to a reference "object" which contains the pointer to data, which needs to be updated? Then you only need to update a single pointer when moving data, but every dereferene incurs an extra layer of indirection.

dumael 14 years ago | |

Concurrent GC is somewhat non-trival as the mutator (i.e. the program you write) can delete references to objects from an area of the heap that has not been examined by the GC and introduce references to those same objects in an area that has been examined. This means those objects will be reclaimed since the GC never saw them.

To cope with this the mutator is modified at compile/run time to inform the GC of object updates that could lead to this situation.

During compaction of any sort, the first time an object is encountered that is to be moved it is copied somewhere else and the old copy's header is overwritten with it's forwarding address. Every time the GC encounters a reference to the old object, it re-writes the reference to the old object with it's new location (conveniently located in the old copy's header).

> Is it possible that a reference value is a pointer to a reference "object" which contains the pointer to data, which needs to be updated?

Look up something called 'Brook's style forwarding pointer', it is essentially what you've described.

ruggeri 14 years ago | | |

Thanks; upvoted! I think I'm beginning to appreciate that concurrent marking is tougher than I thought; specifically, I can see how it can be hard to prove an object is unreachable. So I imagine the marking side is where the difficulties are.

Your explanation for compaction makes perfect sense. Of course, this won't work trivially concurrently. Only if you stop the world can you complete examination of the entire live heap and know you've updated all references to the moved object and can collect the original space.

valyala 14 years ago |

While the article is interesting, it skips important things, which have high influence on practical GC speed - write barriers and finalizers. The following ancient article from Microsoft has better coverage of GC internals http://msdn.microsoft.com/en-us/library/ms973837.aspx (somewhat biased to .NET :) ).

rubashov 14 years ago |

> The default choice of garbage collector in Hotspot is the throughput collector, which is ... entirely optimized for throughput

I just want to confirm this is true? Say you're doing a long running simulation. You don't care about pauses at all. You just want it to finish fast. The default GC with no particular options is the way to go?