Repository files navigation

This is version 7.5.0 (next release development) of a conservative garbagecollector for C and C++.

You might find a more recent versionhere, orhere.

This is intended to be a general purpose, garbage collecting storageallocator. The algorithms used are described in:

• Boehm, H., and M. Weiser, "Garbage Collection in an UncooperativeEnvironment", Software Practice & Experience, September 1988, pp. 807-820.

• Boehm, H., A. Demers, and S. Shenker, "Mostly Parallel Garbage Collection",Proceedings of the ACM SIGPLAN '91 Conference on Programming Language Designand Implementation, SIGPLAN Notices 26, 6 (June 1991), pp. 157-164.

• Boehm, H., "Space Efficient Conservative Garbage Collection", Proceedingsof the ACM SIGPLAN '91 Conference on Programming Language Design andImplementation, SIGPLAN Notices 28, 6 (June 1993), pp. 197-206.

• Boehm H., "Reducing Garbage Collector Cache Misses", Proceedings of the2000 International Symposium on Memory Management.

Possible interactions between the collector and optimizing compilers arediscussed in

• Boehm, H., and D. Chase, "A Proposal for GC-safe C Compilation",The Journal of C Language Translation 4, 2 (December 1992).

and

• Boehm H., "Simple GC-safe Compilation", Proceedings of the ACM SIGPLAN '96Conference on Programming Language Design and Implementation.

Unlike the collector described in the second reference, this collectoroperates either with the mutator stopped during the entire collection(default) or incrementally during allocations. (The latter is supportedon fewer machines.) On the most common platforms, it can be builtwith or without thread support. On a few platforms, it can take advantageof a multiprocessor to speed up garbage collection.

Many of the ideas underlying the collector have previously been exploredby others. Notably, some of the run-time systems developed at Xerox PARCin the early 1980s conservatively scanned thread stacks to locate possiblepointers (cf. Paul Rovner, "On Adding Garbage Collection and Runtime Typesto a Strongly-Typed Statically Checked, Concurrent Language" Xerox PARCCSL 84-7). Doug McIlroy wrote a simpler fully conservative collector thatwas part of version 8 UNIX (tm), but appears to not have receivedwidespread use.

Rudimentary tools for use of the collector as aleak detector are included,as is a fairly sophisticated string package "cord" that makes use of thecollector. (See doc/README.cords and H.-J. Boehm, R. Atkinson, and M. Plass,"Ropes: An Alternative to Strings", Software Practice and Experience 25, 12(December 1995), pp. 1315-1330. This is very similar to the "rope" packagein Xerox Cedar, or the "rope" package in the SGI STL or the g++ distribution.)

Further collector documentation can be foundhere.

This is a garbage collecting storage allocator that is intended to beused as a plug-in replacement for C's malloc.

Since the collector does not require pointers to be tagged, it does notattempt to ensure that all inaccessible storage is reclaimed. However,in our experience, it is typically more successful at reclaiming unusedmemory than most C programs using explicit deallocation. Unlike manuallyintroduced leaks, the amount of unreclaimed memory typically staysbounded.

In the following, an "object" is defined to be a region of memory allocatedby the routines described below.

Any objects not intended to be collected must be pointed to eitherfrom other such accessible objects, or from the registers,stack, data, or statically allocated bss segments. Pointers fromthe stack or registers may point to anywhere inside an object.The same is true for heap pointers if the collector is compiled withALL_INTERIOR_POINTERS defined, orGC_all_interior_pointers is otherwiseset, as is now the default.

Compiling withoutALL_INTERIOR_POINTERS may reduce accidental retentionof garbage objects, by requiring pointers from the heap to the beginningof an object. But this no longer appears to be a significantissue for most programs occupying a small fraction of the possibleaddress space.

There are a number of routines which modify the pointer recognitionalgorithm.GC_register_displacement allows certain interior pointersto be recognized even ifALL_INTERIOR_POINTERS is nor defined.GC_malloc_ignore_off_page allows some pointers into the middle oflarge objects to be disregarded, greatly reducing the probability ofaccidental retention of large objects. For most purposes it seemsbest to compile withALL_INTERIOR_POINTERS and to useGC_malloc_ignore_off_page if you get collector warnings fromallocations of very large objects. See doc/debugging.html for details.

WARNING: pointers inside memory allocated by the standardmalloc are notseen by the garbage collector. Thus objects pointed to only from such aregion may be prematurely deallocated. It is thus suggested that thestandardmalloc be used only for memory regions, such as I/O buffers, thatare guaranteed not to contain pointers to garbage collectible memory.Pointers in C language automatic, static, or register variables,are correctly recognized. (Note thatGC_malloc_uncollectable hassemantics similar to standard malloc, but allocates objects that aretraced by the collector.)

WARNING: the collector does not always know how to find pointers in dataareas that are associated with dynamic libraries. This is easy toremedy IF you know how to find those data areas on your operatingsystem (seeGC_add_roots). Code for doing this under SunOS, IRIX5.X and 6.X, HP/UX, Alpha OSF/1, Linux, and win32 is included and usedby default. (See doc/README.win32 for Win32 details.) On other systemspointers from dynamic library data areas may not be considered by thecollector. If you're writing a program that depends on the collectorscanning dynamic library data areas, it may be a good idea to includeat least one call toGC_is_visible to ensure that those areas arevisible to the collector.

Note that the garbage collector does not need to be informed of sharedread-only data. However if the shared library mechanism can introducediscontiguous data areas that may contain pointers, then the collector doesneed to be informed.

Signal processing for most signals may be deferred during collection,and during uninterruptible parts of the allocation process.Like standard ANSI C mallocs, by default it is unsafe to invokemalloc (and other GC routines) from a signal handler while anothermalloc call may be in progress.

The allocator/collector can also be configured for thread-safe operation.(Full signal safety can also be achieved, but only at the cost of two systemcalls per malloc, which is usually unacceptable.)

WARNING: the collector does not guarantee to scan thread-local storage(e.g. of the kind accessed withpthread_getspecific). The collectordoes scan thread stacks, though, so generally the best solution is toensure that any pointers stored in thread-local storage are alsostored on the thread's stack for the duration of their lifetime.(This is arguably a longstanding bug, but it hasn't been fixed yet.)

As distributed, the collector operates silentlyIn the event of problems, this can usually be changed by defining theGC_PRINT_STATS orGC_PRINT_VERBOSE_STATS environment variables. Thiswill result in a few lines of descriptive output for each collection.(The given statistics exhibit a few peculiarities.Things don't appear to add up for a variety of reasons, most notablyfragmentation losses. These are probably much more significant for thecontrived program "test.c" than for your application.)

On most Unix-like platforms, the collector can be built either using aGNU autoconf-based build infrastructure (type./configure; make in thesimplest case), or with a classic makefile by itself (typemake -f Makefile.direct).

Please note that the collector source repository does not contain configureand similar auto-generated files, thus the full procedure of autoconf-basedbuild ofmaster branch of the collector (usingmaster branch oflibatomic_ops source repository as well) could look like:

git clone git://github.com/ivmai/bdwgc.gitcd bdwgcgit clone git://github.com/ivmai/libatomic_ops.gitautoreconf -vifautomake --add-missing./configuremakemake check

Below we focus on the collector build using classic makefile.For the Makefile.direct-based process, typingmake test instead ofmakewill automatically build the collector and then runsetjmp_test andgctest.Setjmp_test will give you information about configuring the collector, which isuseful primarily if you have a machine that's not already supported. Gctest isa somewhat superficial test of collector functionality. Failure is indicatedby a core dump or a message to the effect that the collector is broken. Gctesttakes about a second to two to run on reasonable 2007 vintage desktops. It mayuse up to about 30MB of memory. (The multi-threaded version will use more.64-bit versions may use more.)make test will also, as its last step, attemptto build and test the "cord" string library.)

Makefile.direct will generate a library gc.a which you should link against.Typing "make cords" will add the cord library to gc.a.

The GNU style build process understands the usual targets.make checkruns a number of tests.make install installs at least libgc, and libcord.Try./configure --help to see the configuration options. It is currentlynot possible to exercise all combinations of build options this way.

It is suggested that if you need to replace a piece of the collector(e.g. GC_mark_rts.c) you simply list your version ahead of gc.a on theld command line, rather than replacing the one in gc.a. (This willgenerate numerous warnings under some versions of AIX, but it stillworks.)

All include files that need to be used by clients will be put in theinclude subdirectory. (Normally this is just gc.h.make cords adds"cord.h" and "ec.h".)

The collector currently is designed to run essentially unmodified onmachines that use a flat 32-bit or 64-bit address space.That includes the vast majority of Workstations and X86 (X >= 3) PCs.(The list here was deleted because it was getting too long and constantlyout of date.)

In a few cases (Amiga, OS/2, Win32, MacOS) a separate makefileor equivalent is supplied. Many of these have separate README.systemfiles.

Dynamic libraries are completely supported only under SunOS/Solaris,(and even that support is not functional on the last Sun 3 release),Linux, FreeBSD, NetBSD, IRIX 5&6, HP/UX, Win32 (not Win32S) and OSF/1on DEC AXP machines plus perhaps a few others listed near the topof dyn_load.c. On other machines we recommend that you do one ofthe following:

1. Add dynamic library support (and send us the code).
2. Use static versions of the libraries.
3. Arrange for dynamic libraries to use the standard malloc.This is still dangerous if the library stores a pointer to agarbage collected object. But nearly all standard interfacesprohibit this, because they deal correctly with pointersto stack allocated objects. (Strtok is an exception. Don'tuse it.)

In all cases we assume that pointer alignment is consistent with thatenforced by the standard C compilers. If you use a nonstandard compileryou may have to adjust the alignment parameters defined in gc_priv.h.Note that this may also be an issue with packed records/structs, if thoseenforce less alignment for pointers.

A port to a machine that is not byte addressed, or does not use 32 bitor 64 bit addresses will require a major effort. A port to plain MSDOSor win16 is hard.

For machines not already mentioned, or for nonstandard compilers,some porting suggestions are provided in doc/porting.html.

The following routines are intended to be directly called by the user.Note that usually onlyGC_malloc is necessary.GC_clear_roots andGC_add_roots calls may be required if the collector has to tracefrom nonstandard places (e.g. from dynamic library data areas on amachine on which the collector doesn't already understand them.) Onsome machines, it may be desirable to setGC_stacktop to a goodapproximation of the stack base. (This enhances code portability onHP PA machines, since there is no good way for the collector tocompute this value.) Client code may include "gc.h", which definesall of the following, plus many others.

1. GC_malloc(nbytes)

   • Allocate an object of size nbytes. Unlike malloc, the object iscleared before being returned to the user.GC_malloc willinvoke the garbage collector when it determines this to be appropriate.GC_malloc may return 0 if it is unable to acquire sufficientspace from the operating system. This is the most probableconsequence of running out of space. Other possible consequencesare that a function call will fail due to lack of stack space,or that the collector will fail in other ways because it cannotmaintain its internal data structures, or that a crucial systemprocess will fail and take down the machine. Most of thesepossibilities are independent of the malloc implementation.

2. GC_malloc_atomic(nbytes)

   • Allocate an object of size nbytes that is guaranteed not to contain anypointers. The returned object is not guaranteed to be cleared.(Can always be replaced byGC_malloc, but results in faster collectiontimes. The collector will probably run faster if large characterarrays, etc. are allocated withGC_malloc_atomic than if they arestatically allocated.)

3. GC_realloc(object, new_size)

   • Change the size of object to benew_size. Returns a pointer to thenew object, which may, or may not, be the same as the pointer tothe old object. The new object is taken to be atomic if and only if theold one was. If the new object is composite and larger than the originalobject,then the newly added bytes are cleared (we hope). This is verylikely to allocate a new object, unlessMERGE_SIZES is defined ingc_priv.h. Even then, it is likely to recycle the old object only if theobject is grown in small additive increments (which, we claim, isgenerally bad coding practice.)

4. GC_free(object)

   • Explicitly deallocate an object returned byGC_malloc orGC_malloc_atomic. Not necessary, but can be used to minimizecollections if performance is critical. Probably a performanceloss for very small objects (<= 8 bytes).

5. GC_expand_hp(bytes)

   • Explicitly increase the heap size. (This is normally done automaticallyif a garbage collection failed toGC_reclaim enough memory. Explicitcalls toGC_expand_hp may prevent unnecessarily frequent collections atprogram startup.)

6. GC_malloc_ignore_off_page(bytes)

   • Identical toGC_malloc, but the client promises to keep a pointer tothe somewhere within the first 256 bytes of the object while it islive. (This pointer should normally be declared volatile to preventinterference from compiler optimizations.) This is the recommendedway to allocate anything that is likely to be larger than 100 Kbytesor so. (GC_malloc may result in failure to reclaim such objects.)

7. GC_set_warn_proc(proc)

   • Can be used to redirect warnings from the collector. Such warningsshould be rare, and should not be ignored during code development.

8. GC_enable_incremental()

   • Enables generational and incremental collection. Useful for largeheaps on machines that provide access to page dirty information.Some dirty bit implementations may interfere with debugging(by catching address faults) and place restrictions on heap argumentsto system calls (since write faults inside a system call may not behandled well).

9. Several routines to allow for registration of finalization code.User supplied finalization code may be invoked when an object becomesunreachable. To call(*f)(obj, x) when obj becomes inaccessible, useGC_register_finalizer(obj, f, x, 0, 0);For more sophisticated uses, and for finalization ordering issues,see gc.h.

The global variableGC_free_space_divisor may be adjusted up from itdefault value of 3 to use less space and more collection time, or down forthe opposite effect. Setting it to 1 will almost disable collectionsand cause all allocations to simply grow the heap.

The variableGC_non_gc_bytes, which is normally 0, may be changed to reflectthe amount of memory allocated by the above routines that should not beconsidered as a candidate for collection. Careless use may, of course, resultin excessive memory consumption.

Some additional tuning is possible through the parameters definednear the top of gc_priv.h.

If onlyGC_malloc is intended to be used, it might be appropriate to define:

#define malloc(n) GC_malloc(n)#define calloc(m,n) GC_malloc((m)*(n))

For small pieces of VERY allocation intensive code, gc_inl.h includessome allocation macros that may be used in place ofGC_malloc andfriends.

All externally visible names in the garbage collector start withGC_.To avoid name conflicts, client code should avoid this prefix, except whenaccessing garbage collector routines or variables.

There are provisions for allocation with explicit type information.This is rarely necessary. Details can be found in gc_typed.h.

The Ellis-Hull C++ interface to the collector is included inthe collector distribution. If you intend to use this, typemake c++ after the initial build of the collector is complete.See gc_cpp.h for the definition of the interface. This interfacetries to approximate the Ellis-Detlefs C++ garbage collectionproposal without compiler changes.

Very often it will also be necessary to use gc_allocator.h and theallocator declared there to construct STL data structures. Otherwisesubobjects of STL data structures will be allocated using a systemallocator, and objects they refer to may be prematurely collected.

The collector may be used to track down leaks in C programs that areintended to run with malloc/free (e.g. code with extreme real-time orportability constraints). To do so defineFIND_LEAK in Makefile.This will cause the collector to invoke thereport_leakroutine defined near the top of reclaim.c whenever an inaccessibleobject is found that has not been explicitly freed. Such objects willalso be automatically reclaimed.

If all objects are allocated withGC_DEBUG_MALLOC (see next section), thenthe default version of report_leak will report at least the source file andline number at which the leaked object was allocated. This may sometimes besufficient. (On a few machines, it will also report a cryptic stack trace.If this is not symbolic, it can sometimes be called into a symbolic stacktrace by invoking program "foo" with "tools/callprocs.sh foo". It is a shortshell script that invokes adb to expand program counter values to symbolicaddresses. It was largely supplied by Scott Schwartz.)

Note that the debugging facilities described in the next section cansometimes be slightly LESS effective in leak finding mode, since inleak finding mode,GC_debug_free actually results in reuse of the object.(Otherwise the object is simply marked invalid.) Also note that the testprogram is not designed to run meaningfully inFIND_LEAK mode.Use "make gc.a" to build the collector.

The routinesGC_debug_malloc,GC_debug_malloc_atomic,GC_debug_realloc,andGC_debug_free provide an alternate interface to the collector, whichprovides some help with memory overwrite errors, and the like.Objects allocated in this way are annotated with additionalinformation. Some of this information is checked during garbagecollections, and detected inconsistencies are reported to stderr.

Simple cases of writing past the end of an allocated object shouldbe caught if the object is explicitly deallocated, or if thecollector is invoked while the object is live. The first deallocationof an object will clear the debugging info associated with anobject, so accidentally repeated calls toGC_debug_free will report thedeallocation of an object without debugging information. Out ofmemory errors will be reported to stderr, in addition to returningNULL.

GC_debug_malloc checking during garbage collection is enabledwith the first call toGC_debug_malloc. This will result in someslowdown during collections. If frequent heap checks are desired,this can be achieved by explicitly invokingGC_gcollect, e.g. fromthe debugger.

GC_debug_malloc allocated objects should not be passed toGC_reallocorGC_free, and conversely. It is however acceptable to allocate onlysome objects withGC_debug_malloc, and to useGC_malloc for other objects,provided the two pools are kept distinct. In this case, there is a verylow probability thatGC_malloc allocated objects may be misidentified ashaving been overwritten. This should happen with probability at mostone in 2**32. This probability is zero ifGC_debug_malloc is never called.

GC_debug_malloc,GC_malloc_atomic, andGC_debug_realloc take twoadditional trailing arguments, a string and an integer. These are notinterpreted by the allocator. They are stored in the object (the string isnot copied). If an error involving the object is detected, they are printed.

The macrosGC_MALLOC,GC_MALLOC_ATOMIC,GC_REALLOC,GC_FREE, andGC_REGISTER_FINALIZER are also provided. These require the same argumentsas the corresponding (nondebugging) routines. If gc.h is includedwithGC_DEBUG defined, they call the debugging versions of thesefunctions, passing the current file name and line number as the twoextra arguments, where appropriate. If gc.h is included withoutGC_DEBUGdefined, then all these macros will instead be defined to their nondebuggingequivalents. (GC_REGISTER_FINALIZER is necessary, since pointers toobjects with debugging information are really pointers to a displacementof 16 bytes form the object beginning, and some translation is necessarywhen finalization routines are invoked. For details, about what's storedin the header, see the definition of the type oh in debug_malloc.c)

The collector normally interrupts client code for the duration ofa garbage collection mark phase. This may be unacceptable if interactiveresponse is needed for programs with large heaps. The collectorcan also run in a "generational" mode, in which it usually attempts tocollect only objects allocated since the last garbage collection.Furthermore, in this mode, garbage collections run mostly incrementally,with a small amount of work performed in response to each of a large number ofGC_malloc requests.

This mode is enabled by a call toGC_enable_incremental.

Incremental and generational collection is effective in reducingpause times only if the collector has some way to tell which objectsor pages have been recently modified. The collector uses two sourcesof information:

1. Information provided by the VM system. This may be provided inone of several forms. Under Solaris 2.X (and potentially under othersimilar systems) information on dirty pages can be read from the/proc file system. Under other systems (currently SunOS4.X) it ispossible to write-protect the heap, and catch the resulting faults.On these systems we require that system calls writing to the heap(other than read) be handled specially by client code.See os_dep.c for details.

2. Information supplied by the programmer. We define "stubborn"objects to be objects that are rarely changed. Such an objectcan be allocated (and enabled for writing) withGC_malloc_stubborn.Once it has been initialized, the collector should be informed witha call toGC_end_stubborn_change. Subsequent writes that storepointers into the object must be preceded by a call toGC_change_stubborn.

This mechanism performs best for objects that are written only forinitialization, and such that only one stubborn object is writableat once. It is typically not worth using for short-livedobjects. Stubborn objects are treated less efficiently than pointer-free(atomic) objects.

A rough rule of thumb is that, in the absence of VM information, garbagecollection pauses are proportional to the amount of pointerful storageplus the amount of modified "stubborn" storage that is reachable duringthe collection.

Initial allocation of stubborn objects takes longer than allocationof other objects, since other data structures need to be maintained.

We recommend against random use of stubborn objects in clientcode, since bugs caused by inappropriate writes to stubborn objectsare likely to be very infrequently observed and hard to trace.However, their use may be appropriate in a few carefully writtenlibrary routines that do not make the objects themselves availablefor writing by client code.

Any memory that does not have a recognizable pointer to it will bereclaimed. Exclusive-or'ing forward and backward links in a listdoesn't cut it.

Some C optimizers may lose the last undisguised pointer to a memoryobject as a consequence of clever optimizations. This has almostnever been observed in practice.

This is not a real-time collector. In the standard configuration,percentage of time required for collection should be constant acrossheap sizes. But collection pauses will increase for larger heaps.They will decrease with the number of processors if parallel markingis enabled.

(On 2007 vintage machines, GC times may be on the order of 5 msecsper MB of accessible memory that needs to be scanned and processor.Your mileage may vary.) The incremental/generational collection facilitymay help in some cases.

Please address bug reportshere.If you are contemplating a major addition, you might also send mail to askwhether it's already been done (or whether we tried and discarded it).

• Copyright (c) 1988, 1989 Hans-J. Boehm, Alan J. Demers
• Copyright (c) 1991-1996 by Xerox Corporation. All rights reserved.
• Copyright (c) 1996-1999 by Silicon Graphics. All rights reserved.
• Copyright (c) 1999-2011 by Hewlett-Packard Development Company.

The file linux_threads.c is also

• Copyright (c) 1998 by Fergus Henderson. All rights reserved.

The files Makefile.am, and configure.in are

• Copyright (c) 2001 by Red Hat Inc. All rights reserved.

Several files supporting GNU-style builds are copyrighted by the FreeSoftware Foundation, and carry a different license from that givenbelow. The files included in the libatomic_ops distribution (includedhere) use either the license below, or a similar MIT-style license,or, for some files not actually used by the garbage-collector library, theGPL.

THIS MATERIAL IS PROVIDED AS IS, WITH ABSOLUTELY NO WARRANTY EXPRESSEDOR IMPLIED. ANY USE IS AT YOUR OWN RISK.

Permission is hereby granted to use or copy this programfor any purpose, provided the above notices are retained on all copies.Permission to modify the code and to distribute modified code is granted,provided the above notices are retained, and a notice that the code wasmodified is included with the above copyright notice.

A few of the files needed to use the GNU-style build procedure come withslightly different licenses, though they are all similar in spirit. A feware GPL'ed, but with an exception that should cover all uses in thecollector. (If you are concerned about such things, I recommend you lookat the notice in config.guess or ltmain.sh.)

The atomic_ops library contains some code that is covered by the GNU GeneralPublic License, but is not needed by, nor linked into the collector library.It is included here only because the atomic_ops distribution is, forsimplicity, included in its entirety.