The originaldbm library and file format was a simpledatabase engine, originally written byKen Thompson and released byAT&T in 1979. The name is athree-letter acronym forDataBase Manager, and can also refer to the family of database engines with APIs and features derived from the originaldbm.
Thedbm library stores arbitrary data by use of a single key (aprimary key) in fixed-size buckets and useshashing techniques to enable fast retrieval of the data by key.
The hashing scheme used is a form ofextendible hashing, so that the hashing scheme expands as new buckets are added to the database, meaning that, when nearly empty, the database starts with one bucket, which is then split when it becomes full. The two resulting child buckets will themselves split when they become full, so the database grows as keys are added.
Thedbm library and its derivatives are pre-relational databases – they manageassociative arrays, implemented as on-diskhash tables. In practice, they can offer a more practical solution for high-speed storage accessed by key, as they do not require the overhead of connecting and preparing queries. This is balanced by the fact that they can generally only be opened for writing by a single process at a time. An agentdaemon can handle requests from multiple processes, but introducesIPC overhead.
The original AT&Tdbm library has been replaced by its many successor implementations. Notable examples include:[3]
ndbm ("new dbm"), based on the original dbm with some new features.
GDBM ("GNU dbm"),GNU rewrite of the library implementingndbm features and its own interface. Also provides new features like crash tolerance for guaranteeing data consistency.[4][5]
sdbm ("small dbm"), apublic domain rewrite ofdbm. It is a part of the standard distribution forPerl and is available as an external library forRuby.[6][7]
qdbm ("Quick Database Manager"), a higher-performancedbm employing many of the same techniques as Tokyo/Kyoto Cabinet. Written by the same author before they moved on to the cabinets.[8]
tdb ("Trivial Database"), a simple database used bySamba that supports multiple writers. Has a gdbm-based API.[9]
Berkeley DB, 1991 replacement of ndbm bySleepycat Software (nowOracle) created to get around the AT&T Unix copyright onBSD. It features many extensions like parallelism, transactional control, hashing, andB-tree storage.
As of 2001, thendbm implementation of DBM was standard on Solaris andIRIX, whereasgdbm is ubiquitous onLinux. The Berkeley DB implementations were standard on some free operating systems.[2][10] After a change of licensing of the Berkeley DB toGNU AGPL in 2013, projects likeDebian have moved to LMDB.[11]
A 2018AFLfuzzing test against many DBM-family databases exposed many problems in implementations when it comes to corrupt or invalid database files. Onlyfreecdb byDaniel J. Bernstein showed no crashes. The authors of gdbm, tdb, and lmdb were prompt to respond. Berkeley DB fell behind due to the sheer amount of other issues;[10] the fixes would be irrelevant toopen-source software users due to the licensing change locking them back on an old version.[11]
^Kew 2007, p. 80: "DBMs have been with us since the early days of computing, when the need for fast keyed lookups was recognized. The original DBM is a UNIX-based library and file format for fast, highly-scalable keyed access to data. It was followed (in order) by NDBM ('new DBM'), GDBM ('GNU DBM'), and the Berkeley DB. This last is by far the most advanced, and the only DBM under active development today. Nevertheless, all of the DBMs from NDBM onward provide the same core functionality used by most programs, including Apache. A minimal-implementation SDBM is also bundled withAPR, and is available to applications along with the other DBMs. Although NDBM is now old - like the city named New Town ('Neapolis') by the Greeks in about 600BC and still called Naples today - it remains the baseline DBM. NDBM was used by early Apache modules such as the Apache 1.x versions ofmod_auth_dbm andmod_rewrite. Both GDBM and Berkeley DB provide NDBM emulations, and Linux distributions ship with one or other of these emulations in place of the 'real' NDBM, which is excluded for licensing reasons. Unfortunately, the various file formats are totally incompatible, and there are subtle differences in behaviour concerning database locking. These issues led a steady stream of Linux users to report problems with DBMs in Apache 1.x."
^abHazel 2001, p. 500: "The most common [single-key] format is called DBM. Most modern versions of Unix have a DBM library installed as standard, though this is not true of some older systems. The two most common DBM libraries arendbm (standard on Solaris and IRIX) and Berkeley DB Version 2 or 3 (standard on several free operating systems). Exim supports both of these, as well as the older Berkeley DB Version 1,gdbm, andtdb."
^abLadd & O'Donnell 2001, pp. 823–824: "Most UNIX systems have some kind of DBM database. DBM is a set of library routines that manages data files consisting of key and value pairs. The DBM routines control how users enter and retrieve information from the database. Although it isn't the most powerful mechanism for storing information, using DBM is a faster method of retrieving information than using a flat file. Because most UNIX sites use one of the DBM libraries, the tools you need to store your information in a DBM database are readily available. Almost as many flavors of the DBM libraries exist as there are UNIX systems. Although most of these libraries are compatible with each other, they all basically work the same way... A list follows of some of the most popular DBM libraries available:
DBM - DBM stores the database in two files. The first has the extension.Pag and contains the bitmap. The second, which has the extension.Dir, contains the data.
NDBM - NDBM is much like DBM but with a few additional features; it was written to provide better storage and retrieval methods. Also, NDBM enables you to open many databases, unlike DBM, in which you are allowed to have only one database open within your script. Like DBM, NDBM stores its information in two files using the extensions.Pag and.Dir.
SDBM - SDBM comes with the Perl archive, which has been ported to many platforms. Therefore, you can use DBM databases as long as a version of Perl exists for your computer. SDBM was written to match the functions provided with NDBM, so portability of code shouldn't be a problem. Perl is available on just about all popular platforms.
GDBM - GDBM is the GNU version of the DBM family of database routines. GDBM also enables you to cache data, reducing the time that it takes to write to the database. The database has no size limit; its size depends completely on your system's resources. GDBM database files have the extension.Db. Unlike DBM and NDBM, both of which use two files, GDBM only uses one file.
Berkeley db - The Berkeley db expands on the original DBM routines significantly. The Berkeley db uses hashed tables the same as the other DBM databases, but the library also can create databases based on a sorted balanced binary tree (BTREE) and store information with a record line number (RECNO). The method that you use depends completely on how you want to store and retrieve the information from a database. Berkeley db creates only one file, which has no extension."
^yigit, ozan."sdbm.bun".cse.yorku.ca. Retrieved8 May 2019.
^"Ruby SDBM library".SDBM on Github.Note that Ruby used to ship SDBM in the standard distribution up until version 2.7, after which it was made available only as an external library, similarly to the DBM and GDBM libraries, removed from the standard library in Ruby 3.1.
