Repository files navigation

CubDB is an embedded key-value database for the Elixir language. It isdesigned for robustness, and for minimal need of resources.

Head to theAPI reference for usagedetails, or read theFrequently AskedQuestions and theHow Tosection for more information.

• Both keys and values can be any Elixir (or Erlang) term.

• Basicget,put, anddelete operations, selection of ranges of entriessorted by key withselect.

• Atomic, Consistent, Isolated, Durable (ACID) transactions.

• Multi version concurrency control (MVCC) allowing concurrent readoperations, that do not block nor are blocked by writes.

• Unexpected shutdowns or crashes won't corrupt the database or breakatomicity of transactions.

• Manual or automatic compaction to reclaim disk space.

To ensure consistency, performance, and robustness to data corruption,CubDBdatabase file uses an append-only, immutable B-tree data structure. Entries arenever changed in-place, and read operations are performed on zero cost immutablesnapshots.

StartCubDB by specifying a directory for its database file (if not existing,it will be created):

{:ok,db}=CubDB.start_link(data_dir:"my/data/directory")

Important {: .warning}

Avoid starting multipleCubDB processes on the same datadirectory. Only oneCubDB process should use a specific data directory at anytime.

get,put, anddelete operations work as you probably expect:

CubDB.put(db,:foo,"some value")#=> :okCubDB.get(db,:foo)#=> "some value"CubDB.delete(db,:foo)#=> :okCubDB.get(db,:foo)#=> nil

Multiple operations can be performed atomically with thetransaction functionand theCubDB.Tx module:

# Swapping `:a` and `:b` atomically:CubDB.transaction(db,fntx->a=CubDB.Tx.get(tx,:a)b=CubDB.Tx.get(tx,:b)tx=CubDB.Tx.put(tx,:a,b)tx=CubDB.Tx.put(tx,:b,a){:commit,tx,:ok}end)#=> :ok

Alternatively, it is possible to useput_multi,delete_multi, and the other[...]_multi functions, which also guarantee atomicity:

CubDB.put_multi(db,[a:1,b:2,c:3,d:4,e:5,f:6,g:7,h:8])#=> :ok

Range of entries sorted by key are retrieved usingselect:

CubDB.select(db,min_key::b,max_key::e)|>Enum.to_list()#=> [b: 2, c: 3, d: 4, e: 5]

Theselect function can select entries in normal or reverse order, and returnsa lazy stream, so one can use functions in theStream andEnum modules tomap, filter, and transform the result, only fetching from the database therelevant entries:

# Take the sum of the last 3 even values:CubDB.select(db,reverse:true)# select entries in reverse order|>Stream.map(fn{_key,value}->valueend)# discard the key and keep only the value|>Stream.filter(fnvalue->is_integer(value)&&Integer.is_even(value)end)# filter only even integers|>Stream.take(3)# take the first 3 values|>Enum.sum()# sum the values#=> 18

Read-only snapshots are useful when one needs to perform several reads orselects, ensuring isolation from concurrent writes, but without blocking them.When nothing needs to be written, using a snapshot is preferable to using atransaction, because it will not block writes.

Snapshots come at no cost: nothing is actually copied or written on disk or inmemory, apart from some small internal bookkeeping. After obtaining a snapshotwithwith_snapshot, one can read from it using the functions in theCubDB.Snapshot module:

# the key of y depends on the value of x, so we ensure consistency by getting# both entries from the same snapshot, isolating from the effects of concurrent# writes{x,y}=CubDB.with_snapshot(db,fnsnap->x=CubDB.Snapshot.get(snap,:x)y=CubDB.Snapshot.get(snap,x){x,y}end)

The functions that read multiple entries likeget_multi,select, etc. areinternally using a snapshot, so they always ensure consistency and isolationfrom concurrent writes, implementing multi version concurrency control (MVCC).

For more details, read theAPI documentation.

CubDB can be installed by adding:cubdb to your list of dependencies inmix.exs:

defdepsdo[{:cubdb,"~> 2.0.2"}]end

The file data structure used byCubDB is inspired byCouchDB. A big thanks goes to the CouchDBmaintainers for the readable codebase and extensive documentation.

Copyright 2022 Luca Ongaro

Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.