Lightweight document-oriented NoSQL Database

🇬🇧 English |🇨🇳 简体中文 |🇪🇸 Spanish

CloverDB is a lightweight NoSQL database designed for being simple and easily maintainable, thanks to its small code base. It has been inspired bytinyDB.

Features

• Document oriented
• Written in pure Golang
• Simple and intuitive api
• Easily maintainable

Why CloverDB?

CloverDB has been written for being easily maintenable. As such, it trades performance with simplicity, and is not intented to be an alternative to more performant databases such asMongoDB orMySQL.However, there are projects where running a separate database server may result overkilled, and, for simple queries, network delay may be the major performance bottleneck.For there scenario,CloverDB may be a more suitable alternative.

Database Layout

CloverDB abstracts the way collections are stored on disk through theStorageEngine interface. The default implementation is based on theBadger database key-value store. However, you could easily write your own storage engine implementation.

Installation

Make sure you have a working Go environment (Go 1.13 or higher is required).

  GO111MODULE=on go get github.com/ostafen/clover

Databases and Collections

CloverDB stores data records as JSON documents, which are grouped together in collections. A database is made up of one or more collections.

Database

To store documents inside collections, you have to open a Clover database using theOpen() function.

import ("log"c "github.com/ostafen/clover")...db, _ := c.Open("clover-db")// or, if you don't need persistencydb, _ := c.Open("", c.InMemoryMode(true))defer db.Close() // remember to close the db when you have done

Collections

CloverDB stores documents inside collections. Collections are theschemaless equivalent of tables in relational databases. A collection is created by calling theCreateCollection() function on a database instance. New documents can be inserted using theInsert() orInsertOne() methods. Each document is uniquely identified by aVersion 4 UUID stored in the_id special field and generated during insertion.

db, _ := c.Open("clover-db")db.CreateCollection("myCollection") // create a new collection named "myCollection"// insert a new document inside the collectiondoc := c.NewDocument()doc.Set("hello", "clover!")// InsertOne returns the id of the inserted documentdocId, _ := db.InsertOne("myCollection", doc)fmt.Println(docId)

Importing and Exporting Collections

CloverDB is capable of easily importing and exporting collections to JSON format regardless of the storage engine used.

// dump the content of the "todos" collection in a "todos.json" filedb.ExportCollection("todos", "todos.json")...// recover the todos collection from the exported json filedb.DropCollection("todos")db.ImportCollection("todos", "todos.json")docs, _ := db.Query("todos").FindAll()for _, doc := range docs {  log.Println(doc)}

Queries

CloverDB is equipped with a fluent and elegant API to query your data. A query is represented by theQuery object, which allows to retrieve documents matching a givencriterion. A query can be created by passing a valid collection name to theQuery() method.

Select All Documents in a Collection

TheFindAll() method is used to retrieve all documents satisfying a given query.

docs, _ := db.Query("myCollection").FindAll()todo := &struct {    Completed bool   `clover:"completed"`    Title     string `clover:"title"`    UserId    int    `clover:"userId"`}{}for _, doc := range docs {    doc.Unmarshal(todo)    log.Println(todo)}

Filter Documents with Criteria

In order to filter the documents returned byFindAll(), you have to specify a query Criteria using theWhere() method. A Criteria object simply represents a predicate on a document, evaluating totrue only if the document satisfies all the query conditions.

The following example shows how to build a simple Criteria, matching all the documents having thecompleted field equal to true.

db.Query("todos").Where(c.Field("completed").Eq(true)).FindAll()// or equivalentlydb.Query("todos").Where(c.Field("completed").IsTrue()).FindAll()

In order to build very complex queries, we chain multiple Criteria objects by using theAnd() andOr() methods, each returning a new Criteria obtained by appling the corresponding logical operator.

// find all completed todos belonging to users with id 5 and 8db.Query("todos").Where(c.Field("completed").Eq(true).And(c.Field("userId").In(5, 8))).FindAll()

Sorting Documents

To sort documents in CloverDB, you need to useSort(). It is a variadic function which accepts a sequence of SortOption, each allowing to specify a field and a sorting direction.A sorting direction can be one of 1 or -1, respectively corresponding to ascending and descending order. If no SortOption is provided,Sort() uses the_id field by default.

// Find any todo belonging to the most recent inserted userdb.Query("todos").Sort(c.SortOption{"userId", -1}).FindFirst()

Skip/Limit Documents

Sometimes, it can be useful to discard some documents from the output, or simply set a limit on the maximum number of results returned by a query. For this purpose, CloverDB provides theSkip() andLimit() functions, both accepting an interger $n$ as parameter.

// discard the first 10 documents from the output,// also limiting the maximum number of query results to 100db.Query("todos").Skip(10).Limit(100).FindAll()

Update/Delete Documents

TheUpdate() method is used to modify specific fields of documents in a collection. TheDelete() method is used to delete documents. Both methods belong to the Query object, so that it is easy to update and delete documents matching a particular query.

// mark all todos belonging to user with id 1 as completedupdates := make(map[string]interface{})updates["completed"] = truedb.Query("todos").Where(c.Field("userId").Eq(1)).Update(updates)// delete all todos belonging to users with id 5 and 8db.Query("todos").Where(c.Field("userId").In(5,8)).Delete()

To update or delete a single document using a specific document id, useUpdateById() orDeleteById(), respectively:

docId := "1dbce353-d3c6-43b3-b5a8-80d8d876389b"// update the document with the specified iddb.Query("todos").UpdateById(docId, map[string]interface{}{"completed": true})// or delete itdb.Query("todos").DeleteById(docId)

Data Types

Internally, CloverDB supports the following primitive data types:int64,uint64,float64,string,bool andtime.Time. When possible, values having different types are silently converted to one of the internal types: signed integer values get converted to int64, while unsigned ones to uint64. Float32 values are extended to float64.

For example, consider the following snippet, which sets an uint8 value on a given document field:

doc := c.NewDocument()doc.Set("myField", uint8(10)) // "myField" is automatically promoted to uint64fmt.Println(doc.Get("myField").(uint64))

Pointer values are dereferenced until eithernil or anon-pointer value is found:

var x int = 10var ptr *int = &xvar ptr1 **int = &ptrdoc.Set("ptr", ptr)doc.Set("ptr1", ptr1)fmt.Println(doc.Get("ptr").(int64) == 10)fmt.Println(doc.Get("ptr1").(int64) == 10)ptr = nildoc.Set("ptr1", ptr1)// ptr1 is not nil, but it points to the nil "ptr" pointer, so the field is set to nilfmt.Println(doc.Get("ptr1") == nil)

Invalid types leaves the document untouched:

doc := c.NewDocument()doc.Set("myField", make(chan struct{}))log.Println(doc.Has("myField")) // will output false

Contributing

CloverDB is actively developed. Any contribution, in the form of a suggestion, bug report or pull request, is well accepted 😊

Major contributions and suggestions have been gratefully received from (in alphabetical order):

• ASWLaunchs
• jsgm
• segfault99

Constants¶

Variables¶

var (ErrCollectionExist    =errors.New("collection already exist")ErrCollectionNotExist =errors.New("no such collection"))

var ErrDocumentNotExist =errors.New("no such document")

var ErrDuplicateKey =errors.New("duplicate key")

Functions¶

func Field(namestring) *field

func NewObjectId()string

Types¶

type Config struct {InMemoryboolStorageStorageEngine}

type Criteria struct {// contains filtered or unexported fields}

func (c *Criteria) And(other *Criteria) *Criteria

func (c *Criteria) Not() *Criteria

func (c *Criteria) Or(other *Criteria) *Criteria

type DB struct {// contains filtered or unexported fields}

func Open(dirstring, opts ...Option) (*DB,error)

func (db *DB) Close()error

func (db *DB) CreateCollection(namestring)error

func (db *DB) DropCollection(namestring)error

func (db *DB) ExportCollection(collectionNamestring, exportPathstring)error

func (db *DB) HasCollection(namestring) (bool,error)

func (db *DB) ImportCollection(collectionNamestring, importPathstring)error

func (db *DB) Insert(collectionNamestring, docs ...*Document)error

func (db *DB) InsertOne(collectionNamestring, doc *Document) (string,error)

func (db *DB) ListCollections() ([]string,error)

func (db *DB) Query(namestring) *Query

func (db *DB) Save(collectionNamestring, doc *Document)error

type Document struct {// contains filtered or unexported fields}

func NewDocument() *Document

func NewDocumentOf(o interface{}) *Document

func (doc *Document) Copy() *Document

func (doc *Document) Get(namestring) interface{}

func (doc *Document) Has(namestring)bool

func (doc *Document) ObjectId()string

func (doc *Document) Set(namestring, value interface{})

func (doc *Document) SetAll(values map[string]interface{})

func (doc *Document) Unmarshal(v interface{})error

type Option func(c *Config)error

func InMemoryMode(enablebool)Option

func WithStorageEngine(engineStorageEngine)Option

type Query struct {// contains filtered or unexported fields}

func (q *Query) Count() (int,error)

func (q *Query) Delete()error

func (q *Query) DeleteById(idstring)error

func (q *Query) Exists() (bool,error)

func (q *Query) FindAll() ([]*Document,error)

func (q *Query) FindById(idstring) (*Document,error)

func (q *Query) FindFirst() (*Document,error)

func (q *Query) ForEach(consumer func(_ *Document)bool)error

func (q *Query) Limit(nint) *Query

func (q *Query) MatchPredicate(p func(doc *Document)bool) *Query

func (q *Query) ReplaceById(docIdstring, doc *Document)error

func (q *Query) Skip(nint) *Query

func (q *Query) Sort(opts ...SortOption) *Query

func (q *Query) Update(updateMap map[string]interface{})error

func (q *Query) UpdateById(docIdstring, updateMap map[string]interface{})error

func (q *Query) Where(c *Criteria) *Query

type SortOption struct {FieldstringDirectionint}

type StorageEngine interface {Open(pathstring)errorClose()errorCreateCollection(namestring)errorListCollections() ([]string,error)DropCollection(namestring)errorHasCollection(namestring) (bool,error)FindAll(q *Query) ([]*Document,error)FindById(collectionNamestring, idstring) (*Document,error)UpdateById(collectionNamestring, docIdstring, updater func(doc *Document) *Document)errorDeleteById(collectionNamestring, idstring)errorIterateDocs(q *Query, consumer docConsumer)errorInsert(collectionstring, docs ...*Document)errorUpdate(q *Query, updater func(doc *Document) *Document)errorDelete(q *Query)error}