Repository files navigation

Fast Python bindings forUnQLite, a lightweight, embedded NoSQL database and JSON document store.

Read the issue tracker forthis databasebefore considering using it. UnQLite has not seen any meaningful developmentsince 2014. It isstrongly recommended that you useSqlite.Sqlite has robust support forjson and isactively developed and maintained.

UnQLite features:

• Embedded, zero-conf database
• Transactional (ACID)
• Single file or in-memory database
• Key/value store
• Cursor support and linear record traversal
• JSON document store
• Thread-safe
• Terabyte-sized databases

UnQLite-Python features:

• Compiled library, extremely fast with minimal overhead.
• Supports key/value operations, cursors, and transactions using Pythonic APIs.
• Support for Jx9 scripting.
• APIs for working with Jx9 JSON document collections.

Links:

• unqlite-python documentation
• UnQLite's C API

You can install unqlite usingpip.

pip install unqlite

Below is a sample interactive console session designed to show some of the basic features and functionality of the unqlite-python library. Also check out thefull API documentation.

To begin, instantiate anUnQLite object. You can specify either the path to a database file, or use UnQLite as an in-memory database.

>>>from unqliteimport UnQLite>>> db= UnQLite()# Create an in-memory database.

UnQLite can be used as a key/value store.

>>> db['foo']='bar'# Use as a key/value store.>>> db['foo']# The key/value store deals in byte-strings.b'bar'>>>for iinrange(4):...     db['k%s'% i]=str(i)...>>>'k3'in dbTrue>>>'k4'in dbFalse>>>del db['k3']>>> db.append('k2','XXXX')>>> db['k2']b'2XXXX'

The database can also be iterated through directly. Note that keys are decodedwhile values are left as bytestrings.

>>> [itemfor itemin db][('foo', b'bar'), ('k0', b'0'), ('k1', b'1'), ('k2', b'2XXXX')]

For finer-grained record traversal, you can use cursors.

>>>with db.cursor()as cursor:...     cursor.seek('k0')...for key, valuein cursor:...print(key,'=>', value.decode('utf8'))...k0 => 0k1 => 1k2 => 2XXXX>>>with db.cursor()as cursor:...     cursor.seek('k2')...print(cursor.value())...b'2XXXX'>>>with db.cursor()as cursor:...     cursor.seek('k0')...print(list(cursor.fetch_until('k2',include_stop_key=False)))...[('k0', b'0'), ('k1', b'1')]

There are many different ways of interacting with cursors, which are described in theCursor API documentation.

In my opinion the most interesting feature of UnQLite is its JSON document store. TheJx9 scripting language is used to interact with the document store, and it is a wacky mix of PHP and maybe JavaScript (?).

Note: as of v0.8.0 the document store and collections APIs treat allstrings as unicode.

Interacting with the document store basically consists of creating a Jx9 script (you might think of it as an imperative SQL query), compiling it, and then executing it.

>>> script="""...     db_create('users');...     db_store('users',$list_of_users);...$users_from_db= db_fetch_all('users');...""">>> list_of_users= [...     {'username':'Huey','age':3},...     {'username':'Mickey','age':5}... ]>>>with db.vm(script)as vm:...     vm['list_of_users']= list_of_users...     vm.execute()...     users_from_db= vm['users_from_db']...True>>> users_from_db# UnQLite assigns items in a collection an ID.[{'username': 'Huey', 'age': 3, '__id': 0}, {'username': 'Mickey', 'age': 5, '__id': 1}]

This is just a taste of what is possible with Jx9. In the near future I may add some wrappers around common Jx9 collection operations, but for now hopefully it is not too difficult to work with.

More information can be found in theVM API documentation.

To simplify working with JSON document collections,unqlite-python provides a light API forexecuting Jx9 queries on collections. A collection is an ordered list of JSON objects(records). Records can be appended or deleted, and in the next major release of UnQLite there willbe support for updates as well.

To begin working with collections, you can use the factory method onUnQLite:

>>> users= db.collection('users')>>> users.create()# Create the collection if it does not exist.>>> users.exists()True

You can use thestore() method to add one or many records. To add a single record just pass in a pythondict. To add multiple records, pass in a list of dicts. Records can be fetched and deleted by ID.

By default, the ID of the last-stored record is returned. At the time ofwriting, IDs start at 0, so when inserting 3 records the last-id is 2:

>>> users.store([...     {'name':'Charlie','color':'green'},...     {'name':'Huey','color':'white'},...     {'name':'Mickey','color':'black'}])2>>> users.store({'name':'Leslie','color':'also green'},return_id=False)True>>> users.fetch(0)# Fetch the first record.{'__id': 0, 'color': 'green', 'name': 'Charlie'}>>> users.delete(0)# Delete the first record.True>>> users.delete(users.last_record_id())# Delete the last record.True

You can retrieve all records in the collection, or specify a filtering function. The filtering function will be registered as a foreign function with the Jx9 VM and calledfrom the VM.

>>> users.all()[{'__id': 1, 'color': 'white', 'name': 'Huey'}, {'__id': 2, 'color': 'black', 'name': 'Mickey'}]>>> users.filter(lambdaobj: obj['name'].startswith('H'))[{'__id': 1, 'color': 'white', 'name': 'Huey'}]

UnQLite supports transactions for file-backed databases (since transactions occur at the filesystem level, they have no effect on in-memory databases).

The easiest way to create a transaction is with the context manager:

>>> db= UnQLite('/tmp/test.db')>>>with db.transaction():...     db['k1']='v1'...     db['k2']='v2'...>>> db['k1']b'v1'

You can also use the transaction decorator which will wrap a function call in a transaction and commit upon successful execution (rolling back if an exception occurs).

>>>@db.commit_on_success...defsave_value(key,value,exc=False):...     db[key]= value...if exc:...raiseException('uh-oh')...>>> save_value('k3','v3')>>> save_value('k3','vx',True)Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "unqlite/core.py", line 312, in wrapper    return fn(*args, **kwargs)  File "<stdin>", line 5, in save_valueException: uh-oh>>> db['k3']b'v3'

For finer-grained control you can calldb.begin(),db.rollback() anddb.commit() manually:

>>> db.begin()>>> db['k3']='v3-xx'>>> db.commit()True>>> db['k3']b'v3-xx'

This code is based in part onbuaabyl's pyUnQLite.