Write a Custom Storage¶

First, we have support for custom storages. By default TinyDB comes with anin-memory storage and a JSON file storage. But of course you can add your own.Let’s look how you could add aYAML storage usingPyYAML:

importyamlclassYAMLStorage(Storage):def__init__(self,filename):# (1)self.filename=filenamedefread(self):withopen(self.filename)ashandle:try:data=yaml.safe_load(handle.read())# (2)returndataexceptyaml.YAMLError:returnNone# (3)defwrite(self,data):withopen(self.filename,'w+')ashandle:yaml.dump(data,handle)defclose(self):# (4)pass

There are some things we should look closer at:

1. The constructor will receive all arguments passed to TinyDB when creatingthe database instance (exceptstorage which TinyDB itself consumes).In other words callingTinyDB('something',storage=YAMLStorage) willpass'something' as an argument toYAMLStorage.

2. We useyaml.safe_load as recommended by thePyYAML documentationwhen processing data from a potentially untrusted source.

3. If the storage is uninitialized, TinyDB expects the storage to returnNone so it can do any internal initialization that is necessary.

4. If your storage needs any cleanup (like closing file handles) before aninstance is destroyed, you can put it in theclose() method. To runthese, you’ll either have to rundb.close() on yourTinyDB instanceor use it as a context manager, like this:

   withTinyDB('db.yml',storage=YAMLStorage)asdb:# ...

Finally, using the YAML storage is very straight-forward:

db=TinyDB('db.yml',storage=YAMLStorage)# ...

Write Custom Middleware¶

Sometimes you don’t want to write a new storage module but rather modify thebehaviour of an existing one. As an example we’ll build middleware that filtersout empty items.

Because middleware acts as a wrapper around a storage, they needs aread()and awrite(data) method. In addition, they can access the underlying storageviaself.storage. Before we start implementing we should look at the structureof the data that the middleware receives. Here’s what the data that goes throughthe middleware looks like:

{'_default':{1:{'key':'value'},2:{'key':'value'},# other items},# other tables}

Thus, we’ll need two nested loops:

1. Process every table
2. Process every item

Now let’s implement that:

classRemoveEmptyItemsMiddleware(Middleware):def__init__(self,storage_cls):# Any middleware *has* to call the super constructor# with storage_clssuper().__init__(storage_cls)# (1)defread(self):data=self.storage.read()fortable_nameindata:table_data=data[table_name]fordoc_idintable_data:item=table_data[doc_id]ifitem=={}:deltable_data[doc_id]returndatadefwrite(self,data):fortable_nameindata:table_data=data[table_name]fordoc_idintable:item=table_data[doc_id]ifitem=={}:deltable_data[doc_id]self.storage.write(data)defclose(self):self.storage.close()

Note that the constructor calls the middleware constructor (1) and passesthe storage class to the middleware constructor.

To wrap storage with this new middleware, we use it like this:

db=TinyDB(storage=RemoveEmptyItemsMiddleware(SomeStorageClass))

HereSomeStorageClass should be replaced with the storage you want to use.If you leave it empty, the default storage will be used (which is theJSONStorage).

Use hooks and overrides¶

There are cases when neither creating a custom storage nor using a custommiddleware will allow you to adapt TinyDB in the way you need. In this caseyou can modify TinyDB’s behavior by using predefined hooks and override points.For example you can configure the name of the default table by settingTinyDB.default_table_name:

TinyDB.default_table_name='my_table_name'

BothTinyDB and theTableclasses allow modifying their behavior using hooks and overrides. To useTable’s overrides, you can access the class usingTinyDB.table_class:

TinyDB.table_class.default_query_cache_capacity=100

Read theAPI Documentation for more details on the available hooks and overridepoints.

SubclassingTinyDB andTable¶

Finally, there’s the last option to modify TinyDB’s behavior. That way youcan change how TinyDB itself works more deeply than using the other extensionmechanisms.

When creating a subclass you can use it by using hooks and overrides to overridethe default classes that TinyDB uses:

classMyTable(Table):# Add your method overrides...TinyDB.table_class=MyTable# Continue using TinyDB as usual

TinyDB’s source code is documented with extensions in mind, explaining howeverything works even for internal methods and classes. Feel free to dig intothe source and adapt everything you need for your projects.