Advanced Usage¶
Remarks on Serialization¶
Before we dive deeper into the usage of TinyDB, we should stop for a momentand discuss the topic of serializtion.
TinyDB serializes all data using thePython JSON module by default.It’s great for serializing simple data types but cannot handle more complexdata types like custom classes. If you need a better serializer, you can writeyour own storage, that uses a more powerful (but also slower)library likepickle orPyYAML.
Handling Data¶
So let’s start with inserting and retrieving data from your database.
Inserting¶
As already described you can insert an element usingdb.insert(...).In case you want to insert multiple elements, you can usedb.insert_multiple(...):
>>>db.insert_multiple([{'int':1,'char':'a'},{'int':1,'char':'b'}])>>>db.insert_multiple({'int':1,'value':i}foriinrange(2))
Retrieving¶
There are several ways to retrieve data from your database. For instance youcan get the number of stored elements:
>>>len(db)3
Then of course you can usedb.search(...) as described in theGetting Startedsection. But sometimes you want to get only one matching element. Instead of using
>>>try:...result=db.search(where('value')==1)[0]...exceptIndexError:...pass
you can usedb.get(...):
>>>db.get(where('value')==1){'int': 1, 'value': 1}>>>db.get(where('value')==100)None
Caution
If multiple elements match the query, propably a random one of them willbe returned!
Often you don’t want to search for elements but only know whether they arestored in the database. In this casedb.contains(...) is your friend:
>>>db.contains(where('char')=='a')
In a similar manner you can look up the number of elements matching a query:
>>>db.count(where('int')==1)3
Recap¶
Let’s summarize the ways to handle data:
| Inserting | |
| db.insert_multiple(...) | Insert multiple elements |
| Getting data | |
| len(db) | Get the number of elements in the database |
| db.get(query) | Get one element matching the query |
| db.contains(query) | Check if the database contains a matching element |
| db.count(query) | Get the number of matching elements |
Using Element IDs¶
Internally TinyDB associates an ID with every element you insert. It’s returnedafter inserting an element:
>>>db.insert({'value':1})3>>>db.insert_multiple([{...},{...},{...}])[4, 5, 6]
In addition you can get the ID of already inserted elements usingelement.eid:
>>>el=db.get(where('value')==1)>>>el.eid3
Different TinyDB methods also work with IDs, namely:update,remove,contains andget.
>>>db.update({'value':2},eids=[1,2])>>>db.contains(eids=[1])True>>>db.remove(eids=[1,2])>>>db.get(eid=3){...}
Recap¶
Let’s sum up the way TinyDB supports working with IDs:
| Getting an element’s ID | |
| db.insert(...) | Returns the inserted element’s ID |
| db.insert_multiple(...) | Returns the inserted elements’ ID |
| element.eid | Get the ID of an element fetched from the db |
| Working with IDs | |
| db.get(eid=...) | Get the elemtent with the given ID |
| db.contains(eids=[...]) | Check if the db contains elements with one of the given IDs |
| db.update({...},eids=[...]) | Update all elements with the given IDs |
| db.remove(eids=[...]) | Remove all elements with the given IDs |
Queries¶
TinyDB lets you use a rich set of queries. In theGetting Started you’velearned about the basic comparisons (==,<,>, ...). In additionto that TinyDB enables you run logical operations on queries.
>>># Negate a query:>>>db.search(~where('int')==1)
>>># Logical AND:>>>db.search((where('int')==1)&(where('char')=='b'))[{'int': 1, 'char': 'b'}]
>>># Logical OR:>>>db.search((where('char')=='a')|(where('char')=='b'))[{'int': 1, 'char': 'a'}, {'int': 1, 'char': 'b'}]
Note
When using& or|, make sure you wrap the conditions on both sideswith parentheses or Python will mess up the comparison.
You also can search for elements where a specific key exists:
>>>db.search(where('char'))[{'int': 1, 'char': 'a'}, {'int': 1, 'char': 'b'}]
Advanced queries¶
In addition to these checks TinyDB supports checking againsta regex or a custom test function:
>>># Regex:>>>db.search(where('char').matches('[aZ]*'))[{'int': 1, 'char': 'a'}, {'int': 1, 'char': 'b'}]
>>># Custom test:>>>test_func=lambdac:c=='a'>>>db.search(where('char').test(test_func))[{'char': 'a', 'int': 1}]
Nested Queries¶
You can insert nested elements into your database:
>>>db.insert({'field':{'name':{'first_name':'John','last_name':'Doe'}}})
To search for a nested field, usewhere('field').has(...). You can applyany queries you already know to this selector:
>>>db.search(where('field').has('name'))[{'field': ...}]>>>db.search(where('field').has('name').has('last_name')=='Doe')[{'field': ...}]
You also can use lists inside of elements:
>>>db.insert({'field':[{'val':1},{'val':2},{'val':3}])
Usingwhere('field').any(...) andwhere('field').all(...) you canspecify checks for the list’s items. They behave similarly to Python’sany andall:
>>>db.search(where('field').any(where('val')==1))True>>>db.search(where('field').all(where('val')>0))True
Recap¶
Again, let’s recapitulate the query operations:
| Queries | |
| where('field').matches(regex) | Match any element matching the regular expression |
| where('field').test(func) | Matches any element for which the function returnsTrue |
| Combining Queries | |
| ~query | Match elements that don’t match the query |
| (query1)&(query2) | Match elements that match both queries |
| (query1)|(query2) | Match elements that match one of the queries |
| Nested Queries | |
| where('field').has('field') | Match any element that has the specified item. Performmore queries on this selector as needed |
| where('field').any(query) | Match any element where ‘field’ is a list where one ofthe items matches the subquery |
| where('field').all(query) | Match any element where ‘field’ is a list where all itemsmatch the subquery |
Tables¶
TinyDB supports working with multiple tables. They behave just the same astheTinyDB class. To create and use a table, usedb.table(name).
>>>table=db.table('table_name')>>>table.insert({'value':True})>>>table.all()[{'value': True}]
To remove all tables from a database, use:
>>>db.purge_tables()
Note
TinyDB uses a table named_default as default table. All operationson the database object (likedb.insert(...)) operate on this table.
You can get a list with the names of all tables in your database:
>>>db.tables(){'_default', 'table_name'}
Query Caching¶
TinyDB caches query result for performance. You can optimize the query cachesize by passing thecache_size to thetable(...) function:
>>>table=db.table('table_name',cache_size=30)
Hint
You can setcache_size toNone to make the cache unlimited insize.
Smart Query Cache¶
If you perform lots of queries while the data changes only little, you mayenable a smarter query cache. It updates the query cache wheninserting/removing/updating elements so the cache doesn’t get invalidated.
>>>table=db.table('table_name',smart_cache=True)
Storages & Middlewares¶
Storage Types¶
TinyDB comes with two storage types: JSON and in-memory. Bydefault TinyDB stores its data in JSON files so you have to specify the pathwhere to store it:
>>>fromtinydbimportTinyDB,where>>>db=TinyDB('path/to/db.json')
To use the in-memory storage, use:
>>>fromtinydb.storagesimportMemoryStorage>>>db=TinyDB(storage=MemoryStorage)
Middlewares¶
Middlewares wrap around existing storages allowing you to customize theirbehaviour.
>>>fromtinydb.storagesimportJSONStorage>>>fromtinydb.middlewaresimportCachingMiddleware>>>db=TinyDB('/path/to/db.json',storage=CachingMiddleware(JSONStorage))
Hint
You can nest middlewares:
>>>db=TinyDB('/path/to/db.json',storage=FirstMiddleware(SecondMiddleware(JSONStorage)))
CachingMiddleware¶
TheCachingMiddleware improves speed by reducing disk I/O. It caches allread operations and writes data to disk after a configured number ofwrite operations.
To make sure that all data is safely written when closing the table, use oneof these ways:
# Using a context manager:withdatabaseasdb:# Your operations
# Using the close functiondb.close()
Table Of Contents
Useful Links
Quick search
Enter search terms or a module, class or function name.